Senior Data Engineer

Satsyil Corp is seeking a highly skilled and motivated Senior Data Engineer / Databricks Specialist to join our team. In this role, you will play a crucial part in designing, developing, and optimizing high-performance Spark applications and managing enterprise data infrastructure within AWS Databricks. Utilizing PySpark, Python, SQL, and Delta Lake, you will architect robust ETL/ELT pipelines, maintain medallion architecture, and establish efficient big data solutions supporting mission-critical public sector and enterprise environments. Key Roles & Responsibilities · Develop and optimize scalable Spark applications in AWS Databricks using Python, PySpark, and SQL to support data processing and analytics needs. · Design, build, and maintain robust ETL/ELT pipelines and implement Medallion Architecture (Bronze, Silver, Gold layers) ensuring data integrity, completeness, and lineage. · Administer Databricks Workspaces, including cluster provisioning, job scheduling, access controls, Unity Catalog, and notebook management. · Manage AWS cloud infrastructure assets including S3, Lambda, IAM, Aurora, Glue, and provision environments using Terraform Infrastructure-as-Code (IaC). · Perform query optimization, performance tuning, troubleshooting, and debugging for complex Spark jobs and large-scale data solutions. · Merge, consolidate, and process structured, semi-structured, and unstructured data sets from diverse API endpoints and databases. · Embed security gates and compliance controls within DevSecOps CI/CD pipelines (Jenkins, GitLab CI) to support FISMA/FedRAMP requirements, ATO reassessments, and POA&M resolution. · Implement rigorous data quality controls, anomaly detection, validation rules, and automated lineage tracking across all pipelines. · Collaborate within cross-functional Agile/Scrum teams in multi-contractor federal environments, providing technical leadership and production on-call support.

· Experience: 5+ years of dedicated hands-on experience in Databricks development. · Core Big Data Stack: Production proficiency with PySpark, Spark SQL, Apache Spark, and Delta Lake. · Cloud Ecosystem: Strong experience in AWS Cloud services including S3, IAM, Lambda, Glue, and Aurora. · Automation & DevOps: Production experience with Terraform (IaC) and CI/CD tools (Jenkins, GitLab CI, or equivalent). · Languages & Frameworks: Deep proficiency in Python and SQL; working knowledge of REST APIs for ingestion. · Clearance & Citizenship: U.S. Citizenship required; must have the ability to obtain and maintain a Public Trust clearance. · Methodology & Education: Hands-on experience in Agile/Scrum delivery environments. Bachelor’s degree in CS, IT, Data Science, or related engineering discipline.

• Preferred Qualifications & Nice-to-Haves: · Hands-on experience with Unity Catalog, Delta Live Tables (DLT), and AI/ML pipeline support (MLflow, Model Registry). · Experience in Federal Government contracting and compliance frameworks (FISMA / FedRAMP environments). · Active security clearance (Public Trust or higher).

1. Preferred Certifications Candidates possessing any of the following certifications will be prioritized: · Databricks Certified Data Engineer Associate · Databricks Certified Data Engineer Professional · Databricks Certified Machine Learning Associate · AWS Certified Solutions Architect - Associate or Professional · AWS Certified Data Analytics - Specialty