Databricks Engineer

• Design, build and maintain scalable ETL/ELT pipelines using Databricks, PySpark and SQL.
• Develop data ingestion and transformation frameworks supporting large-volume structured and semi-structured datasets.
• Implement Delta Lake patterns, data-quality validations, exception handling and reconciliation processes.
• Optimize Spark workloads and SQL queries for performance and cost efficiency.
• Collaborate with analysts, architects and source-system teams to deliver reliable curated data products.

Minimum Bachelors of Engineering Degree.

5+ years of experience with Databricks, PySpark, SQL, Delta Lake, ETL/ELT, and large-scale data processing.

5+ years of experience with Unity Catalog, Azure/AWS, ADF, Databricks Workflows, CI/CD, and Performance Optimization.

4 - 6 years of overall Data Engineering experience.

Must be located in the United States (Remote).

PREFERRED QUALIFICATIONS

Prior industry experience working on Health Payer projects.

skills:

AWS,Spark,AI,large-scale data processing,CI/CD,data ingestion,Delta Lake,ELT pipelines,Databricks,ETL,Data Engineering,Azure,Performance Optimization,PySpark,SQL,SQL queries,reconciliation processes,data products,Managed Services,exception handling,cost efficiency,Workflows