Senior Databricks Developer - AWS

We are seeking an experienced Senior Databricks Developer with strong hands-on expertise in Databricks, AWS, PySpark, and modern data engineering technologies. The ideal candidate will have proven experience developing, optimizing, and supporting enterprise-scale data pipelines in production environments., * Design, develop, and maintain scalable data pipelines using Databricks, PySpark, and Spark SQL.

• Build and optimize data processing solutions on AWS.
• Implement Delta Lake and Lakehouse architecture for enterprise data platforms.
• Develop and maintain Lakeflow pipelines, Delta Live Tables (DLT), and streaming workflows.
• Perform Spark performance tuning, troubleshooting, and cost optimization.
• Work with large-scale datasets and distributed processing frameworks.
• Implement data quality checks, monitoring, and production support processes.
• Support CI/CD pipelines and automated deployment of data engineering solutions.
• Collaborate with cross-functional teams to deliver reliable and scalable data platforms.

• Strong hands-on experience with Databricks development.
• Expertise in AWS data services, including S3, Glue, Lambda, and related services.
• Advanced knowledge of PySpark, Spark SQL, and Apache Spark.
• Strong experience with Delta Lake and Lakehouse architecture.
• Experience developing production-grade data pipelines.
• Knowledge of Lakeflow, Delta Live Tables (DLT), and streaming data pipelines.
• Strong debugging, performance tuning, and problem-solving skills.

Preferred Qualifications

• Experience with Scala and advanced Spark optimization.
• Knowledge of Databricks MLOps and model deployment workflows.
• Understanding of Data Mesh architecture and data governance.
• Experience with Lakebase.
• Financial services or banking domain experience is a plus.

Important Candidate Requirements

• Must be available to work onsite five days a week in Wilmington, DE or Columbus, OH.
• Strong hands-on Databricks development experience is mandatory.
• Generic Data Engineer profiles without substantial Databricks development experience will not be considered.
• Pure administration or architecture-only profiles are not suitable for this position.
• Candidates should have experience building and supporting production data pipelines.