Urgent Hiring for the Senior Databricks Engineer || 100 % Remote Role

BuzzClan
Dallas, TX, United States
24 days ago
Apply on www.careerjet.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
3 years minimum
Working hours
Regular working hours

Tech stack

Amazon Web Services Amazon S3 Code Review Continuous Integration Apache Hive Performance Tuning Reverse Engineering Software Engineering Data Streaming Data Processing Apache Spark Git
+11 more
Data Lakes Pyspark Infrastructure Automation Frameworks Data Lineage Deployment Automation AWS Glue AWS Data Analytics Terraform Data Pipelines Amazon Redshift Databricks

Job description

Seeking an experienced Senior Databricks Engineer to support a greenfield Databricks implementation. The role involves rebuilding and optimizing existing AWS data pipelines in Databricks, developing scalable batch and streaming workflows, and ensuring data quality, governance, performance, and reliability. Key Responsibilities Design, develop, and deploy scalable Databricks data pipelines and transformation workflows. Reverse engineer existing AWS Glue, S3, Redshift, and related data processing solutions. Build and maintain Bronze, Silver, and Gold layers using Medallion Architecture. Develop ingestion and streaming pipelines using Auto Loader, Lakeflow Declarative Pipelines, Structured Streaming, PySpark, Spark SQL, and Delta Lake. Implement Unity Catalog governance, access controls, data lineage, and security. Optimize Spark workloads, Delta Lake tables, queries, and pipeline performance. Develop Databricks Workflows, CI/CD pipelines, and automated deployment solutions using Git, Databricks Asset Bundles, Terraform, or similar tools. Implement data quality checks, error handling, monitoring, testing, and pipeline recovery. Validate data parity between legacy AWS systems and new Databricks pipelines. Troubleshoot production issues, document solutions, conduct code reviews, and mentor client engineers.

Requirements

6+ years of data or software engineering experience. 3+ years of hands-on Databricks experience in production environments. Strong expertise in Databricks Auto Loader, Lakeflow Declarative Pipelines, Unity Catalog, Delta Lake, Apache Spark, PySpark, and Spark SQL. Experience with Medallion Architecture, batch and streaming processing, and high-volume data pipelines. Strong knowledge of Spark performance tuning, data quality, governance, monitoring, and production troubleshooting. Experience with AWS services, including S3, Glue, and Redshift. Hands-on experience with Databricks Workflows, CI/CD, Git, automated deployments, and infrastructure-as-code tools. Strong analytical, problem-solving, communication, and documentation skills.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.careerjet.com
Prepare application

Good distractions

Loading talks and stories from around this role…