> Markdown version of [/jobs/ext/3388685-databricks-data-engineer](https://www.wearedevelopers.com/jobs/ext/3388685-databricks-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # DataBricks Data Engineer - **Company:** Prodapt Corp - **Location:** Irving, TX, United States - **Experience:** Experienced - **Contract:** Permanent contract - **Skills:** Application Programming Interfaces (APIs), Microsoft Azure, Code Review, Computer Programming, Continuous Integration, Data Architecture, Information Engineering, Extract Transform Load (ETL), DevOps, Apache Hive, Python (Programming Language), Key Management, NoSQL, Object-Oriented Software Development, Performance Tuning, Azure Data Lake, SQL Databases, Systems Integration, Cloud Platform System, Azure Data Factory, ReactJS, Snowflake, Apache Spark, Caching, Git, Build Management, Data Lakes, Pyspark, Information Technology, Deployment Automation, Terraform, Azure Synapse Analytics, Software Version Control, Data Pipelines, Databricks - **Published:** September 3, 2026 - **Apply:** https://www.indeed.com/viewjob?jk=8575f37d6f1acd78 ## About the Role 10+ years of overall Data Engineering experience, including designing and implementing ETL/ELT pipelines with Azure Data Factory and other Azure services. Bachelor's degree in Computer Science, Engineering, or a related field; OR equivalent combination of education and relevant experience. Strong hands-on experience with Azure Databricks; 4+ years of experience across Azure services and Databricks (ADLS, ADF, Azure DevOps, etc.). 7+ years of Python development experience, with the ability to design and build reusable libraries. 4+ years of experience with Snowflake or SQL (No-SQL experience is a plus). Expert-level knowledge of PySpark and Spark SQL. Strong programming experience in Python and SQL. Experience with Delta Lake and Lakehouse architecture. Strong experience with Azure Data Lake Storage (ADLS Gen2). Experience with Azure Data Factory (ADF) and other Azure data services. Strong understanding of ETL/ELT, data modeling, and large-scale data pipelines. Experience with performance tuning and optimization in Databricks/Spark. Experience with Git, CI/CD, and DevOps practices. Preferred Skills Databricks certifications. Experience with Unity Catalog and Databricks governance/security. Experience with Terraform or Infrastructure as Code. Knowledge of Azure DevOps. Experience designing APIs and integrating with React JS within a cloud platform ## Description Design, develop, and implement scalable data pipelines using Azure Databricks. Build and optimize ETL/ELT pipelines using PySpark, Python, and SQL. Develop data solutions using the Medallion Architecture (Bronze, Silver, and Gold layers). Work with Delta Lake, Delta tables, and advanced Databricks optimization techniques. Integrate Databricks with Azure services such as ADLS Gen2, Azure Data Factory, Azure Synapse, and Azure Key Vault. Develop and manage Databricks Workflows, Jobs, and Notebooks. Implement data quality, monitoring, error handling, and performance optimization. Collaborate with Data Architects, Data Engineers, Data Scientists, and business stakeholders. Establish best practices for CI/CD, Git/version control, code reviews, and automated deployments. Lead technical discussions and provide guidance to junior engineers. Analyze existing Python OOP applications and redesign single-node processing logic for distributed Spark execution. Design, develop, and deploy enterprise-scale data pipelines on Azure Databricks; build reusable PySpark frameworks and utility modules. Implement Delta Lake solutions using the Bronze, Silver, Gold architecture. Build robust ETL/ELT pipelines with Azure Data Factory, ADLS Gen2, and Azure Synapse Analytics. Implement data quality, reconciliation, validation, and monitoring frameworks. Optimize Spark jobs using partitioning, bucketing, caching, broadcast joins, Adaptive Query Execution, and Delta optimization. ## Related Videos - [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94) - [Leveraging Real time data in FSIs](https://www.wearedevelopers.com/videos/806-leveraging-real-time-data-in-fsis) - [Shifting Stress to Progress— Understanding DevOps to do DevOps Better](https://www.wearedevelopers.com/videos/268-shifting-stress-to-progress-understanding-devops-to-do-devops-better) - [Fully Orchestrating Databricks from Airflow](https://www.wearedevelopers.com/videos/336-fully-orchestrating-databricks-from-airflow) - [Git for Code Reviews](https://www.wearedevelopers.com/videos/429-git-for-code-reviews) - [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt) ## Related Articles - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [What Are The Top Skills Required For Azure Developers?](https://www.wearedevelopers.com/magazine/77-what-are-the-top-skills-required-for-azure-developers) - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Navigating the AI Shift](https://www.wearedevelopers.com/magazine/629-navigating-the-ai-shift)