> Markdown version of [/jobs/ext/2027154-data-engineer-databricks](https://www.wearedevelopers.com/jobs/ext/2027154-data-engineer-databricks). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Data Engineer - Databricks - **Company:** Resultant, LLC - **Location:** Chicago, IL, United States - **Experience:** Experienced - **Contract:** Permanent contract - **Skills:** Third Normal Form, Artificial Intelligence, Airflow, Amazon Web Services, Microsoft Azure, Big Data, Continuous Integration, Data as a Services, Data Validation, Information Engineering, Data Governance, Extract Transform Load (ETL), Dimensional Modeling, Github, Apache Hive, PostgreSQL, Microsoft SQL Server, Oracle (Applications), Power BI, Standard Sql, Search Technologies, Data Streaming, Systems Integration, Tableau (Software), Azure Service Bus, Cloud Platform System, Azure Data Factory, Snowflake, Code Comments, Gitlab, Containerization, Data Lakes, Pyspark, Git Flow, Kubernetes, Information Technology, Apache Kafka, Data Management, Machine Learning Operations, Terraform, Data Pipelines, Serverless Computing, Docker, Databricks - **Published:** August 11, 2026 - **Apply:** https://jobs.smartrecruiters.com/Resultant/744000142737599-data-engineer-databricks ## About the Role * Bachelor's degree in Computer Science, Engineering, Data Science, or a related field (or equivalent practical experience) * 2+ years of hands-on data engineering experience, including production work on the Databricks platform * Strong hands-on experience with PySpark and Spark SQL * Practical experience with Delta Lake fundamentals - ACID transactions, OPTIMIZE/Z-ORDER, partitioning, and schema evolution * Solid SQL skills across relational platforms (SQL Server, Postgres, Oracle, Snowflake, etc.) * Experience with at least one major cloud platform (Azure, AWS, or GCP) and its data services * Working knowledge of data modeling (dimensional modeling, 3NF) and ETL/ELT design principles * Strong communication skills and comfort working directly with clients and non-technical stakeholders * A collaborative, detail-oriented mindset with a bias toward solution quality and follow-through Preferred / Nice-to-Have * Databricks Certified Data Engineer Associate or Professional * Experience with Unity Catalog, Delta Live Tables, and Auto Loader in production environments * Exposure to MLflow, Feature Store, or Databricks Vector Search for AI/ML-enabled use cases * Experience with Databricks Asset Bundles and CI/CD tooling (GitHub Actions, Azure DevOps, GitLab) * Familiarity with Terraform or other infrastructure-as-code tooling * Experience with dbt, Kafka/Event Hubs, or BI tools (Power BI, Tableau) connected to Databricks * Docker/Kubernetes experience for containerized workloads * Prior consulting experience, or comfort moving across multiple client engagements and industries ## Description We're looking for a Data Engineer to join our Databricks practice and help design, build, and optimize Lakehouse-based data platforms for clients across industries - from public sector agencies to healthcare, financial services, and manufacturing. You'll work hands-on with the Databricks Data Intelligence Platform to turn messy, disconnected client data into governed, trustworthy, analytics- and AI-ready assets. This is a client-facing consulting role. You'll partner with solution architects, data scientists, and project leads to gather requirements, design pipelines, and deliver production-grade solutions - then explain what you built and why it matters in language business stakeholders actually understand. What You'll Do * Design, build, and optimize scalable ETL/ELT pipelines on Databricks using PySpark, Spark SQL, and Delta Lake * Implement Medallion (Bronze/Silver/Gold) architecture patterns, applying data quality checks, schema evolution, and enforcement along the way * Build declarative pipelines with Delta Live Tables (DLT) and ingest streaming/incremental data using Auto Loader and Structured Streaming * Orchestrate and monitor production workloads using Databricks Workflows, integrating with tools like Airflow or Azure Data Factory where needed * Configure and maintain Unity Catalog for data governance - catalogs, schemas, access controls, lineage, and PII masking * Partner with data scientists to prepare feature-engineered, ML-ready datasets and support model deployment workflows using MLflow * Tune cluster configuration, job design, and Photon/serverless compute for performance and cost efficiency * Build and maintain CI/CD pipelines for Databricks notebooks, jobs, and asset bundles (Git-based workflows, Azure DevOps, GitHub Actions, or similar) * Query, profile, and assess the quality of large, complex datasets from a wide variety of source systems * Collaborate with solution leads, architects, and project managers on solution design and technical architecture decisions * Participate directly in client-facing work: requirements gathering, solution reviews, and translating technical tradeoffs into plain-language business impact * Document solutions clearly - architecture diagrams, data flow documentation, code comments, and runbooks ## Related Videos - [Docker Compose: Rediscovered](https://www.wearedevelopers.com/videos/1978-docker-compose-rediscovered) - [WeAreDevelopers LIVE - Modern DevOps for IoT Devices and More](https://www.wearedevelopers.com/videos/1805-wearedevelopers-live-modern-devops-for-iot-devices-and-more) - [Innovating Developer Tools with AI: Insights from GitHub Next](https://www.wearedevelopers.com/videos/1268-innovating-developer-tools-with-ai-insights-from-github-next) - [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt) - [Docker build without Docker](https://www.wearedevelopers.com/videos/100114-docker-build-without-docker) - [Bringing AI Model Testing and Prompt Management to Your Codebase with GitHub Models](https://www.wearedevelopers.com/videos/1536-bringing-ai-model-testing-and-prompt-management-to-your-codebase-with-github-models) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production)