> Markdown version of [/jobs/ext/4588329-data-engineer-with-databricks](https://www.wearedevelopers.com/jobs/ext/4588329-data-engineer-with-databricks). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Data Engineer with Databricks

- **Company:** Quantum Technologies
- **Location:** Albany, NY, United States
- **Experience:** Expert
- **Salary:** $180,960
- **Contract:** Contract
- **Skills:** Application Programming Interfaces (APIs), Amazon Web Services, Microsoft Azure, Big Data, Cloud Database, Cloud Storage, Databases, Continuous Integration, Data Architecture, Information Engineering, Data Governance, Extract Transform Load (ETL), Data Transformation, Python (Programming Language), Operational Databases, Performance Tuning, Software Deployment, SQL Databases, Data Processing, Freeform SQL, Google Cloud, Cloud Platform System, Sql Optimization, Apache Spark, Git, Data Lakes, Pyspark, Software Version Control, Data Pipelines, Databricks
- **Published:** November 5, 2026
- **Apply:** https://www.dice.com/job-detail/074682ea-784b-4beb-b32c-83520195de32

## About the Role

The candidate should be a hands-on Data Engineer who has personally built and supported Databricks solutions, rather than someone with only theoretical exposure. They should be able to explain real examples of PySpark transformations, ETL/ELT pipelines, Delta Lake implementations, performance optimization, data-quality controls, and production troubleshooting., * 5 8 years of overall Data Engineering experience.

 * Strong hands-on experience with Databricks.
 * Strong hands-on development experience using PySpark / Apache Spark.
 * Strong Python programming skills and advanced SQL skills.
 * Experience developing production-grade ETL/ELT pipelines.
 * Experience with Delta Lake and Lakehouse concepts.
 * Experience processing large-volume and complex datasets.
 * Good understanding of data modeling, data quality, and data governance principles.
 * Strong troubleshooting and performance-tuning skills.
 * Experience with at least one cloud platform: Microsoft Azure, AWS, or Google Cloud.
 * Experience with Git and CI/CD practices.

## Description

We are looking for an experienced Data Engineer with strong hands-on expertise in Databricks, PySpark, Python, and SQL. The candidate will be responsible for designing, developing, and optimizing scalable data pipelines and data-processing solutions in a cloud-based data environment., * Design and develop scalable ETL/ELT data pipelines using Databricks.

 * Develop data-processing and transformation solutions using PySpark and Python.
 * Write complex SQL queries for data transformation, validation, and analysis.
 * Build and maintain solutions using Delta Lake and Lakehouse architecture.
 * Integrate data from multiple sources, including databases, APIs, files, and cloud storage.
 * Optimize Spark jobs, Databricks workloads, and SQL queries for performance and cost efficiency.
 * Implement data-quality, validation, reconciliation, monitoring, and error-handling mechanisms.
 * Troubleshoot production data issues and perform root-cause analysis.
 * Collaborate with business analysts, architects, and application teams to translate business requirements into technical solutions.
 * Follow engineering best practices for testing, source control, CI/CD, documentation, and production deployment.