> Markdown version of [/jobs/ext/3954339-databricks-specialist](https://www.wearedevelopers.com/jobs/ext/3954339-databricks-specialist). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# DataBricks Specialist

- **Company:** Q1 Technologies, Inc.
- **Location:** New York, NY, United States
- **Experience:** Expert
- **Contract:** Permanent contract
- **Skills:** Artificial Intelligence, Amazon Web Services, Microsoft Azure, Big Data, Cloud Computing, Continuous Integration, Information Engineering, Extract Transform Load (ETL), Python (Programming Language), Performance Tuning, Scala (Programming Language), Software Engineering, SQL Databases, Scripting, Google Cloud, Cloud Platform System, Delivery Pipeline, Apache Spark, Data Lakes, Machine Learning Operations, Data Pipelines, Databricks, Programming Languages
- **Published:** October 27, 2026
- **Apply:** https://www.careerbuilder.com/job-details/databricks-specialist-aurora-il--9cc295fd-e299-4581-bf8a-ca498a095adc

## About the Role

Software Development, Python Programming/Scripting Language, Apache Spark, Scala Programming Language, SQL (Structured Query Language), Ecosystems, Cloud Computing, Microsoft Windows Azure, GCP (Good Clinical Practices), Amazon Web Services (AWS), · Experience: 5 to 10+ years in data engineering, software development, or solution architecture, with hands-on Databricks production experience.
· Technical Stack: Strong proficiency in Python, SQL, or Scala; deep knowledge of Apache Spark runtime internals and Delta Lake.
· Cloud Ecosystems: Working knowledge of major cloud platforms like Microsoft Azure, AWS, or GCP.
· Certifications: Preferred credentials include Databricks Certified Data Engineer or Solutions Architect credentials. []

## Description

Architecture & Development: Design end-to-end data pipelines, ETL/ELT processes, and lakehouse storage patterns using Apache Spark, Delta Lake, and Python or SQL.

 - Performance Tuning: Optimize cluster sizing, query execution, shuffles, partitions, and storage layouts for large-scale data workflows. [1]
 - AI & Machine Learning Integration: Implement MLOps workflows, manage model lifecycles, or deploy generative AI and vector search solutions using tools like MLflow and Unity Catalog. [1]
 - Collaboration & Consulting: Partner with enterprise stakeholders, data scientists, and software engineers to translate business requirements into technical solutions. [1]
 - Administration & Governance: Oversee security configurations, access controls, workspace administration, and CI/CD deployment pipelines