> Markdown version of [/jobs/ext/4605929-lead-data-engineer-dbt-python-sql-databricks](https://www.wearedevelopers.com/jobs/ext/4605929-lead-data-engineer-dbt-python-sql-databricks). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Lead Data Engineer - dbt, Python, SQL, Databricks

- **Company:** Insight Global
- **Location:** Westbrook, ME, United States
- **Experience:** Expert
- **Contract:** Permanent contract
- **Skills:** Artificial Intelligence, Amazon Web Services, Data Analysis, Cloud Computing, Continuous Integration, Information Engineering, Extract Transform Load (ETL), Data Warehousing, Fault Tolerance, Github, Apache Hive, JSON, Python (Programming Language), Machine Learning, Operational Databases, Cloud Services, SQL Databases, Data Streaming, Parquet, Sql Optimization, Delivery Pipeline, Snowflake, Build Management, Data Management, Data Pipelines, Databricks
- **Published:** November 6, 2026
- **Apply:** https://dejobs.org/x/x/BBB13EE9A1CD48B0B78226B1C6D3DEB8/job/

## About the Role

* Advanced SQL and strong data modeling expertise. * Strong knowledge of data warehousing and ETL/ELT architecture. * Experience with Databricks, Snowflake, AWS, or similar cloud data platforms. * Experience with CI/CD, GitHub, and production data pipelines. * 8+ years of data engineering experience. * dbt Core or dbt Cloud * Spark SQL and Python * Streaming ingestion and schema evolution * Parquet and JSON processing

## Description

Seeking a Senior/Lead Data Engineer to design, build, and optimize scalable data pipelines supporting analytics, AI, and machine learning initiatives. This role will own pipeline development from ingestion through transformation and delivery while driving data quality, reliability, and engineering best practices.

Key Responsibilities * Design and build scalable, fault-tolerant data pipelines using modern cloud data platforms. * Develop complex SQL transformations and data models leveraging medallion architecture. * Build and maintain DBT testing, data quality, and validation frameworks. * Support batch and streaming ingestion, including schema evolution and Parquet/JSON processing. * Optimize pipeline performance, reliability, and scalability. * Manage CI/CD and deployment workflows through GitHub. * Partner with ML engineers and data scientists to support AI/ML data and feature pipelines. * Drive strong documentation, lineage, and engineering standards.