Data Engineer / Architect - Databricks & dbt

data modeling performance optimization SQL DevOps & CI/CD databricks data lineage delta lake

• We are looking for an experienced Data Engineer/Architect with strong hands-on expertise in Databricks and dbt (Data Build Tool) to support the implementation of enterprise data pipelines on a modern cloud-based data platform.
• The role will be responsible for defining and implementing scalable data engineering patterns using Databricks and dbt, establishing development standards and reusable frameworks, and providing hands-on technical guidance to the data engineering team. The candidate will work closely with architecture, DevOps, Data Governance, and delivery teams to ensure solutions are scalable, maintainable, and aligned with enterprise standards., * Define the technical architecture and implementation patterns for dbt-based data transformations on Databricks.
• Design and develop scalable curated and consumption-layer data pipelines using Databricks and dbt.
• Establish dbt project structure, model organization, dependencies, naming conventions, coding standards, and reusable development patterns.
• Define and implement appropriate usage of dbt models, sources, tests, macros, snapshots, incremental models, packages, and documentation.
• Design efficient transformation patterns leveraging Databricks, Delta Lake, Spark SQL, and PySpark.
• Establish reusable and standardized pipeline development patterns for adoption across the engineering team.
• Work with architecture teams to ensure implementation aligns with enterprise architecture, governance, security, and data-platform standards.
• Provide hands-on development support and technical guidance to Data Engineers.
• Review existing data transformation and pipeline patterns and determine the appropriate implementation using Databricks and dbt.
• Define data quality, reconciliation, testing, logging, monitoring, and error-handling approaches.
• Design and implement appropriate incremental data-processing strategies.
• Support performance optimization and troubleshooting of dbt models and Databricks workloads.
• Work with DevOps teams to integrate dbt and Databricks development into the enterprise CI/CD framework.
• Support orchestration and scheduling integration with Databricks Workflows and enterprise scheduling platforms.
• Conduct technical design reviews and code reviews and ensure adherence to defined engineering standards.
• Support testing, deployment, production readiness, and troubleshooting activities.
• Develop reusable templates, frameworks, utilities, and implementation guidelines to accelerate development.
• Prepare technical documentation and provide knowledge transfer to engineering and support teams.
• Mentor Data Engineers and provide technical leadership throughout design, development, testing, and deployment.

• 8+ years of experience in Data Engineering, Data Architecture, or related areas.
• Strong hands-on experience with Databricks.
• Strong hands-on experience implementing enterprise data solutions using dbt.
• Strong understanding of dbt models, sources, macros, tests, snapshots, incremental models, packages, and documentation.
• Strong programming and development skills in SQL, Spark SQL, and PySpark.
• Strong experience with Delta Lake and Databricks data-engineering capabilities.
• Experience designing and implementing Medallion Architecture / multi-layered data platforms.
• Strong understanding of logical and physical data modeling concepts.
• Experience building scalable, reusable, and metadata-driven data-engineering solutions.
• Experience implementing data-quality, validation, reconciliation, lineage, monitoring, and observability capabilities.
• Strong understanding of CI/CD and DevOps practices for dbt and Databricks.
• Experience with Git-based source control and automated deployment processes.
• Strong understanding of performance optimization and troubleshooting of large-scale data pipelines.
• Ability to translate architecture standards and business requirements into practical engineering solutions.
• Experience conducting technical design and code reviews.
• Ability to mentor and provide technical direction to Data Engineers.
• Strong analytical, problem-solving, communication, and stakeholder-management skills.

Preferred Skills:

• Databricks certification.
• dbt certification or significant production implementation experience.
• Experience with AWS-based Databricks environments.
• Experience with enterprise data governance, metadata management, lineage, and data-product concepts.
• Experience with Control-M or other enterprise workload scheduling/orchestration platforms.
• Experience working on large-scale enterprise data-platform transformation programs.

The pay range that the employer in good faith reasonably expects to pay for this position is $46.23/hour - $72.23/hour. Our benefits include medical, dental, vision and retirement benefits. Applications will be accepted on an ongoing basis.