> Markdown version of [/jobs/ext/4486930-databricks-data-engineering-consultant](https://www.wearedevelopers.com/jobs/ext/4486930-databricks-data-engineering-consultant). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Databricks Data Engineering Consultant - **Company:** SDG GROUP, LLC - **Location:** Bedminster, NJ, United States (Remote available) - **Experience:** Experienced - **Contract:** Permanent contract - **Skills:** Query Performance, Artificial Intelligence, Data Analysis, Cloud Computing, Cloud Database, Computer Programming, Data Validation, Information Engineering, Data Integration, Extract Transform Load (ETL), Data Transformation, Data Migration, Data Systems, Data Warehousing, Database Development, Database Queries, Dimensional Modeling, Apache Hive, Python (Programming Language), Microsoft Software, Operational Databases, Performance Tuning, Software Deployment, SQL Databases, Data Processing, Scripting, Cloud Platform System, Snowflake, Git, Data Lakes, Pyspark, Data Management, Software Version Control, Data Pipelines, Databricks - **Published:** October 31, 2026 - **Apply:** https://www.careerbuilder.com/job-details/databricks-data-engineering-consultant-bedminster-nj--127305a6-9789-4cad-a927-e439ca765242 ## About the Role * Professional Experience: Approximately 2-5+ years of experience in data engineering, data warehousing, data integration, analytics engineering, or a related technical discipline. Experience within consulting or another client-facing environment is a plus. * Databricks Experience: Hands-on professional experience working within Databricks is required, with demonstrated use of the platform for production data engineering and analytical workloads, including Delta Lake and Unity Catalog. Databricks Certified Data Engineer Associate certification is a plus. * Databricks SQL: Strong experience using Databricks SQL, including writing, troubleshooting, and optimizing queries used for data transformation, validation, reconciliation, and downstream analytics, and working with SQL Warehouses, notebooks, and the Query Profile. * SQL & Programming: Strong SQL skills, including joins, CTEs, window functions, aggregations, and complex transformations. Working proficiency in Python or another relevant programming or scripting language. * Data Engineering Fundamentals: Strong understanding of data pipelines, ETL/ELT processes, data integration, data warehousing, and data transformation concepts. * Data Modeling: Understanding of dimensional modeling and modern data modeling concepts used within analytical environments. * Cloud & Data Platforms: Experience working with modern cloud environments and enterprise-scale data platforms. * Data Integration Tools: Experience with modern data engineering, orchestration, or transformation tools. Experience with Matillion and/or dbt is highly preferred. * Data Quality: Experience validating, reconciling, and troubleshooting data across multiple source and target systems. * Problem Solving: Strong analytical skills with the ability to troubleshoot data issues, understand complex requirements, and translate business needs into technical solutions. * Consulting & Communication Skills: Strong written and verbal communication skills with the ability to collaborate effectively with clients, technical teams, and business stakeholders. * Growth Mindset: A proactive, collaborative approach and willingness to continuously learn new tools, technologies, and methodologies. Preferred Experience * Databricks SQL Warehouses * Delta Lake / Delta Tables * Databricks Workflows and orchestration * Unity Catalog * Databricks query and performance optimization * Delta Live Tables / Lakeflow Declarative Pipelines * Medallion architecture (bronze/silver/gold) * Databricks Git folders and Databricks Asset Bundles * Matillion * Dbt * Python / PySpark * Snowflake * Experience supporting enterprise-scale data migration or modernization initiatives * Experience working within pharmaceutical, life sciences, healthcare, or another highly regulated industry ## Description As a Databricks Data Engineering Consultant, you will play an important role in designing, developing, and delivering modern data solutions for SDG Group's clients. Working as part of a consulting delivery team, you will partner with both technical and business stakeholders to understand data requirements and build reliable, scalable solutions that support analytics, reporting, AI, and broader data-driven initiatives. This opportunity requires hands-on Databricks experience, with particular emphasis on Databricks SQL and SQL-based data engineering workloads on the Databricks Lakehouse Platform. Your work will center on Delta Lake tables organized in a medallion (bronze/silver/gold) architecture, governed through Unity Catalog, orchestrated with Databricks Workflows, and queried through Databricks SQL Warehouses. You will work with complex enterprise data environments, developing and optimizing transformations, validating data, supporting pipelines, and helping ensure that data is structured and delivered effectively for downstream business and analytical use cases. This role is well suited for someone with a strong foundation in data engineering who enjoys hands-on technical delivery, solving complex problems, working directly with clients, and continuing to expand their expertise across modern cloud and data platforms., * Databricks Development: Design, build, test, and maintain scalable data solutions within Databricks, including Delta Lake tables organized in a medallion (bronze/silver/gold) architecture and jobs orchestrated with Databricks Workflows, with a strong focus on SQL-based transformation and data processing workloads running on Databricks SQL Warehouses. * SQL Development: Write, troubleshoot, and optimize complex SQL and Spark SQL queries used for data transformation, validation, reconciliation, and analytical workloads, including MERGE INTO upserts, Delta time travel, and performance tuning with OPTIMIZE and the Databricks Query Profile. * Data Engineering & Development: Build and maintain scalable data pipelines and processing solutions that support client analytics and business requirements. * Data Integration: Integrate data from a variety of structured and unstructured sources using modern ETL/ELT methodologies and data integration tools, including batch and incremental loads into Delta tables using COPY INTO and MERGE. * Data Modeling & Warehousing: Develop and maintain data models and warehouse structures that support reporting, analytics, and downstream applications. * Cloud Data Solutions: Contribute to the implementation and modernization of cloud-based data platforms using Databricks and related cloud technologies. * Technical Delivery: Transform, validate, and manage enterprise data while following development, testing, documentation, and deployment best practices, including working with Unity Catalog (catalogs, schemas, and permissions) and using Git folders for version control of Databricks code. * Data Quality & Performance: Identify, troubleshoot, and resolve data quality issues while improving reliability, scalability, and query performance. * Client Collaboration: Work directly with clients and internal stakeholders to gather requirements, understand business challenges, communicate technical concepts, and deliver solutions aligned with client needs. * Team Collaboration: Work closely with data engineers, analysts, architects, data scientists, and business stakeholders throughout the project lifecycle. * Continuous Learning: Stay current with evolving Databricks capabilities, data engineering methodologies, and industry best practices while contributing knowledge and ideas to the broader SDG community.