Data Engineer - Databricks

Tech Rakers

Pittsburgh, PA, United States

5 days ago

Apply on www.dice.com

Prepare application

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Experience required

4 years minimum

Working hours

Regular working hours

Job source

www.dice.com

Tech stack

Unity 3d Application Release Automation Microsoft Azure Cloud Computing Continuous Integration Data Architecture Information Engineering Data Infrastructure Data Systems Data Warehousing Relational Databases DevOps

+13 more

Dimensional Modeling Python (Programming Language) OAuth Performance Tuning SQL Databases Enterprise Data Management Git Data Lakes Pyspark Restful APIs Software Version Control Data Pipelines Databricks

Job description

We are seeking a highly skilled Senior Data Engineer to join a growing data and analytics organization. This role will be responsible for designing, developing, optimizing, and supporting enterprise-scale data solutions leveraging the Azure Databricks Lakehouse platform. The ideal candidate will possess deep expertise in Databricks, modern data architecture, data modeling, and cloud-based engineering practices., Design, build, optimize, and maintain complex data pipelines and enterprise data solutions within Azure Databricks. Collaborate with stakeholders and senior team members to gather requirements and design scalable solutions. Perform advanced cluster tuning, monitoring, and troubleshooting of Databricks environments. Develop integrations with REST APIs using OAuth and connect to relational data sources. Implement Medallion Architecture and dimensional data models for reporting and analytics. Leverage Unity Catalog, Delta Live Tables, Delta Sharing, and Delta Lake capabilities. Implement DevOps, CI/CD, source control, and release automation best practices. Mentor team members and contribute to data platform strategy.

Requirements

7+ years of data engineering, data warehousing, or related experience. 4+ years of hands-on Azure Databricks experience. Strong expertise in SQL, Python, and PySpark. Experience designing and tuning complex Databricks pipelines. Experience with REST APIs, OAuth authentication, and relational databases. Strong understanding of Lakehouse and Medallion Architecture concepts. Experience with dimensional modeling and enterprise data warehousing. Experience implementing DevOps and CI/CD practices. Excellent communication and stakeholder management skills. Preferred Qualifications Experience creating logical and physical data models. Databricks and Azure certifications. Experience within financial services or regulated industries. Expert knowledge of Unity Catalog, Delta Live Tables, and Delta Sharing.

Technical Environment Azure Databricks, Delta Lake, Unity Catalog, Delta Live Tables, Delta Sharing, Azure Cloud Services, SQL, Python, PySpark, REST APIs, OAuth, Enterprise Data Warehousing, Dimensional Modeling, CI/CD, DevOps, and Git-based Source Control.

Enterprise Req Skills

Data,databricks,lakehouse,azure