WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
Back

Senior Data Software Engineer with Databricks

EPAM Systems, Inc.
Atlanta, GA, United States
2 days ago
Apply on www.careerbuilder.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
5 years minimum
Working hours
Regular working hours
Languages
English
Job source
www.careerbuilder.com

Tech stack

Artificial Intelligence Microsoft Azure Cloud Computing Code Review Continuous Integration Information Engineering Data Governance Data Integration Data Structures Dimensional Modeling Python (Programming Language) PostgreSQL
+19 more
Operational Data Store Azure Data Lake Software Engineering Data Streaming Data Ingestion Microsoft Power Automate Azure Data Factory Sql Optimization Apache Spark Git Data Lakes Pyspark Data Programming Data Management Terraform Data Pipelines Powerapps Key Vault Databricks

Job description

We are seeking a Senior Data Software Engineer with Databricks to lead the design, development, and maintenance of scalable data integration pipelines across cloud and enterprise systems. The role focuses on building a Databricks Lakehouse on Azure, driving data ingestion, transformation, orchestration, and automation initiatives across the broader Azure data ecosystem.

Responsibilities

  • Lead the design and development of a Databricks Lakehouse on Azure
  • Research source systems, then analyze and document their data structures, data flows and business entities
  • Build batch and near-real-time ingestion pipelines with Databricks Auto Loader and Azure Event Hubs
  • Develop dbt transformations following the medallion architecture
  • Design and maintain an operational data store (ODS) on Databricks Lakebase
  • Implement integrations with Dataverse and the Power Platform
  • Define data engineering standards, code reviews and best practices
  • Ensure data quality, testing and documentation across the platform

Requirements

Software Engineering, Software Engineering, Ecosystems, Automation, Microsoft Windows Azure, Cloud Computing, Data Management, Best Practices, Code Reviews, Standards Development, * 5+ years of data engineering experience, including a technical lead role

  • Expertise in Databricks, Delta Lake and Apache Spark (PySpark)
  • Skills in dbt and Databricks ingestion tools
  • Proficiency in Unity Catalog
  • Advanced SQL and strong Python skills
  • Solid data modeling skills, including dimensional modeling
  • Proven ability to analyze unfamiliar source systems and produce clear data documentation
  • Familiarity with the Azure data ecosystem: ADLS Gen2, Key Vault and Entra ID
  • Experience with Git and CI/CD for data pipelines
  • Flexibility to use AI tools
  • English proficiency at B2 level or higher

Nice to have

  • Background in Databricks Lakebase or PostgreSQL
  • Understanding of data governance, Lakehouse and ODS concepts
  • Experience with Dataverse, Power Apps or Power Automate integration
  • Familiarity with Terraform or Databricks Asset Bundles
  • Databricks certifications
  • AI certifications

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.careerbuilder.com
Prepare application

Good distractions

Loading talks and stories from around this role…