Data Engineer - Azure Databricks

Brillfy Technology
Washington, DC, United States
1 day ago
Apply on www.dice.com
Prepare application

Role details

Contract type
Temporary to permanent
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Job source

Tech stack

Microsoft Azure Continuous Integration Data Architecture Data Governance Extract Transform Load (ETL) DevOps Python (Programming Language) Machine Learning Microsoft SQL Server SQL Azure Standard Sql SQL Databases
+10 more
Retrieval-Augmented Generation Apache Spark Git Data Lakes Pyspark Information Technology Terraform Data Pipelines Serverless Computing Databricks

Job description

We are looking for an experienced Data Engineer with strong hands-on expertise in Azure Databricks, Python, PySpark, SQL, and modern Lakehouse architecture. The ideal candidate will be responsible for designing, developing, and maintaining scalable data pipelines and enterprise data solutions.

Requirements

  • Strong hands-on experience with Azure Databricks.
  • Excellent Python, PySpark, and SQL skills.
  • Experience with Apache Spark and Delta Lake.
  • Strong knowledge of Databricks Unity Catalog.
  • Experience developing ETL/ELT data pipelines.
  • Experience with Azure SQL, SQL Server, and Azure cloud services.
  • Knowledge of Lakehouse and Medallion architecture (Bronze, Silver, Gold).
  • Experience with data modeling, data governance, and data quality.
  • Hands-on experience with CI/CD, Git, and DevOps practices.

Preferred Skills:

  • Databricks Lakeflow Connect / Delta Live Tables.
  • Databricks Workflows, Jobs, and Serverless Compute.
  • Microsoft Purview or Informatica Data Governance.
  • Power BI integration.
  • GenAI, RAG, and Machine Learning data pipelines.
  • Terraform / Infrastructure as Code.

Additional Requirements:

  • Bachelor’s degree in Computer Science or related field, or equivalent experience.
  • Strong communication and problem-solving skills.
  • Ability to work independently and collaborate with cross-functional teams.
  • Background check required.

Interview Process: 1-2 rounds via MS Teams (Technical Panel + Final Round).

Interested candidates: Please share your updated resume along with contact details, current location, visa status, and expected rate.

Hiring #DataEngineer #AzureDatabricks #PySpark #Python #SQL #DeltaLake #UnityCatalog #AzureJobs #RemoteJobs #USITJobs

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.dice.com
Prepare application

Good distractions

Loading talks and stories from around this role…