Databricks Engineer

Voto Consulting LLC

Charlotte, NC, United States

10 days ago

Apply on www.dice.com

Prepare application

Role details

Contract type

Temporary contract

Employment type

Full-time (> 32 hours)

Experience level

Experienced

Experience required

3 years minimum

Working hours

Regular working hours

Job source

www.dice.com

Tech stack

Business Analytics Applications Microsoft Azure Big Data Cloud Database Continuous Integration Information Engineering Data Governance Extract Transform Load (ETL) DevOps Identity and Access Management Python (Programming Language) NoSQL

+16 more

Azure Data Lake Scala (Programming Language) Data Processing Data Storage Technologies Azure Data Factory Sql Optimization Apache Spark Git Build Management Data Lakes Pyspark Infrastructure Automation Frameworks Information Technology Terraform Data Pipelines Databricks

Job description

Databricks Engineer to design, build, and maintain scalable data pipelines, lakehouse architectures, and analytics solutions using Microsoft Azure and the Databricks platform., * Pipeline Development: Design and build Extract, Transform, Load (ETL) and ELT data pipelines using Azure Data Factory (ADF) and Azure Databricks

Big Data Processing: Write optimized data processing code using Apache Spark, PySpark, Python, Scala, and advanced SQL.
Storage & Architecture: Implement and manage data lakes using Azure Data Lake Storage (ADLS Gen2) and Delta Lake / lakehouse patterns.
Governance & Security: Configure data governance, lineage, and access controls using Databricks Unity Catalog and Azure IAM.
Optimization & Operations: Monitor, troubleshoot, and tune Spark jobs and data pipelines for performance, reliability, and cost-efficiency.
CI/CD & DevOps: Support deployment processes utilizing Azure DevOps, Git version control, and infrastructure-as-code tools like Terraform.

Requirements

Experience: 3 to 7+ years of total experience in data engineering or cloud-based data platform development.
Core Technologies: Hands-on proficiency in Azure Databricks, Apache Spark, PySpark, and Azure Data Factory (ADF).
Languages: Advanced SQL and solid programming skills in Python or Scala.
Data Concepts: Strong grasp of Delta Lake, relational/NoSQL data storage, data modeling, and data quality standards.
Education: Bachelor’s degree in Computer Science, Engineering, or a related quantitative field (often preferred)