WeAreDevelopers Logo WeAreDevelopers
Sign in
For You LIVE
Back

Lead Data Engineer

The Judge Group
Columbus, OH, United States
2 days ago
Apply on www.dice.com
Prepare application

Role details

Contract type
Contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
10 years minimum
Compensation
$145,600 - $166,400
Working hours
Regular working hours
Job source
www.dice.com

Tech stack

Sql Data Warehouse Java (Programming Language) Artificial Intelligence Airflow Amazon Web Services Apache HTTP Server Automation of Tests Big Data Cloud Computing Cloud Engineering Code Review Computer Programming
+30 more
Continuous Integration Data Architecture Information Engineering Data Governance Extract Transform Load (ETL) Distributed Computing Environment Distributed Systems Python (Programming Language) Cloud Services Secure Coding Software Engineering Parquet Snowflake Apache Spark Generative AI Change Data Capture AI Coding Agents Agentic-AI Data Lakes Storage Technologies Information Technology Data Lineage Avro Apache Kafka Spark Streaming Video Streaming Cloud Optimization Restful APIs Programming Languages Control M

Job description

  • Apache Spark
  • Parquet
  • Avro

Data Engineering

  • ETL / ELT Development
  • Data Modeling
  • Data Quality
  • Data Governance
  • Data Lineage

Software Engineering

  • REST APIs
  • CI/CD
  • Automated Testing
  • Observability
  • Secure Coding

AI & Automation

  • AI-Assisted Software Development Tools
  • AI Code Review and Test Automation
  • Responsible AI Practices

Requirements

  • Bachelor’s degree in Computer Science, Engineering, or a related technical field, or equivalent practical experience.
  • 10+ years of experience in software engineering with significant focus on data engineering and distributed systems.
  • Strong programming expertise in Java and Python.
  • Experience designing and building enterprise-scale ETL/ELT platforms.
  • Deep understanding of Apache Spark and distributed data processing concepts.
  • Experience with AWS cloud services and cloud-native architectures.
  • Proven experience with data lakes, lakehouse architectures, and large-scale data platforms.
  • Experience working with Parquet, Avro, and related storage technologies.
  • Experience leading technical initiatives and mentoring engineering teams.
  • Demonstrated experience leveraging AI-assisted development tools within software engineering workflows.

Preferred Qualifications

  • Experience with workflow orchestration platforms such as Airflow, Dagster, or Control-M.
  • Experience with streaming technologies including Kafka and Spark Structured Streaming.
  • Knowledge of Change Data Capture (CDC), upsert strategies, and incremental processing frameworks.
  • Experience with Delta Lake, Apache Iceberg, or Apache Hudi.
  • Experience working with Snowflake or other cloud data warehouse technologies.
  • Familiarity with FinOps and cloud cost optimization practices.
  • Experience operating mission-critical systems in regulated environments.
  • Understanding of compliance, audit, privacy, and PII governance requirements.
  • Experience building or supporting Generative AI and Agentic AI solutions.

Required Technical Skills

Programming Languages

  • Java
  • Python

Cloud & Platform

  • AWS
  • Data Lake / Lakehouse Architecture

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.dice.com
Prepare application

Good distractions

Loading talks and stories from around this role…