> Markdown version of [/jobs/ext/1320934-lead-data-engineer](https://www.wearedevelopers.com/jobs/ext/1320934-lead-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Lead Data Engineer

- **Company:** THE JUDGE GROUP, INC.
- **Location:** New York, NY, United States
- **Experience:** Expert
- **Salary:** $166,400.0 - $183,040.0
- **Contract:** Temporary to permanent
- **Skills:** Java (Programming Language), Agile Methodology, Airflow, Amazon Web Services, Amazon S3, JIRA, Cloud Computing, Continuous Integration, Information Engineering, Data Migration, Data Systems, Data Warehousing, Identity and Access Management, JSON, Python (Programming Language), Cloud Services, Shell Script, Parquet, Test-Driven Development (TDD), Apache Spark, HybridCloud, Git, Pyspark, Solid Principles, Kubernetes, Avro, Data Management, Restful APIs, Data Pipelines, Jenkins
- **Published:** July 17, 2026
- **Apply:** https://www.dice.com/job-detail/f58d79b9-1c5f-4ad6-9627-7893a389265a

## About the Role

9-12+ Experience in Data Engineering.

Strong proficiency in Python and/or Java with Spark/PySpark.

Experience with Data Pipelines, Data Modeling, Data Warehousing, and Data Migration.

Hands-on experience with AWS (Glue, EMR, S3, Athena, Kinesis/MSK, Airflow, Lambda, Step Functions, Lake Formation).

Knowledge of Kubernetes, CI/CD, Agile methodologies, and software design principles.

Experience with data formats such as Parquet, Iceberg, Avro, or JSON.

Understanding of AWS security concepts including IAM, KMS, Encryption, and Secrets Manager.

Preferred Qualifications

Experience building REST APIs using Java.

Knowledge of Hybrid Cloud architectures.

Exposure to Jenkins, Git, Linux Scripting, JIRA, and TDD.

Strong leadership, problem-solving, and communication skills.

Tech Stack: Python, Java, Spark, AWS, Kubernetes, Airflow, Glue, EMR, Athena, Kinesis, Parquet, Iceberg, CI/CD.

## Description

We are seeking a Lead Software Engineer - Data Engineering to design and build scalable, cloud-native data platforms and pipelines. You will lead the development of robust data solutions, drive architectural decisions, and mentor engineers while ensuring reliability, security, and operational excellence.

Responsibilities

Design and develop large-scale data pipelines and data platforms.

Build cloud-native solutions using AWS services and Spark.

Lead data modeling, warehousing, migration, and analytics initiatives.

Improve system performance, resilience, monitoring, and automation.

Collaborate with cross-functional teams and mentor junior engineers.


## Related Videos

- [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology)
- [From event streaming to event sourcing 101](https://www.wearedevelopers.com/videos/91-from-event-streaming-to-event-sourcing-101)
- [Improving quality with Agentic AI with Rovo Dev and Xray](https://www.wearedevelopers.com/videos/2005-improving-quality-with-agentic-ai-with-rovo-dev-and-xray)
- [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94)
- [Introducing JSON Structure](https://www.wearedevelopers.com/videos/100219-introducing-json-structure)
- [Collaboration Quantified: Lessons from Open Source Developer Networks](https://www.wearedevelopers.com/videos/1422-collaboration-quantified-lessons-from-open-source-developer-networks)

## Related Articles

- [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)
- [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story)
- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [Dev Digest 121 - AI goes offline](https://www.wearedevelopers.com/magazine/456-dev-digest-121-ai-goes-offline)
- [Top 10 Java Libraries](https://www.wearedevelopers.com/magazine/364-top-10-java-libraries)