> Markdown version of [/jobs/ext/804994-lead-data-engineer](https://www.wearedevelopers.com/jobs/ext/804994-lead-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Lead Data Engineer

- **Company:** Optiver Us, Llc
- **Location:** Chicago, IL, United States
- **Experience:** Expert
- **Salary:** $200,000.0
- **Contract:** Permanent contract
- **Skills:** Amazon Web Services, C++ (Programming Language), Data Architecture, Information Engineering, Data Infrastructure, Extract Transform Load (ETL), Relational Databases, Python (Programming Language), PostgreSQL, Performance Tuning, Cloud Services, DataOps, Data Streaming, Apache Spark, Data Strategy, Data Lakes, Information Technology, Data Analytics, Apache Kafka, Data Management, Machine Learning Operations, Data Pipelines, Databricks
- **Published:** June 2, 2026
- **Apply:** https://www.dice.com/job-detail/576714d3-2768-4ad0-8179-72c11925ef02

## About the Role

* 5+ years of hands-on experience in data engineering, delivering robust pipelines at scale
 * Advanced Python skills and deep experience with Apache Spark and the Databricks platform
 * Familiarity with Delta Lake, streaming data systems (e.g., Kafka), and distributed compute environments
 * Solid understanding of cloud-native data architectures (preferably AWS) and infrastructure cost optimization principles
 * Proficiency in relational databases (e.g., PostgreSQL) and modern orchestration tools
 * Bonus: Experience with system-level languages (e.g., C++, Rust) and exposure to MLOps or MLflow
 * Proven ability to lead projects independently and deliver outcomes in fast-paced environments
 * Clear communicator who collaborates well with researchers, traders, and engineers alike
 * Enthusiastic mentor and strong advocate for engineering rigor and platform scalability
 * Bachelor's or Master's degree in Computer Science, Engineering, or a related technical discipline

## Description

Optiver Chicago is seeking a seasoned Data Engineer to support and advance the data capabilities of our local research team and contribute to our broader Lakehouse architecture vision. In this role, you'll work with cutting-edge tools like Databricks, Apache Spark, and Delta Lake, translating experimental research workflows into scalable, production-grade systems.

As a critical engineering presence in Chicago, you'll directly influence local data strategy while collaborating with our Global Research and Data Platform teams. Your work will drive real-time insights, enable predictive analytics, and power decision-making across the trading lifecycle.

This role combines hands-on data engineering, cross-regional collaboration, and platform innovation-all within a high-performance, data-driven environment.

What you'll do:

 * Design, build, and maintain reliable ETL/ELT pipelines using Spark, Structured Streaming, Databricks, and our in-house high-performance tools
 * Optimize and productionize research workflows with a strong focus on scalability, resilience, and performance tuning
 * Collaborate with power users to develop and share reusable patterns, templates, and onboarding pathways
 * Define, document, and enforce data engineering best practices
 * Mentor junior engineers and drive a culture of continuous learning and DataOps excellence


## Related Videos

- [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning)
- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)
- [Cutting LLM Costs Without Cutting Quality: How to Beat Proprietary LLMs with Fine-Tuned Open Source](https://www.wearedevelopers.com/videos/100151-cutting-llm-costs-without-cutting-quality-how-to-beat-proprietary-llms-with-fine-tuned-open-source)
- [Parquet, Delta, Iceberg & Ducklake - An introduction for developers](https://www.wearedevelopers.com/videos/100075-parquet-delta-iceberg-ducklake-an-introduction-for-developers)
- [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering)
- [From A2A to MCP: How AI’s “Brains” are Connecting to “Arms and Legs”](https://www.wearedevelopers.com/videos/1631-from-a2a-to-mcp-how-ai-s-brains-are-connecting-to-arms-and-legs)

## Related Articles

- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)
- [MLops – Deploying, Maintaining And Evolving Machine Learning Models in Production](https://www.wearedevelopers.com/magazine/115-mlops-deploying-maintaining-and-evolving-machine-learning-models-in-production)
- [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story)
- [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer)