> Markdown version of [/jobs/ext/621370-senior-data-engineer-unified-platform](https://www.wearedevelopers.com/jobs/ext/621370-senior-data-engineer-unified-platform). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Senior Data Engineer, Unified Platform - **Company:** D R W Inc - **Location:** United States - **Experience:** Expert - **Salary:** $200,000.0 - $250,000.0 - **Contract:** Permanent contract - **Skills:** Airflow, Amazon S3, Big Data, Data Governance, Serialization, Data Systems, Relational Databases, Protocol Buffers, JSON, Python (Programming Language), NoSQL, Operational Databases, DataOps, SQL and Java (SQLJ), Data Streaming, Parquet, Data Processing, Data Storage Technologies, Data Ingestion, Apache Spark, Usage Tracking, Kubernetes, Apache Flink, Cassandra, Avro, Dask, Apache Kafka, Vertica, Grpc, Data Pipelines - **Published:** June 16, 2026 - **Apply:** https://www.dice.com/job-detail/01328d0d-2260-4385-a7c6-7a5a160fe30b ## About the Role * Have extensive experience designing and building data pipelines * Have experience working within modern batch and streaming data ecosystems * An expert in SQL and Java or Python * Extensive knowledge of data modeling techniques * Able to own the delivery of data products, working with analysts and stakeholders to understand requirements and implement solutions * Able to contribute to project management and project reporting * Have experience driving nontrivial data migrations across platforms * Able to establish, follow, and enforce data modeling standards across a large ecosystem., * 7+ years of experience working with modern data technologies and building data-first products. * Excellent written and verbal communication skills. * Proven ability to work in a collaborative, agile, and fast-paced environment, prioritizing multiple tasks and projects, and efficiently handle the demands of a trading environment. * Proven ability to deliver rapid results within processes that span multiple stakeholders. * Strong technical problem-solving skills. * Extensive familiarity with SQL and Java or Python, with a proven ability to develop and deliver maintainable data transformations for production data pipelines. * Experience leveraging data modeling techniques and ability to articulate the trade-offs of different approaches. * Experience with one or more data processing technologies (e.g. Flink, Spark, Polars, Dask, etc.). * Experience with multiple data storage technologies (e.g. S3, RDBMS, NoSQL, Delta/Iceberg, Cassandra, Clickhouse, Kafka, etc.) and knowledge of their associated trade-offs. * Experience with multiple data formats and serialization systems (e.g. Arrow, Parquet, Protobuf/gRPC, Avro, Thrift, JSON, etc.). * Experience managing data pipeline orchestration systems (e.g. Kubernetes, Argo Workflows, Airflow, Prefect, Dagster, etc.), delving deeply into low-level details. * Proven experience in managing the operational aspects of large data pipelines such as backfilling datasets, rerunning batch jobs, monitoring event throughput, and handling dead-letter queues. * Prior experience triaging data quality control processes, correcting data gaps and inaccuracies. ## Description * Work closely with Data Strategists to determine appropriate data sources and implement processes that onboard and manage new datasets for trading, research, and back-office purposes. * Design data governance processes that enable discovery, cost-sharing, usage tracking, access controls, and quality control of datasets to address the needs of DRW trading teams and strategies. * Continually monitor data ingestion pipelines and data quality to ensure stability, reliability, and quality of the data. Design monitoring and quality control software and processes. * Own the technical aspects of vendor ingestion pipelines, coordinating with vendor relationship managers on upcoming changes, performing routine data operations without breaking internal users, and contributing to the team's on-call rotation to respond to unanticipated changes. * Rapidly respond to user requests, identifying platform gaps, and self-service opportunities that make the user experience more efficient. ## Related Videos - [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering) - [From event streaming to event sourcing 101](https://www.wearedevelopers.com/videos/91-from-event-streaming-to-event-sourcing-101) - [Exploring the Power of gRPC-Gateway for Writing RESTful Services](https://www.wearedevelopers.com/videos/2072-exploring-the-power-of-grpc-gateway-for-writing-restful-services) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Introducing JSON Structure](https://www.wearedevelopers.com/videos/100219-introducing-json-structure) - [Boosting OpenSearch Performance: gRPC Search in Action](https://www.wearedevelopers.com/videos/1964-boosting-opensearch-performance-grpc-search-in-action) ## Related Articles - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production) - [Why Event-Driven Architecture Isn’t About Speed (and When You Actually Need It)](https://www.wearedevelopers.com/magazine/745-why-event-driven-architecture-isn-t-about-speed-and-when-you-actually-need-it)