> Markdown version of [/jobs/ext/2722784-data-engineer](https://www.wearedevelopers.com/jobs/ext/2722784-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Data Engineer - **Company:** NINJATRADER, LLC - **Location:** Chicago, IL, United States (Remote available) - **Experience:** Expert - **Salary:** $100,000.0 - $150,000.0 - **Contract:** Permanent contract - **Skills:** Application Programming Interfaces (APIs), Agile Methodology, Airflow, Amazon S3, Apache HTTP Server, BigQuery, Cloud Database, Cloud Storage, Software Quality, Data Architecture, Information Engineering, Extract Transform Load (ETL), Data Systems, Fault Tolerance, Machine Learning, Cloud Services, Prometheus, Service-Oriented Architecture, Simple Data Format, Data Streaming, Parquet, Datadog, Data Processing, Macros, Google Data Studio, System Availability, Delivery Pipeline, Grafana, Change Data Capture, Backend, Event Driven Architecture, Data Lakes, Debezium, Kubernetes, Information Technology, Apache Flink, Avro, Real Time Data, Apache Kafka, Spark Streaming, Video Streaming, Terraform, Data Pipelines - **Published:** September 4, 2026 - **Apply:** https://startup.jobs/sr-data-engineer-ninjatrader-8411358 ## About the Role * Bachelor's degree in Computer Science, Engineering, or a related field * 5+ years of experience in data engineering, platform engineering, or backend development * Strong skills in SQL and Python for building and testing data solutions * Hands-on experience with GCP and GCP data products (BigQuery, Cloud SQL, Cloud Storage, etc.) * Experience with AWS cloud services (S3, Glue, Athena, Kinesis) in addition to GCP; bonus for EMR * Experience with CI/CD pipelines, infrastructure-as-code, and version-controlled deployment workflows (e.g., Terraform, GitOps) * Hands-on dbt experience building and maintaining dbt projects (models, tests, macros, documentation, CI) * Proficiency with workflow orchestration tools (e.g., Airflow, Prefect) * Knowledge of data lake architecture, including file formats (Parquet, Avro) and open table formats (Apache Iceberg) * Familiarity with event-driven and service-oriented architecture * A track record of building automated, well-tested, and observable data systems * Comfortable working both independently and collaboratively in a fast-paced Agile environment, * Hands-on Kubernetes experience, especially around data workloads and containerized pipelines * Experience with streaming technologies (e.g., Kafka, Spark Streaming, Flink) and comfort working with high-volume, low-latency data flows * Experience with change data capture tools (e.g., Debezium, Kafka Connect) and real-time data integration patterns * Experience with BI tools like Looker Studio or QuickSight * Experience with observability and monitoring tooling (e.g., Datadog, Grafana, Prometheus) * Background in fintech, trading, or derivatives ## Description As a Sr. Data Engineer, your primary responsibility is to stay one step ahead of your fellow team members by providing the platform, tools, and infrastructure they need to deliver deployable solutions to real-world business problems. This goes beyond maintaining pipelines-it's about delivering best-in-class data architecture, scalable workflows, and real-time solutions that power trading, product development, and machine learning. In this role you will: * Design, build, and maintain robust data pipelines and data lake architecture for both batch and real-time streaming use cases, including high-volume, low-latency data processing * Improve observability, alerting, and SLOs across data systems so pipelines are easier to monitor and issues are caught early * Optimize ETL/ELT workflows for performance, scalability, and fault tolerance * Develop dbt workflows to onboard Evaluation Partners and create end-of-day reporting for partner analysis * Build and support event-driven architectures and scalable platform components * Contribute to the orchestration and automation of workflows * Integrate complex financial APIs and third-party data sources into internal systems * Collaborate with analytics, product, and ML engineers to develop and deploy reliable data products * Work on feature pipelines and model-ready data to support ML engineers * Promote high standards in code quality, testing, and platform reliability * Participate in Agile ceremonies and foster a collaborative, growth-oriented team culture ## Related Videos - [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt) - [Parquet, Delta, Iceberg & Ducklake - An introduction for developers](https://www.wearedevelopers.com/videos/100075-parquet-delta-iceberg-ducklake-an-introduction-for-developers) - [From event streaming to event sourcing 101](https://www.wearedevelopers.com/videos/91-from-event-streaming-to-event-sourcing-101) - [Crafting Custom Frameworks with Rust: A Deep Dive into Procedural Macros](https://www.wearedevelopers.com/videos/849-crafting-custom-frameworks-with-rust-a-deep-dive-into-procedural-macros) - [Tracking vehicles at scale](https://www.wearedevelopers.com/videos/1999-tracking-vehicles-at-scale) - [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [How We Built a Worry-Free System That Runs for 10+ Years – And What We’d Do Again](https://www.wearedevelopers.com/magazine/751-how-we-built-a-worry-free-system-that-runs-for-10-years-and-what-we-d-do-again) - [The Most Popular IT Jobs on the Market](https://www.wearedevelopers.com/magazine/376-the-most-popular-it-jobs-on-the-market)