> Markdown version of [/jobs/ext/2322725-data-engineer-detect-track-distillation](https://www.wearedevelopers.com/jobs/ext/2322725-data-engineer-detect-track-distillation). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Data Engineer (Detect & Track Distillation) - **Company:** Harmattan AI - **Location:** Paris, France - **Contract:** Permanent contract - **Skills:** Information Engineering, Data Files, Data Stores, Distributed Computing Environment, Python (Programming Language), Parsing, Data Streaming, Unstructured Data, Management of Software Versions, Pytorch, Data Layers, Decoding, Data Pipelines - **Published:** August 5, 2026 - **Apply:** https://fr.indeed.com/viewjob?jk=9670f8b6ec23f1d1 ## About the Role * Educational Background: A degree in a STEM field, or equivalent practical experience. Practical data engineering experience matters more than the specific degree. * Data Pipelines at Scale: Built and maintained pipelines for unstructured data at scale (video, images, or sensor data), covering ingestion, decode, storage, curation, and versioning. * Engineering: Strong in Python and data engineering, and comfortable optimizing data loaders for common training frameworks (for example PyTorch). * Bonus: Multimodal sensor data, labeling or dataset construction for ML, dataset versioning tooling, and distributed data processing tooling. * Attributes: Systematic, quality-minded, pragmatic, and service-oriented so the modelers are enabled, with a knack for taming messy data via automation. ## Description Our ML teams train models on datasets derived from large volumes of raw, unstructured data. Model quality depends directly on data quality, and today that data is handled largely by hand. As a Data Engineer, operating out of Paris, Lausanne, or Zurich, you will own the data layer that feeds the team's models, from raw field logs or public datasets through curated, versioned, training-ready datasets. You will manage terabytes of raw, unstructured data and turn it into clean, documented, versioned datasets, so that the modelers spend their time designing and training models, not waiting on data loaders or wrangling corrupted files. You join at an early stage with real influence over how field and public data gets processed for deep-learning pipelines., * Ingestion Pipeline: Ingest, decode, and store raw, unstructured field data (video and other sensor streams) from field logs into efficient, controlled formats. * Multimodal Alignment: Align multiple data streams temporally and spatially so paired data is usable for training. * Curation: Transfer both ingested data and public datasets into high-value data, including parsing, filtering, de-duplication and revision. * Data & Labeling Requirements: Define which data to gather and what and how to label it, and own the dataset-construction workflow and labeling tooling. Coordinate with the teams responsible for data gathering and labeling. * Dataset Construction: Build task-specific datasets for the team's training and evaluation needs, in collaboration with acquisition, annotation and product teams. * Versioning & Lineage: Version datasets and maintain lineage so training runs stay reproducible. * Storage & Formats: Store data in efficient, training-ready formats (such as columnar or sharded formats) and manage storage tiering to balance cost and latency as datasets grow. * Efficient Delivery: Deliver clean, documented datasets and the corresponding tools for loading to keep training from being I/O-bound, shaping their structure with the modelers. ## Related Videos - [Tour de Force: Open-Source LLM Inference Optimization from Simple to Sophisticated](https://www.wearedevelopers.com/videos/100099-tour-de-force-open-source-llm-inference-optimization-from-simple-to-sophisticated) - [Tips and Tricks for Working with JSON](https://www.wearedevelopers.com/videos/1229-tips-and-tricks-for-working-with-json) - [Photonic Computing: Programming a New Class of AI Accelerators (incl. Live Coding)](https://www.wearedevelopers.com/videos/100196-photonic-computing-programming-a-new-class-of-ai-accelerators-incl-live-coding) - [Data Science, ML & AI in the Oil and Gas Industry at NDT Global - Dr. Katja Träumner](https://www.wearedevelopers.com/videos/1308-data-science-ml-ai-in-the-oil-and-gas-industry-at-ndt-global-dr-katja-traumner) - [Geometric deep learning for drug discovery](https://www.wearedevelopers.com/videos/264-geometric-deep-learning-for-drug-discovery) - [Building a Compiler with C#](https://www.wearedevelopers.com/videos/116-building-a-compiler-with-c) ## Related Articles - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production) - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Dev Digest 233: Strip AI Watermarks, Hacked Homelabs & Designing with Code](https://www.wearedevelopers.com/magazine/754-dev-digest-233-strip-ai-watermarks-hacked-homelabs-designing-with-code) - [Dev Digest 120 - Apple and peers](https://www.wearedevelopers.com/magazine/455-dev-digest-120-apple-and-peers)