> Markdown version of [/jobs/ext/3158675-lead-data-engineer-h-f](https://www.wearedevelopers.com/jobs/ext/3158675-lead-data-engineer-h-f). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Lead Data Engineer H/F - **Company:** Technologybehind Sparteo - **Location:** Lille, France - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Java (Programming Language), Artificial Intelligence, Data Analysis, Application Integration Architecture, Automation of Tests, Cluster Analysis, Software Quality, Databases, Continuous Integration, Information Engineering, Data Governance, Data Infrastructure, Data Security, Distributed Data Store, Python (Programming Language), NoSQL, Operational Databases, Prometheus, DataOps, Scala (Programming Language), SQL Databases, Data Streaming, Data Processing, Data Storage Technologies, System Availability, Database Performance, Generative AI, Data Strategy, Kubernetes, Apache Kafka, Hardware Infrastructure, Data Pipelines, Golang - **Published:** September 16, 2026 - **Apply:** https://www.hellowork.com/fr-fr/emplois/83403712.html ## About the Role Experience: 6+ years in Data Engineering, with at least 2 years in a leadership or "referent" role within fast-paced environments. - Database Mastery: Expert proficiency in distributed data systems, clustering, advanced table types, and materialized views. - Optimization: Skilled in designing table sorting keys and fine-tuning database performance. - Stack Mastery: Solid programming skills in Python, Golang, and Scala/Java, with deep expertise in Kafka for streaming/batch data pipelines. - Infrastructure: Hands-on experience with on-premise infrastructure (own datacenter) managed by Kubernetes, and familiarity with monitoring stacks such as Prometheus. - Data Storage: Strong knowledge of column-oriented databases designed for high-volume data processing, vector databases (e.g. Qdrant), and SQL/NoSQL technologies. - Tooling: Comfortable working with project tracking tools (Flyspray) and applying CI/CD & DataOps best practices. - AI & Innovation (Nice to have): Interest or hands-on experience leveraging Generative AI tools in daily engineering workflows to boost productivity and innovation. Your mind set to share our adventure - You want to make an impact and move things forward collectively. Does hearing phrases like "Yes, but we've been doing it this way for years..." make your hair stand on end? We feel the same way: progress is made by questioning what already exists. - You solve problems pragmatically and analytically. - You're looking for a fast-moving environment where your agility will be an asset. The 80-20 (Pareto) principle holds no secrets for you. - Your ability to listen encourages you to challenge and improve yourself on an ongoing basis., Experience: 6+ years in Data Engineering, with at least 2 years in a leadership or "referent" role within fast-paced environments. - Database Mastery: Expert proficiency in distributed data systems, clustering, advanced table types, and materialized views. - Optimization: Skilled in designing table sorting keys and fine-tuning database performance. - Stack Mastery: Solid programming skills in Python, Golang, and Scala/Java, with deep expertise in Kafka for streaming/batch data pipelines. - Infrastructure: Hands-on experience with on-premise infrastructure (own datacenter) managed by Kubernetes, and familiarity with monitoring stacks such as Prometheus. - Data Storage: Strong knowledge of column-oriented databases designed for high-volume data processing, vector databases (e.g. Qdrant), and SQL/NoSQL technologies. - Tooling: Comfortable working with project tracking tools (Flyspray) and applying CI/CD & DataOps best practices. - AI & Innovation (Nice to have): Interest or hands-on experience leveraging Generative AI tools in daily engineering workflows to boost productivity and innovation. - You want to make an impact and move things forward collectively. Does hearing phrases like "Yes, but we've been doing it this way for years..." make your hair stand on end? We feel the same way: progress is made by questioning what already exists. - You solve problems pragmatically and analytically. - You're looking for a fast-moving environment where your agility will be an asset. The 80-20 (Pareto) principle holds no secrets for you. - Your ability to listen encourages you to challenge and improve yourself on an ongoing basis., bachelor degree EducationalOccupationalCredential associate degree ## Description As our Lead Data Engineer, your mission is to architect, scale, and maintain the backbone of our data ecosystem. You will transform massive volumes of raw signals into actionable intelligence, ensuring our infrastructure is as agile as our growth strategy. You will bridge the gap between high-level data strategy and hands-on technical excellence. Your Responsibilities & Key Deliverables Data Infrastructure Design and Optimization - Architectural Leadership: Lead the design, implementation, and optimization of distributed data architectures to support massive Batch & Streaming pipelines. - Scalability & Performance: Ensure the scalability, security, and performance of the infrastructure, maintaining high system availability (99.9%+) for analytics platforms. - Efficiency: Optimize storage solutions to balance high-speed processing with cost-efficiency. - AI Integration: Collaborate with software engineers and data scientists to seamlessly integrate AI-driven models into production data workflows. Technical Leadership and Team Management - Mentorship: Lead and mentor a team of 2 data engineers, fostering a culture of continuous improvement, growth, and technical excellence. - Standards & Quality: Set high standards for code quality, documentation, and rigorous peer reviews. - Execution: Oversee project execution, delegate responsibilities, and act as the technical point of escalation for complex data-related roadblocks. - Best Practices: Guide technical decisions and promote industry best practices in data engineering (CI/CD, DataOps). Collaboration and Data Governance - Cross-Functional Alignment: Work closely with product managers, developers, and analytics teams to define data needs and ensure alignment with business objectives. - Reliability: Implement robust monitoring, alerting, and automated testing (Data Quality) to ensure the integrity of the data team's output. - Compliance: Ensure data security and GDPR compliance across all international jurisdictions and workflows. Innovation and Problem Solving - Emerging Tech: Identify opportunities to leverage new technologies or improve existing workflows to optimize data processing. - Complexity Simplification: Maintain a deep understanding of data trends and possess the ability to simplify complex problems into executable tasks. - Generative AI Adoption: Encourage and champion the use of Generative AI tools across the team to accelerate development cycles, automate repetitive tasks, and enhance data exploration capabilities., As our Lead Data Engineer, your mission is to architect, scale, and maintain the backbone of our data ecosystem. You will transform massive volumes of raw signals into actionable intelligence, ensuring our infrastructure is as agile as our growth strategy. You will bridge the gap between high-level data strategy and hands-on technical excellence. - Architectural Leadership: Lead the design, implementation, and optimization of distributed data architectures to support massive Batch & Streaming pipelines. - Scalability & Performance: Ensure the scalability, security, and performance of the infrastructure, maintaining high system availability (99.9%+) for analytics platforms. - Efficiency: Optimize storage solutions to balance high-speed processing with cost-efficiency. - AI Integration: Collaborate with software engineers and data scientists to seamlessly integrate AI-driven models into production data workflows. - Mentorship: Lead and mentor a team of 2 data engineers, fostering a culture of continuous improvement, growth, and technical excellence. - Standards & Quality: Set high standards for code quality, documentation, and rigorous peer reviews. - Execution: Oversee project execution, delegate responsibilities, and act as the technical point of escalation for complex data-related roadblocks. - Best Practices: Guide technical decisions and promote industry best practices in data engineering (CI/CD, DataOps). - Cross-Functional Alignment: Work closely with product managers, developers, and analytics teams to define data needs and ensure alignment with business objectives. - Reliability: Implement robust monitoring, alerting, and automated testing (Data Quality) to ensure the integrity of the data team's output. - Compliance: Ensure data security and GDPR compliance across all international jurisdictions and workflows. - Emerging Tech: Identify opportunities to leverage new technologies or improve existing workflows to optimize data processing. - Complexity Simplification: Maintain a deep understanding of data trends and possess the ability to simplify complex problems into executable tasks. - Generative AI Adoption: Encourage and champion the use of Generative AI tools across the team to accelerate development cycles, automate repetitive tasks, and enhance data exploration capabilities. ## Related Videos - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Go with the Flow: Stop the Leaks Before Your Memory's a Waterfall!](https://www.wearedevelopers.com/videos/100073-go-with-the-flow-stop-the-leaks-before-your-memory-s-a-waterfall) - [Leveraging Real time data in FSIs](https://www.wearedevelopers.com/videos/806-leveraging-real-time-data-in-fsis) - [Beyond Hiring: Building a Sustainable Talent Acquisition Engine at Deutsche Bahn](https://www.wearedevelopers.com/videos/1856-beyond-hiring-building-a-sustainable-talent-acquisition-engine-at-deutsche-bahn) - [Scoring 2000 Products per Request: Performance Pitfalls in Golang](https://www.wearedevelopers.com/videos/2073-scoring-2000-products-per-request-performance-pitfalls-in-golang) - [Tomorrow's cloud data platforms - fully managed database-as-a-service (DBaaS)](https://www.wearedevelopers.com/videos/254-tomorrow-s-cloud-data-platforms-fully-managed-database-as-a-service-dbaas) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Jobs in Tech: The State of the European Market](https://www.wearedevelopers.com/magazine/575-jobs-in-tech-the-state-of-the-european-market) - [Where To Find Software Engineering Jobs](https://www.wearedevelopers.com/magazine/396-where-to-find-software-engineering-jobs) - [Where to Find Entry-Level Software Engineering Jobs](https://www.wearedevelopers.com/magazine/397-where-to-find-entry-level-software-engineering-jobs) - [7 Most Popular Web Developer Jobs in Europe](https://www.wearedevelopers.com/magazine/163-7-most-popular-web-developer-jobs-in-europe) - [The Most Popular IT Jobs on the Market](https://www.wearedevelopers.com/magazine/376-the-most-popular-it-jobs-on-the-market)