Lead Data Engineer

As our Lead Data Engineer, your mission is to architect, scale, and maintain the backbone of our data ecosystem. You will transform massive volumes of raw signals into actionable intelligence, ensuring our infrastructure is as agile as our growth strategy. You will bridge the gap between high-level data strategy and hands-on technical excellence.

Your Responsibilities & Key Deliverables

Data Infrastructure Design and Optimization

• Architectural Leadership: Lead the design, implementation, and optimization of distributed data architectures to support massive Batch & Streaming pipelines.
• Scalability & Performance: Ensure the scalability, security, and performance of the infrastructure, maintaining high system availability (99.9%+) for analytics platforms.
• Efficiency: Optimize storage solutions to balance high-speed processing with cost-efficiency.
• AI Integration: Collaborate with software engineers and data scientists to seamlessly integrate AI-driven models into production data workflows.

Technical Leadership and Team Management

• Mentorship: Lead and mentor a team of 2 data engineers, fostering a culture of continuous improvement, growth, and technical excellence.
• Standards & Quality: Set high standards for code quality, documentation, and rigorous peer reviews.
• Execution: Oversee project execution, delegate responsibilities, and act as the technical point of escalation for complex data-related roadblocks.
• Best Practices: Guide technical decisions and promote industry best practices in data engineering (CI/CD, DataOps).

Collaboration and Data Governance

• Cross-Functional Alignment: Work closely with product managers, developers, and analytics teams to define data needs and ensure alignment with business objectives.
• Reliability: Implement robust monitoring, alerting, and automated testing (Data Quality) to ensure the integrity of the data team's output.
• Compliance: Ensure data security and GDPR compliance across all international jurisdictions and workflows.

Innovation and Problem Solving

• Emerging Tech: Identify opportunities to leverage new technologies or improve existing workflows to optimize data processing.
• Complexity Simplification: Maintain a deep understanding of data trends and possess the ability to simplify complex problems into executable tasks.
• Generative AI Adoption: Encourage and champion the use of Generative AI tools across the team to accelerate development cycles, automate repetitive tasks, and enhance data exploration capabilities.

Do you have experience in Scala?, * Experience: 6+ years in Data Engineering, with at least 2 years in a leadership or "referent" role within fast-paced environments.

• Database Mastery: Expert proficiency in distributed data systems, clustering, advanced table types, and materialized views.
• Optimization: Skilled in designing table sorting keys and fine-tuning database performance.
• Stack Mastery: Solid programming skills in Python, Golang, and Scala/Java, with deep expertise in Kafka for streaming/batch data pipelines.
• Infrastructure: Hands-on experience with on-premise infrastructure (own datacenter) managed by Kubernetes, and familiarity with monitoring stacks such as Prometheus.
• Data Storage: Strong knowledge of column-oriented databases designed for high-volume data processing, vector databases (e.g. Qdrant), and SQL/NoSQL technologies.
• Tooling: Comfortable working with project tracking tools (Flyspray) and applying CI/CD & DataOps best practices.
• AI & Innovation (Nice to have): Interest or hands-on experience leveraging Generative AI tools in daily engineering workflows to boost productivity and innovation.

Your mind set to share our adventure

• You want to make an impact and move things forward collectively. Does hearing phrases like "Yes, but we've been doing it this way for years..." make your hair stand on end? We feel the same way: progress is made by questioning what already exists.
• You solve problems pragmatically and analytically.
• ️ You're looking for a fast-moving environment where your agility will be an asset. The 80-20 (Pareto) principle holds no secrets for you.
• Your ability to listen encourages you to challenge and improve yourself on an ongoing basis.

• Discussions about your driving forces, your ambitions and our Sparteo mindset will be held with a member of our Talent Acquisition team.
• Analytical and business logic tests
• Discussions with one or more members of the Sparteo team, including your future manager
• Taking up references

Our recruitment process is mainly conducted by videoconference; however, certain stages may require a face-to-face meeting.

We like to play as a team, especially when things get tricky. This is why we’re not only holding on to the awesome people already onboard, but also continue to welcome and search for new team members. Just like all of our other activities, the job interviews, onboardings and bootcamps continue as planned, we’ve simply switched to 100% remote work and moved all our interactions online. What’s more, everyone who joins Spartez these days can work from home until the COVID-19 lockdown is over.