Data Engineer

This is an entirely forward-looking role. While there is an active platform modernisation underway (the engineering team is migrating a .NET platform from SQL Server to PostgreSQL and containerising for Kubernetes), this role's primary focus is building new data pipelines and analytics infrastructure, not maintaining legacy systems. That said, collaboration with the platform engineering team is important, as the data pipelines need to ingest from both the existing SQL Server estate and the new PostgreSQL databases., * Weaviate or other vector database experience - currently used for semantic search in the AI agent layer

Why Join?

• Greenfield data platform - the entire data engineering practice needs to be built from scratch. A lakehouse POC is being evaluated but the production architecture is theirs to define - patterns, tooling, and platform decisions from day one

• Direct AI impact - every pipeline they build directly unblocks AI agent capabilities. The KAI copilot is live in production and evolving towards agentic AI - autonomous agents that take actions, orchestrate workflows, and operate across systems. This role provides the data foundation that makes agentic capabilities possible

• Event streaming architecture - opportunity to design and implement a Kafka-based event backbone, shaping the platform's real-time data architecture

• Manufacturing AI - opportunity to work at the intersection of industrial operations and generative AI, a domain with massive untapped potential. Rich, real-world data: quality inspections, KPIs, maintenance records, safety observations, process parameters across global manufacturing sites

• Small team, high autonomy - the Innovation Hub operates with startup-level autonomy inside a funded, growing company

• Leadership trajectory - the data function is critical to the platform and will grow. As an early hire, there is an open career path that will grow with the function - earned, not guaranteed Similar jobs, GCP - Senior Data Engineer Location: Fully remote - UK Contract length: Initial 6 months Working frequency: Part-time Rate: £450-£500 per day IR35: Outside IR35 Tech: GCP / Datastream / SQL / Python SR2 is supporting a client delivering a large-scale enterprise data..., A global financial institution is seeking a Senior Market Data Lawyer to join its legal team in Birmingham. This role offers the opportunity to advise on complex market data and commercial arrangements within an investment banking environment while playing a key part in a..., A leading global investment banking firm is seeking a Data Engineering Manager to lead their Wealth Data Engineering team in Birmingham. This role involves team leadership, designing robust data platforms, and ensuring data is AI-ready. Ideal candidates have 10+ years in..., A leading global investment banking firm is looking for a Data Engineering Manager in Birmingham. In this pivotal role, you will lead a talented team and manage data platforms to ensure they are AI-ready. The ideal candidate will have over 10 years of experience in data...

• Python - primary language for data pipelines, scripting, and integration with the AI stack. This is a Python-first team

• SQL (SQL Server + PostgreSQL) - deep experience required across both engines. The platform runs a large-scale multi-tenant database estate with per-client schemas (local, global, corporate). The new platform targets Aurora PostgreSQL, with several modules already migrated. Pipelines must bridge both.

• Databricks / Apache Spark - a lakehouse POC is being evaluated with Unity Catalog and Delta Sharing via AWS PrivateLink. Experience with Databricks or equivalent analytics platforms is important for assessing and scaling the right approach

• Apache Kafka - event streaming experience is essential. The platform is evaluating Kafka as the strategic backbone for ordered event logs, consumer group replay, and guaranteed partition ordering as the microservices fleet scales

• ETL/ELT pipeline design - building and maintaining pipelines into a medallion architecture (bronze/silver/gold layers) on Databricks or equivalent analytics platform

• Delta Lake - understanding of lakehouse table formats, time travel, and ACID transaction patterns

• AWS data services - RDS, S3, Lambda, Glue or equivalent. The platform runs entirely on AWS * Linux / CLI proficiency - must be comfortable working

AI-Assisted Development Methodology ->

Solvace is transitioning towards AI-assisted development as a core engineering practice. Candidates should demonstrate:

• Hands-on experience with AI coding tools - Claude Code, OpenAI Codex, GitHub Copilot, Cursor, or similar. We're looking for engineers who have integrated these tools into their professional workflow, not just experimented casually

• Spec-driven development - ability to write clear technical specifications that can be used to drive both human and AI-assisted implementation, with strong evaluation criteria and test coverage

• Portfolio evidence - professional projects or side projects that demonstrate AI-assisted development practices. Contributions to or experimentation with emerging projects like OpenClaw are a strong signal

• Testing and evaluation rigour - experience building robust test suites, automated quality gates, and evaluation frameworks that ensure AI-assisted code meets production standards

Nice-to-Have ->

• Apache Flink - for real-time stream processing as the platform moves from batch to streaming analytics

• Apache Iceberg - experience with open table formats alongside or as an alternative to Delta Lake

• Go or Rust - valued as evidence of strong backend engineering and systems thinking, even though Python is the primary language

• Multi-tenant data architectures - the platform runs per-client schemas with tenant isolation requirements. Understanding tenant isolation patterns is a significant advantage

• Terraform / IaC - infrastructure is managed via Terraform; the Databricks POC has been codified in Terraform

• dbt or similar transformation framework

• Manufacturing / industrial data domain experience (sensor data, quality metrics, OEE, SPC)

• CDC (Change Data Capture) - the platform is building CDC pipelines for the SQL Server to PostgreSQL transition

Job Description Senior Data Engineer £65,000 - £75,000 Manchester or Leeds (Hybrid) This is an opportunity to join a growing, modern banking organisation that is investing heavily in its data platform as it scales. You will have a real impact on how data is engineered,..., Job Description Data Engineer ?? Birmingham, UK | Hybrid At Compass Group UK&I, we're more than just the UK's leading contract catering company - we're driving digital transformation across the business. Our Digital & Technology team is at the heart of this journey,..., Palantir Data Engineer Our client is a Leading Digital Transformation Consultancy who have been named a Times Top 100 Company To Work For for 7 consecutive years. As a Palantir Data Engineer, you will join a highly regarded digital, data, and technology consultancy..., Agilis are currently working exclusively with a key client who are a leading technology consultancy in their search for a Data Engineer. This is a fantastic opportunity to join a fast growing, forward thinking company and helping them take their Data engineering to the next..., Job Description Data Engineer (AWS) - Remote - £80,000 An exciting, high-growth technology scale-up is seeking a Data Engineer to help evolve and strengthen its modern data foundations. This organisation builds a global, API-first platform used by major enterprise brands...