> Markdown version of [/jobs/ext/725327-senior-data-engineer](https://www.wearedevelopers.com/jobs/ext/725327-senior-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Senior Data Engineer - **Company:** drivvn - **Location:** Birmingham, UK - **Experience:** Expert - **Salary:** £70,000.0 - **Contract:** Permanent contract - **Skills:** Query Performance, Apache HTTP Server, Automation of Tests, Microsoft Azure, Cloud Engineering, Continuous Integration, Data Validation, Information Engineering, Data Governance, Data Retention, Database Queries, DevOps, Distributed Computing Environment, Python (Programming Language), DataOps, Azure Data Lake, SQL Databases, Data Streaming, Apache Spark, Git, Data Lakes, Pyspark, Apache Kafka, Data Management, Presto, Terraform, Software Version Control - **Published:** June 29, 2026 - **Apply:** https://uk.indeed.com/viewjob?jk=7d52e1b9eae807ee ## About the Role Do you have experience in Terraform?, * Strong, hands-on data engineering experience, with expert-level Python and distributed data processing using Spark / PySpark. * Practical experience of the lakehouse paradigm and modern open table formats - Apache Iceberg (or Delta Lake / Hudi) - including partitioning strategy, schema evolution and table maintenance. * Experience building and operating data platforms on a major cloud, ideally Microsoft Azure (Azure Data Lake Storage), within cloud-native, containerised environments. * Proficiency with distributed SQL query engines such as Trino / Presto / Starburst, and strong SQL skills at scale. * Experience modelling data and enabling self-serve BI (Metabase or similar). Experience with a semantic / metrics layer such as cube.dev, dbt or LookML is a strong advantage. * Solid software engineering fundamentals: version control (Git), automated testing and CI/CD within container-based environments. Infrastructure-as-code (e.g. Terraform) is desirable. * A working understanding of data observability, data quality, lineage and governance, with an appreciation of security and GDPR considerations. * Specialised expertise in at least one key area: streaming and event processing (e.g. Kafka), advanced data modelling, performance and cost optimisation, or data governance. * Proven ability to work independently with minimal oversight and make pragmatic architectural trade-offs, thriving in a startup environment that values adaptability and proactive problem-solving. * A positive, proactive and energetic attitude, strong communication skills, and a passion for transforming the digital automotive buying experience through data! ## Description * Own the lakehouse. Design, build and operate scalable batch and streaming data pipelines in Python and Spark (PySpark), ingesting and transforming data into well-modelled Apache Iceberg tables on Azure Data Lake Storage. * Architect for scale and cost. Make sound decisions on table design, partitioning, schema evolution and Iceberg table maintenance (compaction, snapshot expiry), balancing query performance against storage and compute cost. * Deliver the query and semantic layers. Optimise the Trino query layer and build out the cube.dev semantic layer so that consistent, governed metrics are available for self-serve analytics. * Enable analytics and BI. Partner with stakeholders to model data and surface reliable, performant insight through Metabase, reducing the distance between a question and a trustworthy answer. * Embed observability and quality. Build monitoring, alerting, data quality checks and lineage into the platform. We believe in a true DevOps culture where engineers run what they build. * Set the engineering standard. Establish best practices for the data function - version control (Git), automated testing, CI/CD, infrastructure-as-code and clear documentation. * Champion governance and security. Ensure data is handled securely and in line with GDPR and our obligations across UK and European markets, including appropriate access controls and data retention. * Partner across the business. Work closely with product, engineering and commercial teams to understand data needs and translate them into robust, reusable data products. * Lead technically. Evaluate trade-offs, run proofs of concept, advocate for the right technologies, and provide the technical direction for data engineering as the team grows. ## Related Videos - [From DevOps to Scaled DevOps: How We’re Rebuilding Continuous Delivery as a Platform](https://www.wearedevelopers.com/videos/100018-from-devops-to-scaled-devops-how-we-re-rebuilding-continuous-delivery-as-a-platform) - [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [DevOps Maturity Check – a way to balance autonomy and alignment](https://www.wearedevelopers.com/videos/58-devops-maturity-check-a-way-to-balance-autonomy-and-alignment) - [Git for Code Reviews](https://www.wearedevelopers.com/videos/429-git-for-code-reviews) - [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [How software is steering vehicle technology](https://www.wearedevelopers.com/magazine/515-how-software-is-steering-vehicle-technology) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Dev Digest 121 - AI goes offline](https://www.wearedevelopers.com/magazine/456-dev-digest-121-ai-goes-offline) - [Dev Digest 120 - Apple and peers](https://www.wearedevelopers.com/magazine/455-dev-digest-120-apple-and-peers)