> Markdown version of [/jobs/ext/2821873-data-engineer](https://www.wearedevelopers.com/jobs/ext/2821873-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Data Engineer - **Company:** Krystal Clarity - **Location:** Greater London, UK (Remote available) - **Experience:** Experienced - **Contract:** Permanent contract - **Skills:** Amazon Web Services, Microsoft Azure, Bash Shell, Software Quality, Continuous Integration, Information Engineering, Data Governance, Data Integrity, Cursor (Graphical User Interface Elements), Database Queries, Github, Python (Programming Language), Windows PowerShell, Role-Based Access Control, Power BI, Search Technologies, SQL Databases, Data Streaming, Scripting, Azure Data Factory, Snowflake, Data Lakes, Pyspark, Debezium, Information Technology, Maintaining Code, Apache Kafka, Machine Learning Operations, Restful APIs, Terraform, Data Pipelines, Atlassian Bamboo, Databricks - **Published:** September 10, 2026 - **Apply:** https://www.collegerecruiter.com/job/2858100863-data-engineer ## About the Role * Bachelor's or Master's in a STEM subject (Computer Science, Maths, Physics, Engineering, ...). * 2 to 4 years of experience as a data engineer working on Azure Databricks or in the Azure/AWS ecosystem. * Hands-on experience with Python (PySpark strongly preferred) and strong SQL skills is required. * Hands-on experience using AI coding tools (Claude Code, Cursor, Codex or similar) in real development workflows, with judgement on when and how to apply them. * Experience with infrastructure-as-code, ideally Terraform, is required. * Familiarity with CI/CD workflows and tools like GitHub Actions or Azure Pipelines. * Working knowledge of Unity Catalog and data governance fundamentals. * Experience ingesting data from REST APIs and various third-party systems. * Experience with the wider Azure data ecosystem (Data Factory, Storage, Event Hub) is desirable. * Expertise in other relevant technologies is desirable: Lakeflow Connect, Declarative Pipelines, Databricks Apps, Vector Search or MLOps, dbt, Snowflake, Debezium/Kafka/Streaming, Power BI. * Ability to work collaboratively in a small, fast-moving team. * Strong communication and stakeholder-management skills, with the confidence to work with clients face-to-face from day one and the knack for simplifying complex topics. * Resilience to thrive in a dynamic environment, adeptly managing multiple projects. ## Description We are seeking a Mid Level Data Engineer with 2 to 4 years of experience in data engineering, with strong exposure to Azure, Databricks and Python. You'll work closely with senior engineers on real-world client projects, designing, building, and deploying data pipelines in the cloud. This role is perfect for someone looking to sharpen their technical skills, work in a collaborative consultancy environment, and take ownership of impactful work., * Design, build, and optimise scalable data pipelines using Databricks (PySpark, SQL, Delta Lake) on Azure. * Build ingestion from REST APIs, including incremental and near-real-time load patterns, alongside managed connectors such as Lakeflow Connect and Azure Data Factory. * Collaborate with senior engineers to understand requirements and implement solutions for clients. * Work directly with client stakeholders, including face-to-face: gathering requirements, presenting solutions and communicating progress to technical and non-technical audiences. * Write clean, maintainable code in Python, applying best practices for testing and code quality. * Use AI coding assistants (Claude Code, Cursor, Codex or similar) day to day to accelerate development, testing and documentation, while maintaining code quality. * Contribute to CI/CD workflows using GitHub Actions and Azure Pipelines, including automation of infrastructure and deployments. * Use Terraform and Declarative Automation Bundles to manage infrastructure as code, alongside scripting tools (Bash, PowerShell). * Apply strong data governance and quality practices, including Unity Catalog, RBAC and PII handling, to ensure data integrity. * Document solutions, pipelines, and design decisions for future maintainability. * Continuously learn and explore new tools, frameworks, and best practices in data engineering. ## Related Videos - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [Python-Based Data Streaming Pipelines Within Minutes](https://www.wearedevelopers.com/videos/1233-python-based-data-streaming-pipelines-within-minutes) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Innovating Developer Tools with AI: Insights from GitHub Next](https://www.wearedevelopers.com/videos/1268-innovating-developer-tools-with-ai-insights-from-github-next) - [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail) - [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk) - [The Most Popular IT Jobs on the Market](https://www.wearedevelopers.com/magazine/376-the-most-popular-it-jobs-on-the-market) - [Software Engineer Salary London](https://www.wearedevelopers.com/magazine/252-software-engineer-salary-london) - [The 12 Best Jobs for Software Engineers](https://www.wearedevelopers.com/magazine/401-the-12-best-jobs-for-software-engineers) - [Top-Paying Tech Jobs (with Salaries)](https://www.wearedevelopers.com/magazine/372-top-paying-tech-jobs-with-salaries)