> Markdown version of [/jobs/ext/639065-remote-senior-data-engineer-data-quality-pyspark-databricks](https://www.wearedevelopers.com/jobs/ext/639065-remote-senior-data-engineer-data-quality-pyspark-databricks). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Remote Senior Data Engineer (Data Quality, PySpark, Databricks) - **Company:** PEXA Group - **Location:** Perth, UK (Remote available) - **Experience:** Expert - **Salary:** £65,000.0 - £75,000.0 - **Contract:** Permanent contract - **Skills:** Airflow, Computer Programming, Data Governance, Data Infrastructure, Data Transformation, Data Structures, Database Queries, Identity and Access Management, Meta-Data Management, Performance Tuning, Strategies of Testing, Pyspark, Semi-structured Data, Production Code, Data Delivery, Data Pipelines, Databricks - **Published:** June 25, 2026 - **Apply:** https://find.jobs/jobs-near-me/remote-senior-data-engineer-data-quality-pyspark-databricks-perth-perth-kinross/2837743384-2/ ## About the Role * Extensive hands-on experience with PySpark, including performance optimisation * Deep working knowledge of Databricks (development, architecture, and operations) * Proven experience working with Airflow for orchestration * Proven track record in managing and securing PII data, with GDPR compliance in mind * Experience in data governance processes; Alation experience preferred, but similar toolswelcome * Strong SQL skills and experience optimising complex queries * Strong experience in handling and transforming semi-structured data * High competency in programming, with a focus on clean, efficient, and production-quality code * Demonstrated ability to work with stakeholders to understand data needs and guide the validation and delivery process * Experience implementing and maintaining data quality tests and monitoring solutions * Strong verbal and written communication skills * Ability to think holistically about data reliability and how it serves business decisions ## Description Great question! We pride ourselves on attracting, developing and retaining a diverse range of people in an equally diverse range of roles and specialisms - who together achieve outstanding results. Our transparent approach and open-door policy make Smoove a great place to work and as our business expands, we are looking for ambitious, talented people to join us. We are looking for a technically proficient Senior Data Engineer to join our growing Data team. Your primary focus will be on ensuring data quality, stability, and reliability - from the moment data arrives in its rawest form to when it is used in decision-making dashboards and customer-facing reports. You will optimise the transformation pipeline from start to finish, guaranteeing that datasets are robust, tested, secure, and business-ready. Our data platform is built using Databricks, with data pipelines written in PySpark and orchestrated using Airflow. You will be expected to challenge and improve current transformations, ensuring they meet our performance, scalability, and data governance needs. This includes work with complex, nested data structures, ensuring they are reliably parsed and transformed. Experience in managing sensitive data (PII) and implementing GDPR policies is required. You'll work closely with analysts, engineers, and business stakeholders to ensure that datasets are not only accurate but also trusted. You will collaborate with product and engineering teams to incorporate data from new products into our core business datasets, ensuring that these new sources meet our data standards and are quickly usable for business intelligence. You'll help put controls in place - such as access policies, metadata layers, and automated data quality checks - to ensure long-term stability. Experience with a data governance platform like Alation is desirable. While predominantly remote / home based the team meet up to 20-25 days per year for meaningful collaboration in either Leeds or Thame., * Ensure end-to-end data quality, from raw ingested data to business-ready datasets * Optimise PySpark-based data transformation logic for performance and reliability * Build scalable and maintainable pipelines in Databricks and Airflow * Implement and uphold GDPR-compliant processes around PII data * Collaborate with stakeholders to define what "business-ready" means, and confidently sign off datasets as fit for consumption * Put testing strategies in place to detect data issues early and often * Contribute to access management, metadata management, and wider data governance practices * Help shape our approach to reliable data delivery for internal and external customers ## Related Videos - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail) - [Cutting LLM Costs Without Cutting Quality: How to Beat Proprietary LLMs with Fine-Tuned Open Source](https://www.wearedevelopers.com/videos/100151-cutting-llm-costs-without-cutting-quality-how-to-beat-proprietary-llms-with-fine-tuned-open-source) - [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt) - [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk) - [Fully Remote Software Engineer Jobs](https://www.wearedevelopers.com/magazine/447-fully-remote-software-engineer-jobs) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [The 12 Best Jobs for Software Engineers](https://www.wearedevelopers.com/magazine/401-the-12-best-jobs-for-software-engineers)