> Markdown version of [/jobs/ext/330572-senior-data-engineer](https://www.wearedevelopers.com/jobs/ext/330572-senior-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Senior Data Engineer - **Company:** Airband Community Internet Ltd. - **Location:** Droitwich Spa, UK (Remote available) - **Experience:** Expert - **Salary:** £70,000.0 - **Contract:** Permanent contract - **Skills:** Application Programming Interfaces (APIs), Artificial Intelligence, Data Analysis, Microsoft Azure, Cloud Computing, Collaborative Software, Information Systems, Databases, Data Architecture, Data Cleansing, Information Engineering, Data Governance, Data Integration, Extract Transform Load (ETL), Database Queries, Distributed Computing Environment, Python (Programming Language), PostgreSQL, Machine Learning, Network Connections, Software Engineering, SQL Databases, Systems Architecture, Data Processing, Data Ingestion, Office365, Apache Spark, Pyspark, Information Technology, Data Management, Restful APIs, Data Pipelines, Databricks - **Published:** June 4, 2026 - **Apply:** https://uk.indeed.com/viewjob?jk=168b5df548ff37e5 ## About the Role Do you have a valid Driving License license?, Do you have experience in Spark?, Do you have a Bachelor's degree?, Core Key Skills * Excellent written and verbal communication * Effective time management and prioritisation * Strong problem-solving skill * Attention to detail and accuracy * Ability to work independently and collaboratively as part of a wider Airband team * Embraces change, is adaptable and is willing to learn * Customer / user-focused mindset * Continuous improvement mindset * Planning and organisational skills * Competent use of Microsoft 365 tools * Embraces diversity * Emotional intelligence and professionalism Technical & Digital Skills * Ability to learn new systems and technologies quickly * Data handling and analysis skills * Understanding of digital ways of working and collaboration tools * Strong proficiency in Python using industry-standard libraries. * Solid experience with Spark and PySpark for distributed data processing. * Strong SQL skills, including performance-focused query design. * Hands-on experience with Databricks on Azure, including platform usage and administration. * Experience integrating data via REST APIs and working with databases such as Postgres. * Experience of automation * Good understanding of data and systems architecture, including connectivity and integration. * Knowledge of operating systems and Azure infrastructure., * Strong SQL skills, including performance-focused query design. * Knowledge of operating systems and Azure infrastructure. Desirable * Bachelor's degree in Computer Science, Software Engineering, Information Systems or a related technical field Experience Essential * Proven experience of Python * Hands-on experience with Databricks on Azure, including platform usage and administration. * Experience working in fast paced environment * Circa 5+ years of experience in data engineering, software engineering, or related technical roles Desirable * Ability to work remotely, on-site and flexibly as require ## Description The Senior Data Engineer is responsible for designing, building, and maintaining scalable data platforms and pipelines that enable reliable analytics, reporting, machine learning, and operational decision-making. This role combines strong software engineering skills deep expertise in data architecture, cloud infrastructure, and data governance. The role will work closely with data analysts, systems, IT, and architecture teams on system and data connectivity, extraction, security, and governance. Responsibilities * Design, develop, and maintain ETL and ELT pipelines. * Write efficient, maintainable Python code using industry-standard libraries, PySpark, and Spark. * Build and manage data ingestion integrations, including APIs and Databricks Lakeflow Connect. * Manage pipelines end to end: setup, monitoring, troubleshooting, optimisation, and issue resolution. * Perform data cleansing, transformation, and validation to ensure high data quality. * Guide the Data Analysts in creating performant SQL queries, views, and materialised views with efficient use of memory and compute. * Present data in analytics-ready formats to support reporting and advanced analysis. * Lead and contribute to Databricks (Azure) platform administration and engineering best practices. * Monitor and optimise the performance of pipelines, tasks, and databases to ensure cost-effective use of compute and storage. * Support system updates, tooling upgrades, and ongoing platform improvements. * Work with systems, IT, and architecture teams on system and data connectivity, extraction, security, and governance. * Implement and maintain appropriate security measures and access controls. * Support compliance with data governance and data management standards. * Collaborate closely with Data Analysts, guiding and mentoring them to support their technical growth. * Support the Manager in team development, process improvement, and platform planning. * Provide data expertise for business-wide projects as they are planned and delivered. * Maintain awareness of new Databricks features, Azure services, and best practices to maximise efficiency and capability. * Demonstrate a strong interest in AI and emerging data technologies, exploring practical applications within and beyond Databricks. * Apply critical thinking, analytical reasoning, and innovative problem-solving to overcome technical challenges ## Related Videos - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [Unlocking the potential of Digital & IT at Vodafone](https://www.wearedevelopers.com/videos/602-unlocking-the-potential-of-digital-it-at-vodafone) - [Kubernetes and Microservices with Multi-Model Databases](https://www.wearedevelopers.com/videos/382-kubernetes-and-microservices-with-multi-model-databases) - [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail) - [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk) - [Fully Remote Software Engineer Jobs](https://www.wearedevelopers.com/magazine/447-fully-remote-software-engineer-jobs) - [Software Engineer Salary London](https://www.wearedevelopers.com/magazine/252-software-engineer-salary-london) - [Software Engineer Salary in The UK](https://www.wearedevelopers.com/magazine/231-software-engineer-salary-in-the-uk) - [The Most Popular IT Jobs on the Market](https://www.wearedevelopers.com/magazine/376-the-most-popular-it-jobs-on-the-market)