Data Engineer_ Spark, Pyspark, github (Developer III - Software Engineering)

UST is seeking a talented Data Engineer to join our growing Data Team in Leeds. In this role, you will play a key part in the design and delivery of high-quality, reliable data solutions that enable insight-driven decision-making and support strategic business objectives.

Working within an Agile delivery model, you will collaborate closely with Product Managers, Data Analysts, Data Architects, and Visualisation Engineers, contributing to a team culture grounded in engineering excellence, continuous improvement, and shared ownership., As a Data Engineer, you will be responsible for building and optimising scalable data platforms that empower analytics and innovation across the organisation. Your key responsibilities will include:

• Designing, developing, and optimising robust data pipelines using PySpark, SparkSQL, and Databricks, ingesting and transforming data from diverse sources.
• Translating business requirements into scalable, performant, and maintainable data solutions, working in close partnership with stakeholders and squad members.
• Engineering solutions to manage structured, semi-structured, and unstructured data, across batch and near-real-time processing scenarios.
• Enabling self-service analytics and insight generation by creating well-structured, discoverable, and trusted data assets.
• Ensuring all engineering deliverables align with data architecture standards and best practices defined by Data Architects.
• Actively contributing to Agile delivery through ceremonies, retrospectives, and continuous improvement initiatives.
• Participating in the Agile Community of Practice, embedding consistent engineering standards, principles, and ways of working.

You are a technically strong and collaborative data engineer, motivated by building high-quality solutions and continuously improving how data platforms operate at scale., * Proven experience designing, building, and supporting data engineering solutions in production environments.

• Strong hands-on development experience with Python and SQL, and ideally PySpark.
• Practical experience with Microsoft Azure data services, including Azure Data Factory, and a solid understanding of cloud-based data architectures such as data lakes, data warehouses, and data vaults.
• Experience working with distributed data processing platforms, particularly Databricks.
• Familiarity with Agile delivery frameworks such as Scrum, Kanban, or Lean.
• Strong grounding in data modelling principles and modern data architecture patterns.
• Experience using GitHub for version control and collaborative engineering workflows.
• Excellent communication and collaboration skills, with a proactive and problem-solving mindset.

Core Skills & Technologies

• Azure Data Factory
• Databricks
• PySpark, SparkSQL
• Data integration and pipeline design
• Cloud-based data architectures
• Agile delivery practices, spark,pyspark,data engineering,github

About UST UST is a global digital transformation solutions provider. For more than 20 years, UST has worked side by side with the world's best companies to make a real impact through transformation. Powered by technology, inspired by people and led by purpose, UST partners with their clients from design to operation. With deep domain expertise and a future-proof philosophy, UST embeds innovation and agility into their clients' organizations. With over 30,000 employees in 30 countries, UST builds for boundless impact-touching billions of lives in the process.