Data Engineer

As a Data Engineer, you will focus on designing, developing, and optimising data ingestion, transformation, and serving layers across Azure Databricks, Delta Lake, and Databricks SQL. Your work will enable reliable data products and insights for stakeholders, and you’ll collaborate with data architects, analysts, and product teams to deliver robust ELT/ETL pipelines, model data for performance and usability, and apply best practices in security and data quality.

Working within the Data & AI team, you will contribute to developing our new Renewables Data Platform on Databricks and migrating existing data solutions from Synapse. This role is hands-on and collaborative, combining pipeline development, data modelling, and quality and security practices to deliver well-structured, reliable data solutions aligned with business needs.

You Will

• Build and maintain metadata-driven pipelines using Azure Databricks to ingest and transform batch and streaming data, ensuring data quality and scalable processing.
• Develop Delta Lake layers following bronze/silver/gold patterns, and model data optimised for BI and analytics, including Power BI integration through Databricks SQL.
• Use Databricks pipelines, jobs, notebooks, PySpark and SQL to perform large-scale transformations and implement data quality, monitoring and performance best practices
• Support Governance, Security & Compliance through Unity Catalog lineage, cataloguing and code-managed permissions, applying data protection controls and contributing to SLAs, SLOs and data contracts.
• Collaborate across DevOps and product teams to refine requirements, deliver version-controlled solutions through CI/CD, and continuously develop your skills with evolving tools and technologies.

• Experience delivering data solutions in a growing business environment with varied data needs, and the ability to manage and prioritise workload effectively.
• Strong communication skills, able to engage clearly with technical and non-technical stakeholders across the IM Apps & Data team and wider business.
• Adaptability and a proactive mindset, with a willingness to learn, develop, and approach challenges with a positive attitude.
• Proven data engineering experience, including building metadata-driven batch and streaming pipelines using Azure Databricks, and developing Delta Lake architectures using bronze/silver/gold patterns.
• Technical capability in large-scale data processing, including Python, PySpark and SQL for transformations and data quality, and designing performant BI/analytics data models with Power BI integration.

Salary: Salary: £36,200- £54,400 + Performance-related bonus and a range of benefits to support your finances, wellbeing and family.

SSE has a bold ambition - to be a leading energy company in a net zero world. We’re building the world’s largest offshore wind farm. Transforming the grid to provide greener electricity for millions of people and investing over £20 billion in homegrown energy, with £20 billion more in the pipeline.

Our IT division powers growth across all SSE business areas by making sure we have the systems, software and security needed to take the lead in a low carbon world. They provide expertise, advice and day-to-day support in emerging technologies, data and analytics, cyber security and more.