Data Engineer
Role details
Job location
Tech stack
Job description
We are embedding agility in our ways of working, building capability for a digital SSE of the future, and meeting the needs of our customers in a new way. SSE IT is building its Agile capability and continues to increase the proportion of its delivery which utilises Agile methodology and tools. As a result, it is desirable role holders would have Agile experience and demonstrate an interest in championing it across the business.
The Data Engineer role is responsible for the creation and maintenance of analytics infrastructure, providing the foundation for data needs. Data Engineers are responsible for developing, constructing, maintenance and testing of architectures, such as databases and large-scale processing systems, implementing data flows to connect operational systems, and writing and executing ETL scripts and codes. Ensuring stability, robustness and resilience of data products designed and built and making changes to these products when
necessary. Data Engineers support continuous improvement of standards and provide leadership to develop other team members by providing technical guidance alongside other data engineering functions for their customers. The bulk of the Data Engineer's work will be in building, managing and optimising data pipelines and data model used by our Data Scientists. They will also work closely with Data Management teams on governance and security as well as business stakeholders around projects and IT teams to deliver these data pipelines and models effectively into production.
You will
- Be involved in the design, engineering, improvement and productionisation of enterprise level data solutions using appropriate Azure data related resources
- Use the data team's standards to ensure appropriate code quality and that agreed patterns and practices are being followed
- Assist other data professionals (data scientists, data analysts and other data consumers)
- Be involved with the maintenance of a curated data model for use by data professionals
- Comply with data governance and SSE Security Standards
Requirements
- Proficient in building and optimising ETL pipelines in Databricks using PySpark
- Data management, engineering and analysis experience
- Experience in automated data driven testing and analysis
- A systematic, disciplined and analytical approach to problem solving
- Fully conversant with Agile and DevOps development methodology and concepts as applied to data driven analytics projects
Benefits & conditions
Enjoy discounts on private healthcare and gym memberships. Wellbeing benefits like a free online GP and 24/7 counselling service. Interest-free loans on tech and transport season tickets, or a new bike with our Cycle to Work scheme. As well as generous family entitlements such as maternity and adoption pay, and paternity leave.