Data Engineer

The purpose of the role is to design, build, and manage scalable data solutions focusing on the ingestion, transformation, and management of large datasets across and real-time analytics environments. This includes developing robust and efficient data pipelines, ensuring secure and well-governed data access, and optimising system performance.

The role also involves analysing business requirements to ensure data is accessible and usable for decision-making, while carrying out platform administration, monitoring, and maintenance to support reliable and secure operations.

MAIN RESPONSIBILITIES:

• Develop and maintain data infrastructure, data warehouses, and big data processing systems.

• Build data pipelines that clean, transform, and aggregate data from disparate sources.

• Build solutions for automated ingestion and data versioning i.e. Time Travel.

• Provide technical skills and support to other areas for Fabric components under the team's control.

• Ensure data accuracy, reliability, security, and compliance with policies.

• Work with analysis engineers, analysts, developers and business teams to understand needs and deliver solutions.

• Re-engineer manual processes for scalability and efficiency.

• Deliver project tasks to agreed timescales and provide technical expertise to ensure project success.

• Ensure business continuity strategy is appropriate for purpose and is always delivering.

• To perform established procedures to accredited standards on an ongoing basis.

• Contribute to the planning, research, recommendation, and implementation of current and future IT strategies.

Essential

• Demonstrable experience of Spark (PySpark) and SQL.

• Demonstrable experience of Fabric components.

• Demonstrable experience of working on enterprise environments.

• Experience of designing architectural Fabric processes.

• Experience of Azure Data Factory.

Desirable

• DP-700 (Microsoft Fabric Data Engineer) certification.

• Demonstrable experience of data visualization/exploration tools.

• Understanding of Project Life Cycles.

• Able to deliver innovative solutions for problems encountered within the scope of Data Engineer duties.

• Continuous improvements to existing team processes / monitoring.

• High attention to detail and ability to work under pressure.

IMSERV is one of the UK's leading data collection and energy metering experts, delivering award winning services to more customers in more places, meeting industry targets and becoming a benchmark for excellence. We offer a range of specialist metering technology for electricity, gas, and water along with highly accurate energy data collection services. All this is wrapped up with an easy-to-view online data management analysis and reporting software., Our people are our main asset. We strive to ensure they remain happy, competitive, and fulfilled - helping to propel our business forward and ensure we remain customer-centric and competitive. We are proud to remain the UK's leading and growing energy data collection and meter operations service provider. Diversity and inclusion have long been at the heart of IMSERV's success. As we continue our growth, our focus remains on ensuring that equality, diversity, and inclusion remain central to our business and recruitment practices. We recognise that we operate in an industry in which there has traditionally been a lack of diversity, and we are keen to encourage applications from as inclusive a group as possible. We recognise that a balanced workforce encourages collaboration and innovation, promotes entrepreneurship and a feeling of ownership. These are the key drivers of our business, that our customers really look to us for. (Please note that we reserve the right to close this position before the expiry date)