Data Engineer

Location: The primary Met Location for this role is either Sidcup, Hendon, Bow or Lambeth. Travel across the Met Estate, to attend meetings as required by the business, is expected.

"Keep checking Met Careers on a daily basis for new and exciting opportunities being released to join DDaT!"

Policing London is a hugely complex challenge that evolves every day. To make sure the Met is up to the task, we need the right IT and systems across our organisation. As a Data Engineer, you will help to enable us to do our best for the London area. Join us and play your part in making London the safest capital in the world and delivering 'More Trust, Less Crime and High Standards'.

As a member of the Met, you will also enjoy great benefits including annual leave, civil service pension arrangements, maternity/paternity support, as well as being part of a continuous learning culture.

Digital, Data and Technology (DDaT)

The mission of Digital, Data and Technology (DDaT) is to provide the Met with information (and specifically data), communications and technology to help frontline officers and Met staff do their jobs more efficiently, to deliver the New Met for London plan. In other words, DDaT is about "passionately delivering trusted technology for precise data driven policing".

As well as keeping up to speed with developments in technology and making recommendations for new equipment and systems that will help modernise policing, DDaT also comprises the Data Office which:

• Owns and drives the data compliance and analytics agenda.
• Brings together the accountabilities that are currently dispersed across the Met business areas.
• Consolidates and further develops existing Met data and insight services.
• Enables data sharing and collaboration to serve the public and external partners.

DDaT is committed to creating an inclusive working environment. We support hybrid working, and the hiring manager will be happy to discuss arrangements for this role.

Job Purpose

Data Engineers provide ETL (Extract, Transform, Load) capability on Met data systems to deliver high-quality data, including automation, for onward processing and consumption. They are data experts and use cutting-edge tools and systems to provide data pipelines that underpin the data-driven culture the Met is developing.

Occasional overtime to support Operational Policing will be required.

Key Role Responsibilities

• Design, development, and maintenance of robust, scalable data pipelines to ensure efficient data flow from various sources to our data warehouses and analytics platforms.
• Implementing data validation and cleansing procedures to maintain high data quality and adhering to data governance policies and standards.
• Administration and optimisation of databases, including tasks such as indexing, partitioning, and query optimisation to enhance performance.
• Developing and managing ETL/ELT processes to prepare data for analysis and reporting, ensuring that data is transformed correctly and loaded into the appropriate systems using SQL Server, AlterYX, FME, azure Data Factory, Data Bricks or equivalent.
• Building and maintain data pipelines in Azure and/or on prem.
• Supporting the development of Junior Data Engineers.
• Collaborating with data analysts, data scientists, and other stakeholders to understand their data needs and provide the necessary support, including troubleshooting and resolving data-related issues.
• Creating and maintaining detailed documentation of data processes, workflows, and systems, and generating reports to provide insights into data operations and performance.
• Responsible for business liaison, requirements gathering, negotiating and agreeing deliverables with other teams/units.
• Dealing with and escalating incidents and problems related to technical infrastructure.
• Monitoring performance of existing databases/data warehouses/data lakes to identify areas of improvement or optimisation opportunities.
• Collaborative working to ensure an understanding of other units' objectives and promote a joined-up approach to ensure an integrated service across.
• Occasional overtime to support Operational Policing., This post requires access to the most sensitive intelligence material on a daily basis. Applicants must hold or be prepared to undergo National Security Vetting (NSV) Security Check (enhanced) (SC(e)) level before taking up the post., mps_a-new-met-for-london_external-summary-rgb-accessible_online-final.pdf - 590KB Opens in a new window Disability Confident Statement

The Met is committed to being an equitable (fair and impartial) and inclusive employer for disabled people, striving to have a diverse and representative workforce at all levels. We encourage applications from people from the widest possible range of backgrounds, cultures and experiences. We particularly welcome applications from people with disabilities and long-term conditions, ethnic minority groups and women.

As a Disability Confident Leader, the Met has committed to making disability equality part of our everyday practice. We ensure that people with disabilities and those with long term conditions have the opportunities to fulfil their potential and realise their aspirations.

The Met is committed to making reasonable adjustments to the recruitment process to ensure disabled applicants can perform at their best. If you need any reasonable adjustments or changes to the application and recruitment process, we ask that you include this information within your application form. All matters will be treated in strict confidence.

Please note, if you are applying for a police officer role or to become a police community support officer (PCSO) or designated detention officer (DDO), there is a minimum requirement that you must pass a job-related fitness test (JRFT) at point of entry. This does not apply to police staff roles. Find out more about police fitness standards.

Do you have experience in Test automation?, Do you have a Bachelor's degree?, Technical Skills

• BSc in STEM subject OR 3+ years of equivalent practical experience in information systems or related field.
• Working knowledge of SQL (queries, functions, stored procedures, views, etc.).
• Working experience of designing and implementing in Azure Data Factory, Data Bricks and/or other cloud platforms.
• Working knowledge of relational databases and data warehouses. Hands-on experience with SQL database design.
• Working experience with ETL/ELT tools such as AlterYX/SSIS/FME/Python or similar.
• Working experience of logging, analysing and managing problems independently to identify and implement appropriate solutions.
• Working experience writing batch/power shell script.
• Working experience in communicating ideas and analyse results effectively both verbally and in writing to a variety of stakeholders.
• Self-motivated to achieve results and develop in the role.

Communication and Planning

• Build effective working relationships with colleagues and peers internally and externally.
• Influence and negotiate with stakeholders, recognising stakeholder needs and escalating complex and sensitive issues.
• Respond proactively to requests and work effectively with others to undertake focused activity to meet and deliver priorities.
• Experience of mentoring junior members of staff.

Desirable Criteria:

• Working experience with Version control (e.g. GIT).
• Working experience with automated testing.
• Previous work using the Agile Methodology.
• Working knowledge of geospatial data processing.
• Working experience with machine learning.
• Working experience with Excel and Business Intelligence tools (Power BI, Power Query, Tableau, etc.).
• Working experience working in a SharePoint environment.
• Working experience with Java/Scala or other programming languages.

Required Languages English

The salary is broken down as £50,531 basic salary, which will increase annually until you reach the top of the scale £55,103 Plus, a location allowance of £1,928 and a non-pensionable allowance of £1,000.