Data Engineer

GIS Recruiter Ltd

Birmingham, UK

7 days ago

Apply on www.adzuna.co.uk

Prepare application

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Compensation

£57,102.0

Working hours

Regular working hours

Job source

www.adzuna.co.uk

Tech stack

Microsoft Azure Big Data Cloud Computing Data Integration Extract Transform Load (ETL) Data Transformation Data Migration Oracle (Applications) Azure Data Lake Microsoft SharePoint Oracle Fusion Middleware Data Streaming

+7 more

Systems Integration Data Ingestion Azure Data Factory Informatica Powercenter Data Pipelines Legacy Systems Databricks

Job description

Design, build and maintain data migration and ingestion pipelines within Azure
Support the setup and development of the Azure Data Lake environment
Build integrations between source systems, Azure Data Lake and Databricks
Develop and manage pipelines using Azure Data Factory (ADF)
Extract and migrate data from source systems including Oracle, Oracle Fusion, Informatica, SharePoint and additional enterprise and legacy systems as the programme develops
Help define scalable data ingestion, transformation and migration patterns for future source systems
Support data quality, validation and troubleshooting throughout the migration process
Work closely with technical and business stakeholders to understand source data, target structures and transformation requirements
Help improve the overall maturity of the Azure data environment, introducing appropriate engineering standards and best practices

Technologies:

Azure
Databricks
ETL
Informatica
Support
Oracle
SharePoint
Cloud

We are supporting a data migration and integration programme within a relatively low maturity Azure environment. The role focuses on building scalable data flows into Azure Data Lake and Databricks, integrating data into a proprietary data transformation solution where applicable, and working with a wide range of legacy and enterprise source systems. We are looking for someone who can help us develop the Azure data environment while partnering closely with technical and business stakeholders.

Requirements

Experience as a Data Engineer supporting data migration and integration programmes
Experience designing and building robust data pipelines
Strong Azure Data Engineering experience
Hands-on experience with Azure Data Factory (ADF)
Experience with Azure Data Lake and Databricks
Experience building ETL/ELT pipelines
Experience with large-scale data migration and integration
Experience working with Oracle and/or Oracle Fusion data sources
Experience extracting and integrating SharePoint data
Experience working with multiple and varied source systems
Experience with data transformation, mapping and validation
Experience designing data pipelines in environments where architecture and processes are still developing