Data Engineer

Halian .

Norwich, UK

11 days ago

Apply on www.collegerecruiter.com

Prepare application

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Working hours

Regular working hours

Job source

www.collegerecruiter.com

Tech stack

Application Programming Interfaces (APIs) Artificial Intelligence Data Analysis Automation of Tests Microsoft Azure Business Systems Cloud Database Databases Continuous Integration Information Engineering Data Governance Data Integration

+23 more

Extract Transform Load (ETL) Data Transformation Data Mining Data Warehousing Relational Databases Database Queries Python (Programming Language) Machine Learning SQL Azure Power BI Cloud Services SQL Databases Tableau (Software) Workflow Management Systems Data Processing Azure Data Factory Snowflake Git Microsoft Fabric Data Lakes Software Version Control Data Pipelines Databricks

Job description

Building, maintaining, and improving reliable data pipelines across multiple business systems
Extracting, transforming, and loading data into centralised analytical environments
Integrating data from multiple internal systems and external sources
Investigating and resolving data quality, consistency, and integrity issues
Identifying discrepancies between systems and establishing reliable sources of truth
Developing reusable data transformation and processing workflows
Writing efficient SQL queries for data extraction, transformation, and analysis
Using Python to automate data processing and engineering tasks
Supporting the development and maintenance of data warehouses and analytical data structures
Monitoring data pipelines and investigating failures or unexpected results
Working with stakeholders to understand data requirements and translate them into technical solutions
Documenting data sources, transformations, pipelines, and processes
Creating trusted datasets to support reporting, analytics, Data Science, and future AI initiatives
Contributing to improvements in data governance, quality, and accessibility across the organisation

Requirements

Commercial experience in Data Engineering, Analytics Engineering, BI Engineering, or a closely related role
Strong SQL skills and experience working with relational data
Good Python skills for data processing and automation
Commercial experience building or maintaining data pipelines
Experience integrating data from multiple sources
Practical experience identifying and resolving data quality and consistency issues
Understanding of data modelling and analytical data structures
Experience with ETL or ELT processes
Experience working with databases, data warehouses, or cloud data platforms
Strong problem-solving skills and a methodical approach to investigating data issues, * Experience with Microsoft Azure and Azure data services
Knowledge of Azure Data Factory or similar orchestration tools
Experience with Databricks, Snowflake, or Microsoft Fabric
Experience with Azure SQL
Knowledge of dbt or similar data transformation frameworks
Experience working with data lakes or cloud data warehouses
Experience integrating data through APIs
Knowledge of Power BI, Tableau, or similar visualisation tools
Understanding of data quality and governance frameworks
Experience with CI/CD or automated testing for data pipelines
Experience supporting machine learning or AI projects
Basic understanding of statistics, machine learning, or Data Science
Experience using Git or another version control system
Excellent communication skills and the ability to work with technical and non-technical stakeholders