Senior Data Engineer

iO Associates

Charing Cross, United Kingdom

2 days ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Working hours

Regular working hours

Languages

English

Experience level

Senior

Compensation

£ 85K

Job location

Remote

Charing Cross, United Kingdom

Tech stack

API

Artificial Intelligence

Amazon Web Services (AWS)

Business Analytics Applications

Data analysis

Azure

Cloud Computing

Continuous Integration

Data Architecture

Data Governance

Identity and Access Management

Python

Operational Databases

Power BI

Software Engineering

SQL Databases

Tableau

Delivery Pipeline

Prompt Engineering

Spark

GIT

SC Clearance

Pandas

PySpark

Data Pipelines

Databricks

Job description

Hands-on Data Engineer working with public sector clients to design, build, and operate production data platforms. Lead technical architecture across cloud environments, mentor delivery teams, and embed analytics/AI capabilities in agile, client-facing projects., * Translate complex business challenges into scalable data pipelines, models, and analytics solutions

Build/operate production-grade pipelines using SQL/Python/Spark across lakehouses and event-driven systems
Architect semantic models and data visualisations for Business Intelligence platforms
Deploy cloud data/AI solutions (AWS/Azure) with robust CI/CD, testing, and security practices
Integrate ML models, experimentation frameworks, and governance workflows
Deliver client-ready dashboards, insight narratives, and consulting materials
Collaborate across multidisciplinary teams in agile, iterative delivery cycles

Technical Environment Core: Advanced SQL/Python, PySpark/Pandas, Spark orchestration tools Cloud: AWS (Databricks/Redshift equivalents), Azure analytics platforms, Git/CI-CD Architecture: Data lakehouses, event streaming, semantic modelling for BI tools AI/ML: Model APIs, prompt engineering, experimentation frameworks Governance: Data catalogues, lineage tracking, security/access controls, GDPR

Requirements

Expert SQL/Python for data modelling, transformation, and automation
Production data engineering: Spark/PySpark, cloud platforms (AWS or Azure)
Data architecture experience: lakehouses, event-driven systems, BI semantic models
Software engineering: Git workflows, CI/CD pipelines, testing frameworks
Analytics: experimentation design, statistical analysis, insight storytelling
Consulting mindset: stakeholder collaboration, client deliverables, agile delivery
ACTIVE SC clearance required

Desirable