SC Cleared Senior Data Engineer
Sanderson Recruitment Plc
Charing Cross, United Kingdom
2 days ago
Role details
Contract type
Temporary contract Employment type
Full-time (> 32 hours) Working hours
Regular working hours Languages
English Experience level
Senior Compensation
£ 140KJob location
Charing Cross, United Kingdom
Tech stack
API
Agile Methodologies
Confluence
JIRA
Azure
Batch Processing
Code Review
Continuous Integration
Data as a Services
Information Engineering
Data Systems
Data Warehousing
DevOps
Hive
Python
Power BI
SQL Databases
Data Processing
Gitlab
GIT
SC Clearance
Microsoft Fabric
Data Lake
PySpark
Terraform
Software Version Control
Data Pipelines
Job description
We are seeking an experienced Senior Data Engineer with active SC clearance to support a Central Government department on a mission-critical data and analytics programme. This role suits someone with strong Microsoft Fabric expertise and experience delivering secure, scalable data solutions in complex public sector environments., You'll design, build, and operate production-grade data pipelines that underpin reporting, MI, and analytics. Working closely with product, data, and platform teams, you'll help modernise Legacy data estates and embed strong data engineering standards., * Build and operate scalable data pipelines using Microsoft Fabric (OneLake/Delta Lake, Data Factory, Synapse Data Engineering).
- Develop batch processing solutions using PySpark, Spark SQL, Python, and SQL, with a focus on performance, resilience, and data quality.
- Support reporting and MI use cases, including transformations and data models feeding Power BI.
- Own CI/CD and version control practices; review code and uphold engineering standards.
- Mentor engineers and provide technical input into design and architecture.
- Work in Agile teams, collaborating via Jira and Confluence.
- Embed security and compliance by design, in line with government data handling policies.
Requirements
Essential Experience:
- Proven experience as a Senior Data Engineer or equivalent.
- Active SC clearance (BPSS minimum).
- Strong hands-on experience with Microsoft Fabric and Azure data services.
- Advanced skills in PySpark, Spark SQL, Python, and SQL.
- Experience delivering data engineering solutions in government or similarly regulated environments.
- CI/CD, DevOps, and IaC experience (eg Terraform).
- Strong Git/GitLab experience, including code reviews and branching strategies.
- Experience integrating data via APIs and services.
- Clear communication skills and experience working in Agile delivery teams.
Nice to Have:
- Data warehousing and dimensional modelling (eg dbt).
- Basic Power BI knowledge.
- Relevant Microsoft or Azure certifications.
