Data Engineer - SC Cleared
Role details
Job location
Tech stack
Job description
Role: Data Engineer
Location: Edinburgh or Luton (hybrid/onsite as required) Engagement: Contract - Inside IR35 Security Clearance: Active SC Clearance required
Rate: £500 - £600 per day - inside IR35 Project Description
We are seeking an experienced Data Engineer to design, build, deploy, and maintain robust data platforms and pipelines within a secure environment. You will be responsible for the end-to-end data engineering life cycle, transforming raw data into high-quality, consumable datasets that support analytics, reporting, and advanced modelling.
You will own and optimise the data operations infrastructure, ensuring performance, reliability, scalability, and security as data volumes and processing demands grow. This role requires strong problem-solving skills, the ability to integrate data from multiple sources, and hands-on experience with modern data engineering tools and practices. Key Responsibilities
- Design, develop, deploy, and support scalable data infrastructure, pipelines, and architectures
- Orchestrate ingestion and storage of raw data into structured and unstructured data solutions
- Implement reliable, automated, and well-tested data ingestion and processing workflows
- Build and maintain batch and Real Time data processing systems
- Manage and optimise performance, reliability, scalability, and security of data platforms
- Support data governance, quality, and compliance requirements
- Prepare data pipelines for descriptive, predictive, and prescriptive analytics
- Collaborate closely with data scientists, architects, IT teams, and business stakeholders
- Identify opportunities for new data acquisition and improved data utilisation
- Monitor, manage, and enhance data quality and reliability through automated tooling
Skills and Experience Required
- Active SC Clearance (mandatory)
- Strong experience designing and maintaining data pipelines, data warehouses, and data platforms
- Solid knowledge of DataOps practices, including CI/CD, containerisation, and workflow orchestration
- Hands-on experience with ETL/ELT frameworks and big data tools (eg Spark, Airflow, Hive)
- Proficiency in programming languages such as Python, Java, and SQL
- Experience with SQL and NoSQL database design and optimisation
- Strong understanding of batch and streaming data processing
- Degree in a STEM-related field; Master's degree desirable
- Data engineering certifications (eg IBM Certified Data Engineer) are advantageous
Requirements
- Active SC Clearance (mandatory)
- Strong experience designing and maintaining data pipelines, data warehouses, and data platforms
- Solid knowledge of DataOps practices, including CI/CD, containerisation, and workflow orchestration
- Hands-on experience with ETL/ELT frameworks and big data tools (eg Spark, Airflow, Hive)
- Proficiency in programming languages such as Python, Java, and SQL
- Experience with SQL and NoSQL database design and optimisation
- Strong understanding of batch and streaming data processing
- Degree in a STEM-related field; Master's degree desirable
- Data engineering certifications (eg IBM Certified Data Engineer) are advantageous
