Data Engineer

TESTQ Technologies
Leicester, United Kingdom
1 day ago
Apply on www.jobs.service.gov.uk
Prepare application

Role details

Contract type
Temporary contract
Employment type
Part-time / full-time
Experience level
Experienced
Experience required
4 years minimum
Compensation
£55,500 - £60,000
Working hours
Regular working hours

Tech stack

Java (Programming Language) Application Programming Interfaces (APIs) Airflow Amazon Web Services Microsoft Azure BigQuery Cloud Computing Cloud Database Code Review Computer Programming Databases Continuous Integration
+30 more
Data Files Data Governance Extract Transform Load (ETL) Data Transformation Data Security Data Vault Modeling Data Warehousing Dimensional Modeling Document-Oriented Databases Github Python (Programming Language) Metadata NoSQL Cloud Services SQL Databases Data Streaming Azure Data Factory Snowflake Apache Spark Git Data Lakes Pyspark Data Lineage Deployment Automation Apache Kafka Data Management Video Streaming Terraform Data Pipelines Databricks

Job description

TESTQ Technologies is an IT services and solutions company whose offerings spans over variety of industry sectors with strong technical, domain and process expertise helping clients grow their businesses and decrease operational costs on continuous basis in an ever-changing business environment.

This opportunity is in the solution design and development arena for Data Engineer who will play a major role in the technical design and development of the company’s technical offerings. The position is based at our Leicester office with occasional assignments at client locations.

Job Description (Main Duties and Responsibilities):

  • Design, build and maintain scalable and reliable data pipelines and ETL/ELT processes.

  • Analyse data requirements and design data models, data warehouses, data lakes and lakehouse solutions.

  • Ingest and integrate data from multiple sources including databases, APIs, files and streaming platforms.

  • Develop and optimise data transformations for performance, cost and reliability.

  • Implement data quality checks, validation, monitoring and alerting across pipelines.

  • Work with data architects, analysts and data scientists to deliver data sets that meet business needs.

  • Automate deployments using CI/CD and infrastructure as code.

  • Implement data security, access controls and governance in line with UK GDPR and company policies.

  • Troubleshoot and resolve data and pipeline issues in production environments.

  • Document data flows, data models, pipelines and operational procedures.

  • Migrate legacy data platforms and workloads to modern cloud technologies.

  • Review code and mentor junior engineers on engineering best practice.

  • Continually identify opportunities to improve platform performance and create additional value.

Requirements

Key Skills, Qualifications and Experience Needed [The candidate must demonstrate these in all stages of assessment]

  • Min 4+yrs of experience as a data engineer or in a related field.

  • Strong programming skills in Python and SQL; knowledge of Scala or Java is desirable.

  • Hands-on experience with cloud data platforms such as Azure (Data Factory, Synapse, Fabric), AWS (Glue, Redshift) or GCP (BigQuery).

  • Experience with Databricks, Apache Spark/PySpark and Delta Lake.

  • Experience with data warehousing technologies such as Snowflake, Synapse or Redshift.

  • Strong knowledge of data modelling, including dimensional modelling, Data Vault and medallion architecture.

  • Experience with orchestration and transformation tools such as Apache Airflow, Azure Data Factory or dbt.

  • Experience with streaming technologies such as Kafka or Azure Event Hubs.

  • Experience with CI/CD, Git, Azure DevOps or GitHub Actions and infrastructure as code (Terraform).

  • Knowledge of relational and NoSQL databases.

  • Understanding of data quality, data governance, metadata and data lineage.

  • Good awareness of data security and UK GDPR.

  • Proven ability to manage stakeholders and SMEs to deliver results within demanding timelines.

  • Strong communication skills, being able to understand technical detail and tailor summary to the audience in a simple and clear manner - both verbally and written.

  • Cloud data engineering certification (e.g. Microsoft, AWS or Databricks) is desirable.

Other Key skills:

  • Good analytical and Problem-solving skills

  • Good communication skills

  • A thorough approach and Self starter

  • Focus on quality and delivery

  • Working together in teams.

  • Leadership and effective decision making.

  • Flexible Attitude

  • Excellent customer service

Qualifications: Bachelor’s degree or above in the UK or Equivalent.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.jobs.service.gov.uk
Prepare application

Good distractions

Loading talks and stories from around this role…