Data Engineer

SearchWorks
Greater London, UK
1 day ago
Apply on www.collegerecruiter.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours

Tech stack

Application Programming Interfaces (APIs) Airflow Amazon Web Services Data Analysis Microsoft Azure Continuous Integration Data Architecture Information Engineering Data Integration Extract Transform Load (ETL) Data Warehousing Database Design
+14 more
Django Web Framework Python (Programming Language) Tensorflow DataOps SQL Databases Management of Software Versions Web Applications Data Storage Technologies Apache Spark Backend Data Lakes Deployment Automation Terraform Data Pipelines

Job description

You’ll combine data engineering, analytics, visualisation and infrastructure to make high-quality data accessible across the business., * Build and maintain scalable data pipelines feeding mobile and web apps, plus internal dashboards.

  • Design and implement data architecture to optimise storage, retrieval and processing.
  • Develop ETL processes to ingest, transform and load data from multiple sources, especially APIs.
  • Work closely with data scientists and software engineers to understand data needs for product features.
  • Partner with operations and commercial stakeholders to shape data requirements for reporting and decision-making.
  • Create and maintain data documentation, monitor pipeline performance and troubleshoot issues.

Requirements

  • Strong problem-solving skills, shown through academic or professional work.
  • Solid foundations in data architecture, data engineering best practices and scalable data solutions.
  • Good understanding of data modelling, database design and normalisation.
  • Proficiency in Python and SQL, ideally with experience of frameworks such as Airflow, TensorFlow or Spark.
  • Willingness to build basic-intermediate backend skills (e.g. Python/Django) to support data integrations.
  • Familiarity with data versioning, data quality management, build/deployment automation and CI/CD., * Experience building and maintaining data pipelines in production.
  • Exposure to data lakes, data warehousing and modern data storage patterns.
  • Experience with cloud platforms (AWS/Azure) and tools like Apache Airflow, Terraform or SageMaker.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.collegerecruiter.com
Prepare application

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

2:15 min

Empowering domain teams with an open data platform

Sandhya Menon Sandhya Menon · World Congress 2026 Europe

2:00 min

Separating dataset creation from low-level software implementation steps

Jan Zawadzki · World Congress 2022

1:52 min

Structuring and scaling the backend engineering team

Stefan Lingler Stefan Lingler +1 · Coffee With Developers

2:57 min

Core technical practices for robust data engineering

Sandhya Menon Sandhya Menon · World Congress 2026 Europe

1:12 min

Choosing TypeScript for complex backend applications

Maximilian Otto Maximilian Otto · World Congress 2024

3:03 min

Career evolution in data engineering and AI platforms

Maria Apazoglou · Coffee With Developers

Videos

See all

Related articles

See all