Data Engineer
SearchWorks
Greater London, UK
1 day ago
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Apply on www.collegerecruiter.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Job source
Tech stack
Application Programming Interfaces (APIs)
Airflow
Amazon Web Services
Data Analysis
Microsoft Azure
Continuous Integration
Data Architecture
Information Engineering
Data Integration
Extract Transform Load (ETL)
Data Warehousing
Database Design
+14 more
Django Web Framework
Python (Programming Language)
Tensorflow
DataOps
SQL Databases
Management of Software Versions
Web Applications
Data Storage Technologies
Apache Spark
Backend
Data Lakes
Deployment Automation
Terraform
Data Pipelines
Job description
You’ll combine data engineering, analytics, visualisation and infrastructure to make high-quality data accessible across the business., * Build and maintain scalable data pipelines feeding mobile and web apps, plus internal dashboards.
- Design and implement data architecture to optimise storage, retrieval and processing.
- Develop ETL processes to ingest, transform and load data from multiple sources, especially APIs.
- Work closely with data scientists and software engineers to understand data needs for product features.
- Partner with operations and commercial stakeholders to shape data requirements for reporting and decision-making.
- Create and maintain data documentation, monitor pipeline performance and troubleshoot issues.
Requirements
- Strong problem-solving skills, shown through academic or professional work.
- Solid foundations in data architecture, data engineering best practices and scalable data solutions.
- Good understanding of data modelling, database design and normalisation.
- Proficiency in Python and SQL, ideally with experience of frameworks such as Airflow, TensorFlow or Spark.
- Willingness to build basic-intermediate backend skills (e.g. Python/Django) to support data integrations.
- Familiarity with data versioning, data quality management, build/deployment automation and CI/CD., * Experience building and maintaining data pipelines in production.
- Exposure to data lakes, data warehousing and modern data storage patterns.
- Experience with cloud platforms (AWS/Azure) and tools like Apache Airflow, Terraform or SageMaker.
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Apply on www.collegerecruiter.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Talks and stories from around this role — technically off-topic, practically not.
Moments
Explore playlistsVideos
See allRelated articles
See all
EM
Eli McGarvie
about 3 years ago
LM
Luis Minvielle
The Most Popular IT Jobs on the Market
over 2 years ago
EM
Eli McGarvie
Software Engineer Salary London
about 3 years ago
EM
Eli McGarvie
Data Analyst Salary in the UK
about 3 years ago
LM
Luis Minvielle
Top-Paying Tech Jobs (with Salaries)
over 2 years ago
EM
Eli McGarvie
Highest Paying Tech Companies for Developers
over 3 years ago