Data Engineer

Searchability

London, UK

1 day ago

Apply on www.collegerecruiter.com

Prepare application

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Compensation

£85,000.0

Working hours

Regular working hours

Job source

www.collegerecruiter.com

Tech stack

Java (Programming Language) Amazon Web Services Amazon S3 Big Data Cloud Computing Databases Continuous Integration Information Engineering Extract Transform Load (ETL) Data Systems Data Warehousing DevOps

+19 more

Amazon DynamoDB Apache Hadoop Python (Programming Language) Open Source Technology Cloud Services Scala (Programming Language) Amazon Simple Notification Service (SNS) Unstructured Data Data Storage Technologies State Machines Build Management Amazon Relational Database Service Data Lakes Pyspark Infrastructure Automation Frameworks AWS Data Analytics Cloudwatch Data Pipelines Amazon Redshift

Job description

This role sits within our client’s rapidly growing Cloud Data Platforms team, part of the Insights and Data Global Practice. You will join a multidisciplinary group of data and platform specialists who deliver modern cloud-based transformation for clients across a range of sectors. In this role, you will design and build data pipelines, develop ETL/ELT processes, and create innovative data solutions using the latest cloud technologies and frameworks across AWS., * Build data pipelines to ingest, process and transform data for analytics and reporting.

Develop ETL/ELT workflows to move data efficiently into data warehouses, data lakes and lake houses using open-source and AWS tooling.
Apply DevOps practices, including CI/CD, infrastructure as code and automation, to improve and streamline data engineering processes.
Design effective data solutions that meet complex business needs and support informed decision-making.

Requirements

Strong AWS expertise, including tools such as Glue, Lambda, Kinesis, EMR, Athena, DynamoDB, CloudWatch, SNS and Step Functions.
Skilled in modern programming, particularly Python, Java, Scala and PySpark.
Solid knowledge of data storage and big data technologies, including data warehouses, databases, Redshift, RDS and Hadoop.
Experience building and managing AWS data lakes on S3 for both structured and unstructured data.