GCP Data Engineer
WA Consultants
Charing Cross, United Kingdom
2 days ago
Role details
Contract type
Temporary contract Employment type
Full-time (> 32 hours) Working hours
Regular working hours Languages
English Compensation
£ 114KJob location
Remote
Charing Cross, United Kingdom
Tech stack
Java
API
Agile Methodologies
Business Analytics Applications
Google BigQuery
Computer Programming
Data Architecture
Information Engineering
ETL
Data Systems
Relational Databases
Database Queries
Data Flow Control
Hive
Python
NoSQL
Powerpivot
Query Optimization
Power BI
Cloudera
Parquet
Google Cloud Platform
Spark
Technical Debt
Powerquery
Apache Flink
Avro
Google Cloud Functions
Integration Frameworks
Terraform
Data Pipelines
Job description
- Own the end-to-end delivery of data products, from development to support
- Design and implement scalable data pipelines and efficient data models
- Apply strong data modelling techniques and understand trade-offs
- Manage CI/CD pipelines and infrastructure using Terraform
- Build and maintain solutions on Google Cloud Platform
- Drive continuous improvement and reduce technical debt
- Collaborate with cross-functional teams to deliver data solutions
- Develop and optimise ETL/ELT processes
- Support automation using APIs and programming
Requirements
- Several years' experience in data engineering or similar roles
- Strong experience with GCP services including Cloud Functions, Dataflow, Dataproc and BigQuery
- Experience with data processing frameworks such as Beam, Spark, Hive or Flink
- Strong SQL skills and query optimisation experience
- Experience with ETL processes and tools
- Programming experience in Python, Java or Scala
- Knowledge of RDBMS and NoSQL databases
- Experience working with data formats such as Avro and Parquet
- Hands-on experience with CI/CD pipelines and Terraform
- Experience with data visualisation tools such as Power BI or similar
- Understanding of analytical tools including DAX, Power Pivot, Power Query and Tabular Editor
- Strong data modelling experience
Desirable Skills
- GCP Data Engineering certification
- Experience with modern data architectures such as data mesh or lakehouse
- Experience working in agile environments
Soft Skills
- Strong communication skills in English
- Collaborative mindset with ability to work independently
- Proactive approach to problem-solving and decision-making
About the company
WA Consultants is an Employment Business and an Employment Agency as described within The Conduct of Employment Agencies and Employment Businesses Regulations 2003.