Remote Senior Data Engineer - AWS/Databricks/PySpark - August Start Date

WüNDER TALENT

Glasgow, UK

about 2 months ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Compensation

£80,000.0 - £90,000.0

Working hours

Regular working hours

Job source

find.jobs

Tech stack

Airflow Amazon Web Services Amazon S3 Continuous Integration Data Cleansing Data Governance Extract Transform Load (ETL) Data Security Data Systems DevOps Github Standard Sql

+15 more

Software Engineering Transact-SQL Unstructured Data Scripting Cloud Platform System Apache Spark Git Data Lakes Pyspark Infrastructure Automation Frameworks Real Time Data Terraform Software Version Control Data Pipelines Databricks

Job description

Our partner is looking for a Senior Data Engineer to join a high-impact engineering team delivering scalable data solutions for complex marketing and customer insight use cases. This is an opportunity to work on cutting-edge data pipelines, cloud-native platforms and real-time data flows in a collaborative, forward-thinking environment.

You’ll be involved in designing and building production-grade ETL pipelines, driving DevOps practices across data systems and contributing to high-availability architectures using tools like Databricks, Spark and Airflow- all within a modern AWS ecosystem.

Responsibilities

Architect and build scalable, secure data pipelines using AWS, Databricks and PySpark.
Design and implement robust ETL/ELT solutions for both structured and unstructured data.
Automate workflows and orchestrate jobs using Airflow and GitHub Actions.
Integrate data with third-party APIs to support real-time marketing insights.
Collaborate closely with cross-functional teams including Data Science, Software Engineering and Product.
Champion best practices in data governance, observability and compliance.
Contribute to CI/CD pipeline development and infrastructure automation (Terraform, AWS DevOps).
Provide input into technical decisions, peer reviews and solution design.

Requirements

Proven experience as a Data Engineer in cloud-first environments.
Strong commercial knowledge of AWS services (e.g. S3, Glue, Redshift).
Advanced PySpark and Databricks experience (Delta Lake, Unity Catalog, Databricks Jobs etc).
Proficient in SQL (T-SQL/SparkSQL) and Python for data transformation and scripting.
Hands-on experience with workflow orchestration tools such as Airflow.
Strong version control and DevOps exposure (Git, GitHub Actions, Terraform).
Familiar with data quality tools and metadata/cataloguing (e.g. Great Expectations, Unity Catalog).
Beneficial: MarTech domain knowledge.

Notable: This is a hybrid engagement represented by 2 days/week onsite, either in Central London or Glasgow. You must be able to start in August.