> Markdown version of [/jobs/ext/3234304-data-engineer](https://www.wearedevelopers.com/jobs/ext/3234304-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Data Engineer - **Company:** ETeam Inc - **Location:** Glasgow, UK - **Salary:** £104,520.0 - **Contract:** Temporary contract - **Skills:** Unity 3d, Artificial Intelligence, Airflow, Amazon Web Services, Amazon S3, Data Analysis, Unit Testing, Cloud Computing, Code Review, Continuous Integration, Information Engineering, Extract Transform Load (ETL), Data Warehousing, DevOps, Distributed Computing Environment, Identity and Access Management, Python (Programming Language), SQL Databases, Data Processing, Cloud Platform System, Apache Spark, Amazon Virtual Private Cloud (VPC), Gitlab, Cloudformation, Data Lakes, Pyspark, Real Time Data, Apache Kafka, Machine Learning Operations, Video Streaming, Data Pipelines, Databricks - **Published:** September 5, 2026 - **Apply:** https://www.careerboard.com/pt/en/find-jobs-in-United-Kingdom/-27CACA411BD8E92191/ ## About the Role * Strong hands-on experience in Data Engineering with delivery of production-grade solutions. * Expertise in PySpark, Apache Spark, Python, and SQL. * Strong experience designing and optimizing complex data pipelines and ETL/ELT frameworks. * Hands-on experience with AWS services including: + S3 + Glue + Lambda + Step Functions + Athena + ECS + IAM + KMS + VPC + SageMaker (preferred) * Experience with AWS CloudFormation for Infrastructure as Code. * Strong understanding of data lakes, data warehouses, and distributed data processing. * Experience with GitLab, CI/CD, Unit Testing, and DevOps practices. * Excellent problem-solving skills and ability to work independently. * Strong stakeholder management and communication skills. Nice to Have * Experience with Databricks, Delta Lake, Unity Catalog, and migration projects. * Knowledge of AI/ML and MLOps frameworks. * Experience with streaming technologies such as Kafka or Kinesis. Ideal Candidate: A hands-on Data Engineer with strong expertise in Spark, PySpark, AWS, and CloudFormation, capable of building scalable enterprise data solutions while driving modernization and cloud transformation initiatives ## Description To design, build, and maintain scalable data pipelines, data lakes, and data warehouse solutions on AWS. The role focuses on developing high-performance data engineering solutions using PySpark, Spark, Python, and AWS services, enabling secure, reliable, and efficient data processing and analytics across enterprise platforms., * Design, develop, and maintain scalable batch and Real Time data pipelines using PySpark, Spark, Python, and AWS services. * Build and optimize data lakes and data warehouse solutions ensuring data quality, security, and accessibility. * Develop reusable, production-grade ETL/ELT frameworks and data processing solutions. * Implement orchestration workflows using AWS Step Functions, Airflow, and other automation tools. * Develop and maintain cloud infrastructure using AWS CloudFormation. * Collaborate with business stakeholders to understand requirements and translate them into scalable technical solutions. * Optimize data processing performance, monitoring, and operational support. * Implement unit testing, code reviews, and CI/CD best practices using GitLab. * Support platform modernization and migration initiatives leveraging Spark-based architectures. * Work closely with Data Scientists and Analytics teams to enable AI/ML use cases. ## Related Videos - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Fully Orchestrating Databricks from Airflow](https://www.wearedevelopers.com/videos/336-fully-orchestrating-databricks-from-airflow) - [Shifting Stress to Progress— Understanding DevOps to do DevOps Better](https://www.wearedevelopers.com/videos/268-shifting-stress-to-progress-understanding-devops-to-do-devops-better) - [GitLab CI pipelines for a whole company](https://www.wearedevelopers.com/videos/143-gitlab-ci-pipelines-for-a-whole-company) - [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt) - [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk) - [The Most Popular IT Jobs on the Market](https://www.wearedevelopers.com/magazine/376-the-most-popular-it-jobs-on-the-market) - [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story)