> Markdown version of [/jobs/ext/2729895-senior-data-engineer](https://www.wearedevelopers.com/jobs/ext/2729895-senior-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Senior Data Engineer

- **Company:** Optima Partners
- **Location:** Edinburgh, UK
- **Experience:** Expert
- **Salary:** £57,026.0
- **Contract:** Permanent contract
- **Skills:** Java (Programming Language), Agile Methodology, Airflow, Amazon Web Services, Microsoft Azure, Big Data, BigQuery, Cloud Computing, Computer Programming, Continuous Integration, Data Architecture, Information Engineering, Data Integration, Extract Transform Load (ETL), Data Systems, Data Warehousing, Relational Databases, Apache Hadoop, Python (Programming Language), PostgreSQL, Microsoft SQL Server, Performance Tuning, Scrum Methodology, Scala (Programming Language), Talend, Data Processing, Google Cloud, Data Storage Technologies, Cloud Platform System, Snowflake, Apache Spark, Git, Data Lakes, Apache Nifi, Software Version Control, Data Pipelines, Docker, Amazon Redshift, Databricks
- **Published:** September 5, 2026
- **Apply:** https://www.adzuna.co.uk/jobs/details/5869808019

## About the Role

* Programming: Proficiency in Python, Java, Scala, or similar languages for data processing.
 * Big Data Technologies: Hands-on experience with tools such as Databricks, Apache Spark, and Hadoop.
 * Cloud Platforms: Familiarity with AWS, Azure, GCP, or other cloud ecosystems for data engineering tasks.
 * Database Management: Expertise in relational databases (e.g. Postgres, SQL Server).
 * Data Integration Tools: Knowledge of platforms like Airflow, Apache NiFi, or Talend. * Data Storage and Modelling: Experience with data warehousing tools (e.g. Snowflake, Redshift, BigQuery) and schema design.
 * Version Control and CI/CD: Familiarity with Git, Docker, and CI/CD pipelines for deployment.
 * Experience: 5+ years in data engineering or a related role, with a proven track record of delivering advanced data solutions in consulting or client-facing environments.
 * Methodologies: Experience with Agile or Scrum practices is beneficial. Personal Qualities
 * Strong problem-solving and analytical skills.
 * Excellent communication abilities to explain complex technical concepts to non-technical stakeholders.
 * A collaborative, adaptable, and team-oriented mindset.

## Description

* Data Pipeline Development: Design and implement robust, scalable, and efficient data pipelines to collect, transform, and integrate data from various sources.
 * Data Architecture: Develop and optimise data architectures, including data warehouses, data lakes, and other storage solutions.
 * ETL/ELT Processes: Create and maintain reliable ETL/ELT workflows to ensure data quality and accessibility.
 * Collaborate: Partner with data scientists, analysts, and client stakeholders to understand requirements and deliver impactful solutions.
 * Performance Optimisation: Monitor and enhance the performance of data systems to ensure minimal downtime and rapid query responses.
 * Automation and Tools: Identify automation opportunities and recommend tools to improve data engineering workflows.
 * Documentation: Maintain detailed technical documentation for all solutions and processes.
 * Leadership: Mentor junior engineers and lead technical initiatives within cross-functional teams. Skills and Experience


## Related Videos

- [Docker Compose: Rediscovered](https://www.wearedevelopers.com/videos/1978-docker-compose-rediscovered)
- [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94)
- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)
- [Docker build without Docker](https://www.wearedevelopers.com/videos/100114-docker-build-without-docker)
- [Git for Code Reviews](https://www.wearedevelopers.com/videos/429-git-for-code-reviews)
- [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering)

## Related Articles

- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk)
- [The Most Popular IT Jobs on the Market](https://www.wearedevelopers.com/magazine/376-the-most-popular-it-jobs-on-the-market)
- [Software Engineer Salary London](https://www.wearedevelopers.com/magazine/252-software-engineer-salary-london)
- [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)