> Markdown version of [/jobs/ext/2970723-core-data-engineer](https://www.wearedevelopers.com/jobs/ext/2970723-core-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Core Data Engineer

- **Company:** Optima Partners
- **Location:** Banavie, UK
- **Experience:** Experienced
- **Contract:** Permanent contract
- **Skills:** Java (Programming Language), Agile Methodology, Airflow, Amazon Web Services, Microsoft Azure, Big Data, BigQuery, Cloud Computing, Computer Programming, Continuous Integration, Customer Data Management, Data Architecture, Information Engineering, Data Integration, Extract Transform Load (ETL), Data Systems, Data Warehousing, Relational Databases, Apache Hadoop, Python (Programming Language), PostgreSQL, Microsoft SQL Server, Performance Tuning, Scrum Methodology, Scala (Programming Language), Talend, Google Cloud, Data Storage Technologies, Cloud Platform System, Snowflake, Apache Spark, Git, Data Lakes, Core Data, Data Analytics, Apache Nifi, Data Management, Software Version Control, Data Pipelines, Docker, Amazon Redshift, Databricks
- **Published:** September 18, 2026
- **Apply:** https://www.apply4u.co.uk/jobs/core-data-engineer/47370985

## About the Role

opportunities for automation and recommend tools to improve data engineering workflows. Documentation: Maintain detailed technical documentation for all solutions and processes. Technical Skills Programming: Proficiency in Python, Java, Scala, or similar languages. Big Data Technologies: Hands-on experience with big data tools e.g. (Databricks, Apache Spark, Hadoop). Cloud Platforms: Familiarity with AWS, Azure, GCP, or other cloud ecosystems for data engineering tasks. Expertise in relational databases (e.g. postgres, sql server) Data Integration Tools: Knowledge of platforms like Airflow, Apache NiFi, or Talend. Data Storage and Modelling: Experience with data warehousing tools (e.g. Snowflake, Redshift, BigQuery) and schema design. Version Control and CI/CD: Familiarity with Git, Docker, and CI/CD pipelines for deployment. Experience 2+ years of experience in data engineering or a similar role. Proven track record of delivering data solutions in a consulting or client-facing
environment is a plus. Experience with Agile or Scrum methodologies is beneficial. Personal Qualities Strong problem-solving and analytical thinking abilities. Excellent communication skills to explain complex technical concepts to non-technical stakeholders. A team-oriented mindset with a focus on collaboration and adaptability The Company Optima Partners is an advanced data and business consultancy headquartered in Edinburgh, UK. We are a practitioner-led organisation that collaborates with top consumer brands to drive transformation and foster customer-centricity through our expertise in customer strategy, innovative design, and advanced data science and engineering. We help our clients get closer to their customers, to truly understand them, and deliver exactly the right products, engagement, and experiences across all channels and interactions. We specialise in unlocking latent value within organizations through a three-pronged approach that focuses on
identifying and enhancing value for customers, within the customer base, and within the business itself. In doing so, we foster sustainable value, paving the way for consistent business growth. We work with leading consumer brands to tackle and overcome complex business and customer problems to drive transformation and champion customer-centric agendas. We are proud to include some of the leading UK and global brands among our current clients such as Lloyds Banking Group, NatWest Group, Bank of Ireland, Nationwide, Aviva, Biogen, Eon Next, OVO, Virgin Media O2, BT, HMD Global, Centrica, and GSK. We are obsessive about delivering value for our clients and work in a collaborative, engaged, and creative way with our colleagues and clients. We strive to support the transition of knowledge and capability into strategic teams. #J-18808-Ljbffr

## Description

you'll be contributing to our collective effort to stay at the forefront of a fast-paced, data-driven world. Moreover, there are chances to learn and grow as an engineer in different disciplines, as well as a clear path to career development and promotion within the role. Key Responsibilities Data Pipeline Development: Design and implement robust, scalable, and efficient data pipelines to collect, transform, and integrate data from various sources. Data Architecture: Develop and optimise data architectures, including data warehouses, data lakes, and other storage solutions. ETL/ELT Processes: Create and maintain reliable ETL/ELT workflows that ensure data quality and accessibility. Collaboration: Work closely with data scientists, analysts, and client stakeholders to understand requirements and deliver impactful solutions. Performance Optimisation: Monitor and enhance the performance of data systems to ensure minimal downtime and fast query responses. Automation and Tools: Identify


## Related Videos

- [Docker Compose: Rediscovered](https://www.wearedevelopers.com/videos/1978-docker-compose-rediscovered)
- [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94)
- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)
- [Docker build without Docker](https://www.wearedevelopers.com/videos/100114-docker-build-without-docker)
- [Git for Code Reviews](https://www.wearedevelopers.com/videos/429-git-for-code-reviews)
- [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering)

## Related Articles

- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [Best Companies to work for in London: Top 25 Companies in 2023](https://www.wearedevelopers.com/magazine/187-best-companies-to-work-for-in-london-top-25-companies-in-2023)
- [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [Fully Remote Software Engineer Jobs](https://www.wearedevelopers.com/magazine/447-fully-remote-software-engineer-jobs)
- [The Most Popular IT Jobs on the Market](https://www.wearedevelopers.com/magazine/376-the-most-popular-it-jobs-on-the-market)