Lead Data Engineer

VanRath
UK
22 days ago
Apply on www.vanrath.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Working hours
Shift work

Tech stack

Application Programming Interfaces (APIs) Artificial Intelligence Amazon Web Services Data Analysis Big Data Cloud Computing Data Architecture Information Engineering Data Governance Data Infrastructure Extract Transform Load (ETL) Data Warehousing
+8 more
Python (Programming Language) Apache Spark Data Lakes Pyspark AWS Glue Data Analytics Data Pipelines Databricks

Job description

You will work closely with colleagues across Data, Analytics, IT and the wider business, providing technical leadership and ensuring the Data Engineering team delivers robust, scalable and high-quality solutions., As Data Engineering Team Lead, you will be responsible for providing both people and technical leadership across the Data Engineering function.

Your responsibilities will include:

  • Leading, supporting and developing a team of Data Engineers
  • Setting technical direction and establishing best practice across the team
  • Managing priorities and ensuring effective delivery of data engineering projects
  • Designing, building and maintaining scalable data pipelines and ETL/ELT processes
  • Working hands-on with Databricks, Delta Lake, Python/PySpark and cloud technologies
  • Contributing to the development and evolution of the organisation’s data architecture
  • Developing robust data models to support BI, reporting and analytics
  • Monitoring and optimising data pipelines, Spark jobs and workflows
  • Ensuring appropriate standards around data quality, governance, security and compliance
  • Working closely with BI, Analytics, IT and business stakeholders
  • Translating business requirements into effective and scalable data solutions
  • Supporting advanced analytics, AI and machine learning initiatives
  • Identifying opportunities to improve data engineering capability, processes and performance
  • Acting as a technical point of escalation and providing guidance on complex engineering challenges
  • Building a culture of collaboration, knowledge sharing and continuous improvement within the team

Requirements

An established and growing organisation is investing heavily in its Data & Analytics capability and is now seeking an experienced Data Engineering Team Lead to join its team. This is a fully remote opportunity for an experienced data professional with a proven track record of leading Data Engineering teams, providing technical direction and delivering scalable data solutions across a complex organisation., You will be an experienced Data Engineering Team Lead, Data Engineering Manager or Senior Data Engineering professional with existing team leadership experience.

You must be based in Northern Ireland and have unrestricted eligibility to work in the UK without sponsorship. This is a mandatory requirement for the position.

You will ideally have:

  • Proven experience leading and developing a Data Engineering team
  • Strong commercial experience within Data Engineering, BI or Data & Analytics
  • Strong technical expertise across modern data engineering environments
  • Experience with Databricks and Python/PySpark
  • Strong knowledge of data warehousing, data modelling and big data concepts
  • Experience developing scalable ETL/ELT processes and data pipelines
  • Strong understanding of Delta Lake and modern data platform architecture
  • Experience working with cloud technologies, ideally AWS
  • Experience providing technical direction and setting engineering standards
  • Strong stakeholder management and communication skills
  • Excellent problem-solving and analytical abilities
  • The ability to balance strategic thinking, team leadership and hands-on technical delivery

Experience with AWS Glue, APIs, advanced analytics, AI/ML, data governance or Agile/SCRUM environments would be advantageous., This role would suit an experienced Data Engineering Team Lead who is comfortable leading people, influencing stakeholders and remaining technically engaged, with the opportunity to make a significant impact on the organisation’s wider Data & Analytics strategy.

Benefits & conditions

  • Competitive salary
  • Fully remote working
  • 36.5-hour working week
  • Flexible working arrangements
  • Excellent benefits package
  • Opportunity to lead and develop an established Data Engineering team
  • Modern cloud and data technology environment
  • Excellent opportunities for professional and career development
  • Candidates must be based in Northern Ireland
  • Unrestricted right to work in the UK is essential - visa sponsorship is not available

About the company

Your next employer is an established organisation with a strong presence across Northern Ireland and a significant focus on technology, data and digital transformation.

As the organisation continues to develop its Data & Analytics capability, this is an exciting opportunity to join a collaborative technology function and play a key role in shaping the future of its data engineering capability.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.vanrath.com
Prepare application

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

3:28 min

Defining big data and machine learning fundamentals

Ayon Roy · LIVE

3:03 min

Career evolution in data engineering and AI platforms

Maria Apazoglou · Coffee With Developers

4:32 min

Harnessing Spark with Python using PySpark and Py4J

Ayon Roy · LIVE

55 sec

Validating data processing architectures via containerized events

Modood Alvi · World Congress 2025

2:10 min

Why organizations combine big data and machine learning

Ayon Roy · LIVE

3:09 min

Balancing data science skillings alongside systems engineering rigor

Nico Schmidt · LIVE

Videos

See all

Related articles

See all