Data Pipeline Manager

17918
London, UK
20 days ago
Apply on www.apply4u.co.uk
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours

Tech stack

Java (Programming Language) Microsoft Excel Microsoft Windows Application Programming Interfaces (APIs) Databases Data Files Data Integration Extract Transform Load (ETL) Github DataOps SQL Databases Data Streaming
+9 more
Talend Transaction Data Data Processing Delivery Pipeline Data Analytics Qlikview Data Management Software Version Control Data Pipelines

Job description

scalable, reliable data pipelines that support enterprise reporting and analytics. You will oversee the development and optimisation of Talend-based data workflows, ensuring high data quality and availability across systems such as Qlik and our central databases. Collaborating with data stewards, analysts, and support teams, you will guide the implementation of best practices in data integration, transformation, and validation. This role is critical in ensuring the stability and performance of data operations that underpin business insights and application functionality. The Valpak Data Insights (VDI) department regularly receive data files from their customers systems with regards to Product and transactional data. This data is transformed and loaded into our database using an inhouse built application. VDI will be migrating to the Qlik Talend ETL software on the customer files to load data into our databases. Responsibilities : Liaise with VDI and IT to allow the smooth transition of

Requirements

data through Qlik Talend Put a structure in place to create an environment for Qlik Talend and GitHub to allow easy development of pipelines Ensure that Qlik Talend Developers are allocated suitable well-defined requirements. Act as a point of escalation for any Qlik Talend questions. Make sure that the Qlik environment is running and that automated pipelines can be ran. Generate and consistently improve Qlik Talend pipelines Create and maintain pipelines for each customers data. Create a reusable toolkit of code for validation and transformations that can be used by other customers Generate automated validation checks on the received data so that it can be handled more efficiently before loading into the database. Become an expert in using the Qlik Talend Studio software so that any data transformations requested by the business can be performed. Use Java code to perform advanced manipulation of the data Integrate Apis into the Talend Studio Keep up to date with improvements in the Qlik Talend Studio to improve data handling. Ensure standards are maintained Make sure that data is handled correctly by Qlik Talend and that poor data is not loaded into the Database. Use GitHub to allow source control of the created pipelines. Implement Qlik Talend with Data Streams Liaise with Account Managers to make sure that pipelines can process the data received from Customers Liaise with business to understand customer requirements, and to keep the business informed of pipeline progress. Provide feedback to users about the quality of the customers data. Establish and maintain a data stewardship model that works for the operational team What we need from you Management experience Excellent written and verbal communication skills Great attention to detail Excellent organisational skills. Strong analytical and problem-solving abilities Experience using Microsoft packages, including Excel Knowledge of ETL tools and data pipelines, SQL, Java and GitHub is required Applicants should be educated to degree level or possess solid work experience in related field. What we offer Dual Location / Hybrid worker Status Participation in our annual Incentive Plan (VIP) 25 days annual leave plus bank holidays, volunteering day and your birthday off! Option to buy and sell up to 9 days annual leave Access to voluntary benefits including private medical insurance, cycle to work scheme, and subsidised gym membership Automatic inclusion in Life Assurance, Critical Illness and Disability Income protection schemes Pension scheme up to 8% employer contribution Access to reward & discount platform Wellbeing initiatives TPBN1_UKTJ

About the company

As the UKs leading provider of environmental compliance, Valpak strives to work together towards a profitable, sustainable, waste-free world. To date, our teams work in partnership with more than 400 customers, in nearly every corner of the globe. Valpak contributes to achieving the mission of Reconomys complete global compliance offering Comply Loop. Solving increasingly complex environmental regulatory challenges using data, expert local knowledge, and through leadership to drive business accountability across the globe. Our core values drive everything we do. We believe in promoting a supportive environment for our colleagues , delivering exceptional service to our customers , contributing to the community , and working towards a more sustainable environment . If you share these values and are passionate about making a positive impact, we’d love to have you on our team. About the role As the Data Pipeline Manager, you will lead a team responsible for building and maintaining

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.apply4u.co.uk
Prepare application

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

2:00 min

Separating dataset creation from low-level software implementation steps

Jan Zawadzki · World Congress 2022

6:36 min

Funding open source through GitHub Accelerator and Sponsors

Stormy Peters · World Congress 2023

5:39 min

Orchestrating interdependent pipelines across disconnected processing jobs

Álvaro Martín Lozano · LIVE

2:40 min

Using GitHub primitives for internal documentation and corporate operations

Kyle Daigle · Coffee With Developers

47 sec

Building modern data pipelines for legacy exports

Dr. Alexander Wachtel Dr. Alexander Wachtel +1 · World Congress 2025

2:43 min

Deploying a web application through an AI agent workflow

Mike Mike · World Congress 2025

Videos

See all

Related articles

See all