Data Engineer
iO Associates
Charing Cross, United Kingdom
2 days ago
Role details
Contract type
Permanent contract Employment type
Full-time (> 32 hours) Working hours
Regular working hours Languages
English Compensation
£ 75KJob location
Charing Cross, United Kingdom
Tech stack
Amazon Web Services (AWS)
Automation of Tests
Big Data
Continuous Integration
Information Engineering
Data Governance
ETL
Data Systems
Python
Cloud Services
DataOps
Cloudera
Talend
Data Processing
Data Ingestion
Data Lake
PySpark
Data Management
Job description
This pivotal role is a response to strategic growth and the ongoing digital transformation within our Client. As a Data Engineer, you will be at the forefront of designing and implementing high-quality data solutions that significantly impact decision-making processes. Your expertise will help shape the future of data management and analytics, underpinning critical initiatives in a fast-paced, technology-enabled environment., * Lead technical designs for scalable data platforms within an AWS Cloud environment, guiding the implementation of robust solutions.
- Provide expert advice on architecture to ensure data solutions meet business needs and comply with regulatory standards.
- Review deliverables from various teams, ensuring adherence to best practices such as CI/CD, automated testing, and high-quality coding standards.
- Identify opportunities for product enhancements, proposing innovative solutions that deliver measurable benefits.
- Collaborate with senior professionals, stakeholders, and third-party providers to facilitate technology change and promote a data-centric approach.
- Foster a culture of agility, continuous improvement, and rapid delivery by applying modern engineering practices.
Requirements
- Extensive experience designing and developing end-to-end ETL pipelines on AWS and Cloudera data lake environments, supporting high-volume data ingestion with secure, scalable architectures.
- Proven track record delivering enterprise data solutions using tools such as Talend ETL, Python, and PySpark.
- Strong understanding of big data technologies, data governance, security, and compliance frameworks.
- Demonstrated ability to work effectively across diverse teams and manage complex integrations.
- Passionate about innovation with a focus on enhancing performance, reliability, and operational efficiency in data engineering.
Desirable Skills & Experience
- Prior experience with DataOps practices and automated testing approaches such as TDD and BDD.
- Familiarity with additional data processing tools and cloud services.
- Knowledge of regulatory requirements specific to data handling within a regulated environment.