Senior Data Engineer
Ingenium Digital
2 days ago
Role details
Contract type
Temporary contract Employment type
Full-time (> 32 hours) Working hours
Regular working hours Languages
English Experience level
Senior Compensation
£ 146KJob location
Remote
Tech stack
Java
Artificial Intelligence
Apache HTTP Server
Computer Programming
Data Architecture
Data Control
Information Engineering
ETL
Data Systems
Distributed Systems
Python
PostgreSQL
Data Build Tool (dbt)
Spark
Information Technology
Data Lineage
Amazon Web Services (AWS)
Data Pipelines
Databricks
Job description
- Set up and optimize best practices for batch and streaming data pipelines.
- Implement data quality, data monitoring, and data lineage tooling and reference architectures.
- Collaborate closely with Data Architects to define schema, governance standards, and data modeling.
- Develop and maintain ETL/ELT workflows, ensuring scalability and performance.
- Work cross-functionally with product, engineering, and analytics teams to support data requirements.
- Drive capability-based design principles to build reusable and modular components.
- Troubleshoot and optimize performance issues across data systems.
Requirements
Do you have experience in Spark?, Do you have a Master's degree?, We are seeking a highly skilled Senior Data Engineer to design, build, and optimize robust data pipelines and platforms. The ideal candidate will have strong expertise in modern data engineering tools and frameworks, with a focus on building scalable, high-quality data systems that support analytics, AI/ML, and enterprise data needs., * Strong experience with Postgres Aurora
- Expertise in Apache Iceberg
- Hands-on with AWS Glue Jobs
- Proficient in Apache Spark
- Strong programming skills in Python
- Experience working with Trino
- Solid understanding of capability-based design and data architecture fundamentals
Preferred (Bonus) Skills
- Experience with Great Expectations for data quality
- Exposure to DBT (Data Build Tool)
- Awareness of Databricks
- Optional: Knowledge of Java, * Bachelor's or Master's degree in Computer Science, Engineering, or related field
- 7+ years of relevant experience in data engineering
- Proven experience building scalable data pipelines and distributed systems
- Strong problem-solving skills and understanding of modern data architectures