Data Engineer
Role details
Job location
Tech stack
Job description
As a Data Engineer, you will play a pivotal role in shaping the infrastructure that handles the company's data assets, ensuring data flows efficiently and securely from source to insight. You will be part of a team responsible for designing, building, and maintaining scalable data pipelines and storage systems that support analytics and data-driven decision-making using AWS cloud technologies.
The successful candidate will be capable of working independently, proactively driving improvements, and embracing new technologies throughout the product lifecycle. The role requires both a willingness to share knowledge and the ability to learn from other team members.
This is a newly formed team supporting a new project, providing an opportunity to influence architectural decisions, challenge assumptions, and collaborate closely with colleagues across Data Engineering, Data Science, and DevOps. As part of a small, agile team, you may also contribute to reporting, dashboard creation, and other activities required to ensure project success.
What you will do:
- Design, develop, test, and maintain data solutions, applications, and data pipelines.
- Participate in Agile ceremonies, backlog grooming, sprint planning, and prioritisation activities.
- Contribute to the design, implementation, and maintenance of scalable, secure, compliant, and reliable data architectures.
- Collaborate with data scientists, analysts, software engineers, and business stakeholders to deliver data-driven solutions.
- Monitor data platform performance and implement improvements to optimise efficiency and reduce latency.
- Actively ensure data quality, reliability, and operational excellence across data systems.
- Evaluate and support the adoption of new data technologies, tools, and architectural approaches.
- Help shape and execute the vision for Business Intelligence and Data Warehousing across the organisation.
- Build cross-functional relationships with Data Scientists, Product Managers, Software Engineers, and business stakeholders to understand and meet data needs.
- Promote best practices in data engineering and foster a culture of continuous improvement.
- Participate in strategic planning for data initiatives, helping define objectives and execution roadmaps.
- Communicate complex technical concepts and project progress to both technical and non-technical audiences.
- Mentor and support team members where appropriate.
- Foster a collaborative team culture built on transparency, accountability, and growth.
Requirements
Do you have experience in Spark?, * Commercial experience with Python.
- Strong SQL skills (any major SQL dialect).
- Experience using Git for source control and version management.
- Experience working within Agile/Scrum environments.
- Hands-on experience with AWS services.
- Understanding of AWS security best practices for data-related products.
- Experience containerising applications using Docker.
- Experience developing and maintaining data pipelines.
- Understanding of data quality, data governance, and data engineering best practices.
- Good understanding of the software development lifecycle (SDLC).
- Strong analytical and problem-solving skills.
- Excellent communication and stakeholder management skills.
Even better if you have:
- Advanced AWS expertise, including relevant AWS certifications.
- Experience designing analytical data models, particularly Kimball-style star schemas.
- Experience with Data Lakes and AWS Lake Formation.
- Experience with Data Lakehouse architectures.
- Experience with Bitbucket.
- Experience with Spark and PySpark.
- Understanding of MPP databases, columnar storage technologies, and Parquet file formats.
- Experience with datasets, analytics, and data visualisation tools.
- Experience with AWS deployment tools such as CodePipeline and CodeDeploy.
- Experience using Infrastructure as Code tools such as CloudFormation.
- Good understanding of GDPR and data governance principles.
- Experience supporting Business Intelligence and Data Warehousing initiatives.
- Exposure to Data Science or Machine Learning environments.