Data Engineer

• Monitoring daily BAU data pipelines and ensure our data solution is refreshed up to date every day
• Working with the team within an Agile framework using agile methodology tooling that controls our development and CI/CD release processes
• Get up to speed with existing data infrastructure, tools, and processes.
• Monitoring and responding to IT incidents.
• Build familiarity with key pipelines/processes and support documentation and enhance documentation.
• Begin proactive monitoring and suggest quick wins for stability and efficiency.
• Working on incident RCA along with other engineers within the team.
• Create and maintain technical documentation, RCA reports, and knowledge base articles.
• Implement monitoring dashboards and alerts for key jobs and data quality checks.
• Mentor junior engineers or act as a technical go-to for pipeline and cloud topics.

• Experience with data solution BAU processes (ETL, table refresh etc.)
• Experience in Big Data data integration technologies such as Spark, Kafka
• Experience in programming language such as Python.
• Experience using AWS (Athena, Glue, EMR, Step functions, Cloud watch), DBT and Snowflake.
• Analytical and problem-solving skills, applied to data solution
• Experience of CI/CD
• Experience in handling IT incidents.
• Experience with IT incident management process.
• Basic understanding of event streaming.
• Basic knowledge on Scala(nice to have).

Desirable

• Experience of ETL technologies
• Previous exposure to own data solution BAU monitoring and enhancement
• Exposure to grafana