Senior Data Engineer - Streaming
Role details
Job location
Tech stack
Job description
Checkout.com is looking for an ambitious Senior Data Engineer to join our Data Platform Team. Our team's mission is to build a world-class data platform that powers our products and analytics.
The Data Platform team is here to ensure internal stakeholders can easily collect, store, process and utilise data to build reports or products aiming to solve business problems. Our focus is on maximising the amount of time business stakeholders spend on solving business problems and minimising time spent on technical details around implementation, deployment, and monitoring of their solutions.
We're building for scale. As such, much of what we design and implement today will be the technology/infrastructure which will serve hundreds of teams and petabyte-level volumes of data.
Key Responsibilities
-
Work with stream processing technologies (Kafka, kSQL & Flink) to build a continuously available large-scale event streaming platform.
-
Wherever possible, automate workflows and processes, we're aiming for the platform to be as self-sustaining as possible.
-
Stay up-to-date with the latest data and streaming engineering technologies and trends.
-
Use that knowledge and subject matter expertise to mentor the more junior members of the team, and work with other "application" teams to provide guidance and best practice.
-
Build tooling (SDKs/DSLs) and associated documentation to foster the adoption of the streaming platform by enabling upstream teams and systems to easily publish data and deploy streaming applications.
-
Experience describing infrastructure as code (Terraform or similar) as well as designing and implementing CI/CD pipelines.
-
Provide consultancy across the technology organisation to drive the adoption of the platform and unlock use-cases.
-
Promote data quality and governance as a first class citizen of the platform.
-
Provide hands-on support for all event-based systems including incident triage and root cause analysis.
Requirements
- Strong engineering background with a track record of implementing and owning components of a data platform.
- Experience working with stream technologies, ideally Kafka, but Kinesis or similar would also be applicable.
- Experience designing and implementing stream processing applications (kStreams, kSQL, Flink, Spark Streaming).
- Experience with Data Warehousing tools like Bigquery / Databricks, and building pipelines on these.
- Experience working with modern cloud-based stacks such as AWS, Azure or GCP
- Excellent programming skills with at least one of Python, Java, Scala or C#.
- You're a mentor, raising the bar for your colleagues.
- You're a collaborator, always ready to dive in and partner to solve tough problems.
- You're a listener, and seek to understand the underlying problems, before pitching solutions.
- You are able to drive through best practices by taking teams and organisations as a whole with you.
- You are a thought leader, so we'd love to see articles, podcasts, meetups or conference talks if you've done them.