Data Engineer
Role details
Job location
Tech stack
Job description
Certain Advantage are seeking Engineers skilled in Python, PySpark and Azure infrastructure services to join a platform team supporting a major data and analytics initiative within a globally renowned Trading business in London.
You must be willing to work 3 days a week onsite in central London.
This Engineer position is critical to enhancing the team's ability to deliver re-usable, high quality data pipelines, reduce bottlenecks, and accelerate insight availability for business stakeholders.
We need Engineers to have a solid architectural foundation to be able to solve problems independently at a fast pace. You'll need to offer strong effective communication skills to be able to understand requirements and communicate with technical leaders asynchronously across 5 time zones.
Your initial work will involve abstracting code from our product teams into a shared, common python library leveraging PySpark/dataframes.
You'll also be building microservices in the form of python-based Azure Functions.
After the initial pre-defined work, you'll serve as an extension of the product teams building microservices and libraries to solve the common needs across the teams.
Requirements
- Python
- PySpark
- SQL
- Azure infrastructure
- Understanding of Containers, Microservices, and Functional design patterns
- Experience with Agile processes
- Experience with Terraform
- Experience with Unit Testing
- Preferably PyTest
Optional, but recommended skills:
- HTML/CSS
- React
- Typescript
- FastAPI framework