> Markdown version of [/jobs/ext/2308446-data-engineer](https://www.wearedevelopers.com/jobs/ext/2308446-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Data Engineer - **Company:** Smarkets - **Location:** Greater London, UK - **Experience:** Experienced - **Contract:** Permanent contract - **Skills:** Clean Code Principles, Application Programming Interfaces (APIs), Agile Methodology, Airflow, Data Analysis, JIRA, Unit Testing, BigQuery, Databases, Information Engineering, Extract Transform Load (ETL), Data Warehousing, Python (Programming Language), PostgreSQL, Machine Learning, NumPy, Recommender Systems, Web Application Frameworks, Flask (Web Framework), Pandas, Data Lakes, Scikit Learn, Kubernetes, Information Technology, Luigi, Data Management, Machine Learning Operations, Restful APIs - **Published:** August 30, 2026 - **Apply:** https://www.collegerecruiter.com/job/2815445000-data-engineer ## About the Role * 3+ years of experience * You have an understanding of developing ETL pipelines using Python frameworks such as luigi or airflow; * You have experience with the development of Python-based REST APIs / services and their integration with databases (e.g. Postgres); * You are familiar with the key tools of the Python data science stack, e.g. Pandas, Numpy and scikit-learn; * Some experience with training and deploying ML models; * You enjoy writing elegant, well-tested and maintainable code; * You are a team player that enjoys contributing to the success of the team in a proactive and friendly environment; * You have a bachelor's in Computer Science, Math or equivalent field, or possess relevant experience. ## Description The Data Team is responsible for taking the wealth of data that Smarkets generates and using it to drive insights which improve the business. Since Smarkets produces a huge amount of data - including sports event data, payments information, order flow and user analytics - there are many opportunities for the team to add real business value. The team's responsibilities currently span across three different areas: * Data Engineering: development and maintenance of ETL pipelines, services and APIs, and data-related infrastructure like Redshift or BigQuery; * Data Science and Machine Learning: data exploration, ML models training and ML Ops to extract new insights from data; * Analytics and Reporting: creation of data models and dashboards as well as automation of reporting pipelines for different teams, stakeholders and third-parties. In a typical week, a data engineer in the Data team would: * Add a new python ETL pipeline that segments users interested in specific sports through analysing behaviour which streamlines and tailors marketing communications to those users; * Develop a new endpoint to a Flask API, add unit tests, and deploy the new version of the API into our production Kubernetes cluster; * Train and evaluate an ML model to identify certain user patterns and provide it as service to other engineering teams in a Flask API, You will work very closely with the Data team lead and the other team members who will be assisting you whenever needed, making your integration in the company as smooth as possible. The Data team works in an organised way using Agile methodologies and tools such as Jira and regular standups. You will find an environment where you have a clear engineering direction, can focus on your work and hone your skills as a data engineer through exciting projects. You will always be able to count on the support of many engineers across the company. What you will do: As a member of the data team, your responsibilities will include contributions to: * Developing and maintaining our Data ETL pipelines, some of which are real-time. The pipelines are fundamental to helping teams and stakeholders understand and drive business direction. Data components can also be user facing e.g. sending notifications to users; * Ensuring our data lake is kept in a healthy state, particularly our data warehouses: Redshift and Bigquery; * Developing and maintaining Flask services and Postgres databases within the Data team to provide access to data or manage certain business entities relevant to Data. * Assisting the different teams in the company with reporting, especially when it comes to automated reporting pipelines. * Doing data exploration and training & deploying ML models used to perform different kinds of user segmentation, detect operational anomalies or estimate important business quantities. In particular maintain and improve our existing recommender service that provides users with suggestions on new sport competitions. ## Related Videos - [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail) - [Improving quality with Agentic AI with Rovo Dev and Xray](https://www.wearedevelopers.com/videos/2005-improving-quality-with-agentic-ai-with-rovo-dev-and-xray) - [Vectorize all the things! Using linear algebra and NumPy to make your Python code lightning fast.](https://www.wearedevelopers.com/videos/562-vectorize-all-the-things-using-linear-algebra-and-numpy-to-make-your-python-code-lightning-fast) - [Python Data Visualization @ Deepnote (w/ PyViz overview)](https://www.wearedevelopers.com/videos/113-python-data-visualization-deepnote-w-pyviz-overview) - [How to implement convenient Python bindings to C++](https://www.wearedevelopers.com/videos/618-how-to-implement-convenient-python-bindings-to-c) - [Collaboration Quantified: Lessons from Open Source Developer Networks](https://www.wearedevelopers.com/videos/1422-collaboration-quantified-lessons-from-open-source-developer-networks) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Software Engineer Salary London](https://www.wearedevelopers.com/magazine/252-software-engineer-salary-london) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk) - [The Most Popular IT Jobs on the Market](https://www.wearedevelopers.com/magazine/376-the-most-popular-it-jobs-on-the-market) - [The 13 Best Python Libraries for Developers in 2025](https://www.wearedevelopers.com/magazine/371-the-13-best-python-libraries-for-developers-in-2025) - [Why Upskilling And Reskilling is Important For Developers](https://www.wearedevelopers.com/magazine/428-why-upskilling-and-reskilling-is-important-for-developers)