Data Engineer

RocketFin Consulting Ltd

London, UK

about 2 months ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Working hours

Regular working hours

Job source

indeed.com

Tech stack

Application Programming Interfaces (APIs) Data Analysis Microsoft Azure Big Data Databases Information Engineering Web Scraping Data Systems Data Warehousing Python (Programming Language) Machine Learning Power BI

+13 more

SQL Databases Data Processing Real Time Systems Azure Data Factory Snowflake Grafana Backend Playwright Data Management Api Design Streamlit Framework Data Pipelines Databricks

Job description

Data Engineering & Pipelines

Build and enhance data pipelines and automation workflows on Azure for large-scale data ingestion and processing
Maintain and optimise existing toolkits, databases, dashboards, and API-based solutions
Develop end-to-end automated data solutions across commodities (Gas/LNG, Power, Weather)
Design and implement web scraping pipelines (e.g. US LNG data)

Analytics & Real-Time Systems

Support real-time and batch analytics use cases in a dynamic trading environment
Enable and support analytics dashboards and GUI tools with reliable backend infrastructure
Develop real-time flow tracking solutions at critical market points

Data Modelling & Architecture

Design and manage Snowflake data models, ensuring performance and scalability
Integrate external data sources including ENTSOE, EPEX, KPLER, MetDesk, WoodMac, and others
Collaborate with analytics teams to structure data for models and advanced analytics use cases
Build and maintain data platforms supporting ML models such as demand forecasting

Key Deliverables

LNG Sendout Optimisation Model
Ship Tracking & Flow Monitoring Tools
Prompt Price & Forward Curve Bootstrapping
Web scraping pipelines for US LNG and other market data sources
Real-time flow tracking infrastructure at critical market points
Data platforms supporting ML and demand forecasting models

Requirements

Do you have experience in SQL?, Technical

Strong proficiency in Python for data engineering and pipeline development
Hands-on experience with Azure data services (Data Factory, Databricks, Blob Storage, etc.)
Experience designing and optimising Snowflake data models
Proven track record building and maintaining data pipelines at scale
Experience integrating third-party market data APIs and external data feeds
Familiarity with SQL and data warehousing concepts

Domain

Experience or strong interest in energy, commodities, or financial markets data
Understanding of real-time and batch data processing patterns in a trading context
Familiarity with market data providers such as ENTSOE, EPEX, KPLER, or similar, * Experience with web scraping frameworks (e.g. Scrapy, BeautifulSoup, Playwright)
Knowledge of LNG, Gas, or Power market fundamentals
Experience supporting ML model pipelines or demand forecasting workflows
Exposure to GUI/dashboard tools such as Grafana, Power BI, or Streamlit
Knowledge of ship tracking data or AIS feeds