Databricks Consultant

Trust In Soda Ltd
Charing Cross, United Kingdom
2 days ago

Role details

Contract type
Temporary contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior

Job location

Charing Cross, United Kingdom

Tech stack

Unity
Airflow
Amazon Web Services (AWS)
Data analysis
Azure
Data Governance
Hive
Python
Performance Tuning
SQL Databases
Data Streaming
Workflow Management Systems
Azure
Data Lake
PySpark
Kafka
Databricks

Job description

Data Architecture: Build and optimize ETL/ELT pipelines using Databricks, Spark (PySpark/Scala), and Delta Lake for structured/unstructured energy market data.

Trading Analytics: Partner with Quants, Risk Managers, and Traders to implement models, curve generation, position tracking, and risk metrics (e.g., VaR). Performance Optimization: Tune clusters and SQL warehouses for low-latency, time-sensitive trading environments. Data Governance: Enforce governance, access controls, and compliance with UK/EU financial and energy regulations via Unity Catalog. Consultative Leadership: Provide architectural best practices, mentor engineers, and advise stakeholders on maximizing Databricks ROI.

Requirements

  • Energy Trading Expertise: Proven background in energy trading firms, banks, utilities, or oil/gas majors. Strong knowledge of physical/financial markets (Power, Gas, LNG, Crude, Carbon/Renewables) and trading data types (Reuters, Bloomberg, Trayport, weather, ETRM).
  • Databricks Mastery: Hands-on experience with production workloads on Databricks (Azure/AWS), PySpark, Spark SQL, Delta Lake, Unity Catalog, Python, SQL, orchestration tools (Airflow, Workflows, ADF), and streaming (Kafka/Event Hubs).
  • Professional Skills: Strong consultative mindset, excellent communication, and ability to interface with traders and senior stakeholders.

Apply for this position