AI Engineer

Certain Advantage are recruiting for an AI Engineer with hands on Databricks and Spark expertise. Industry experience in commodity or financial trading is strongly preferred. This is a fantastic opportunity to partner with traders and trading analysts to rapidly build AI-powered analytics over market pricing and fundamentals data, using Databricks and Spark to deliver value at speed. Someone with familiarity in market microstructure, supply-demand fundamentals, risk management with confident collaboration and engagement skills with trading teams would be ideal.

Responsibilities to include

• Design and ship AI-driven analytics for front-office use (seasonality, correlation, regression, forecasting, scenario modelling).
• Build reusable and scalable data pipelines in Databricks (PySpark/Spark, Delta/Unity Catalog), optimizing cost, reliability, and performance.
• Run statistical/econometric analyses on large datasets (e.g., market & fundamental time series data).
• Collaborate directly with traders/analysts-translate ambiguous questions into shippable solutions; communicate insights clearly.
• Implement LLM/agentic workflows: prompt engineering, LangGraph orchestration, MCP integrations, tool calling, retrieval, and guardrails.
• Productionize solutions with testing, observability, versioning, and documentation.

• Hands-on Databricks + Spark expertise (PySpark, SQL, Delta, Unity Catalog).
• Proven data engineering skills (ingestion, modelling, orchestration, performance tuning).
• Strong statistics/economics/data science fundamentals for market time-series.
• Experience building LLM solutions (prompting, retrieval, agent flows; LangGraph, MCP) and integrating with trading data/services.
• Experience with CI/CD, Terraform, MLflow/feature stores, vector DBs, and governance (PII handling, data lineage).
• Excellent stakeholder skills; able to work on-desk with traders/analysts and deliver fast.