> Markdown version of [/jobs/ext/3029394-it-data-engineer](https://www.wearedevelopers.com/jobs/ext/3029394-it-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# IT Data Engineer

- **Company:** Allied Llc
- **Location:** Houston, TX, United States
- **Experience:** Expert
- **Salary:** $107,900.0 - $195,050.0
- **Contract:** Permanent contract
- **Skills:** Artificial Intelligence, Automation of Tests, Big Data, Continuous Integration, Information Engineering, Performance Tuning, SQL Databases, Systems Integration, Management of Software Versions, Large Language Models, Prompt Engineering, Apache Spark, Pyspark, Data Lineage, Data Analytics, Machine Learning Operations, Terraform, Data Pipelines, Databricks
- **Published:** September 22, 2026
- **Apply:** https://www.careerjet.com/job/usd97f18b5cec25676f1eac9474ad6e5af/eaa

## About the Role

We're seeking an AI Engineer with a strong data engineering foundation and excellent communication skills-ideally with commodity or financial trading experience. You'll partner with traders and trading analysts to rapidly build AI powered analytics over market pricing and fundamentals data, using Databricks and Spark to deliver value at speed., · Hands on Databricks + Spark expertise (PySpark, SQL, Delta, Unity Catalog).
· Proven data engineering skills (ingestion, modelling, orchestration, performance tuning).
· Strong statistics/economics/data science fundamentals for market time series.
· Experience building LLM solutions (prompting, retrieval, agent flows; LangGraph, MCP) and integrating with trading data/services.
· Experience with CI/CD, Terraform, MLflow/feature stores, vector DBs, and governance (PII handling, data lineage).
· Excellent stakeholder skills; able to work on desk with traders/analysts and deliver fast.
· Maintain a Hybrid; high touch collaboration with trading teams.
· Maintain operational stability of production pipelines using secure, modern CI/CD engineering practices - with automated testing, quality gates, and built in reliability across development, security, and operations.
Bonus skills
· Background in commodity or financial trading.
· Familiarity with market microstructure, supply demand fundamentals, risk management.
#LI-TS1

## Description

· Design and ship AI driven analytics for front office use (seasonality, correlation, regression, forecasting, scenario modelling).
· Build reusable and scalable data pipelines in Databricks (PySpark/Spark, Delta/Unity Catalog), optimizing cost, reliability, and performance.
· Run statistical/econometric analyses on large datasets (e.g., market & fundamental time series data).
· Collaborate directly with traders/analysts-translate ambiguous questions into shippable solutions; communicate insights clearly.
· Implement LLM/agentic workflows: prompt engineering, LangGraph orchestration, MCP integrations, tool calling, retrieval, and guardrails.
· Productionize solutions with testing, observability, versioning, and documentation.


## Related Videos

- [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data)
- [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology)
- [Infrastructure as Code: The Developer's Secret Weapon](https://www.wearedevelopers.com/videos/1221-infrastructure-as-code-the-developer-s-secret-weapon)
- [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning)
- [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail)
- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)

## Related Articles

- [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer)
- [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production)
- [What Industries Outside of AI Are Hiring The Most AI Experts?](https://www.wearedevelopers.com/magazine/98-what-industries-outside-of-ai-are-hiring-the-most-ai-experts)
- [Navigating the AI Shift](https://www.wearedevelopers.com/magazine/629-navigating-the-ai-shift)
- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)