> Markdown version of [/jobs/ext/4417615-machine-learning-engineer-sme-llm-eval-rag-testing-sme-ts-sci](https://www.wearedevelopers.com/jobs/ext/4417615-machine-learning-engineer-sme-llm-eval-rag-testing-sme-ts-sci). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Machine Learning Engineer / SME (LLM Eval & RAG Testing) - SME | TS/SCI

- **Company:** Falls Tech, Inc.
- **Location:** McLean, VA, United States
- **Contract:** Permanent contract
- **Skills:** Artificial Intelligence, Artificial Neural Networks, Cloud Computing, Data Transformation, Github, Python (Programming Language), Machine Learning, Performance Tuning, Tensorflow, SQL Databases, Web Application Frameworks, Graphics Processing Unit (GPU), Pytorch, Retrieval-Augmented Generation, Flask (Web Framework), Large Language Models, Deep Learning, Generative AI, Keras, HuggingFace, Evaluation of Large Language Models, Software Version Control, Jenkins
- **Published:** October 27, 2026
- **Apply:** https://www.indeed.com/viewjob?jk=ba4a7a62086d0598

## About the Role

* Active TS/SCI
 * Hands-on RAG testing experience (must-have)
 * Tuning neural nets / LLMs on custom datasets for real use cases
 * Building AI/ML models and ensembles in Python or R
 * PyTorch, TensorFlow or Keras (key)
 * GPU-accelerated computing (key)
 * Testing, evaluating or validating LLM-based AI systems (key)
 * Experiment design to benchmark LLMs (key)
 * SQL including CTEs, set ops, aggregates and nested subqueries
 * GitHub and Jenkins (or equivalent) version control / CI

Nice to have

 * Hugging Face Transformers library and hub
 * Building benchmarks that measure LLM performance
 * Cloud computing development and architecture
 * Flask or similar front-end/web frameworks

## Description

Prove which LLMs actually work for the mission. You'll design RAG and LLM evaluation experiments, tune models on custom data, and run GPU-accelerated benchmarks so the team ships AI systems they can trust.

What you'll do

 * Test, evaluate and validate AI systems with a focus on LLM-based systems (including RAG testing)
 * Design and run experiments that benchmark LLMs against defined measures
 * Tune neural networks / LLMs on custom datasets and apply results to concrete use cases
 * Build AI/ML models and ensembles in Python or R (library selection, preprocessing, model assessment)
 * Use deep learning frameworks such as PyTorch, TensorFlow or Keras
 * Leverage GPUs for accelerated training and evaluation
 * Write solid SQL (CTEs, set operations, aggregates, nested subqueries) against analytical datasets
 * Work in GitHub and Jenkins-style CI for reproducible experiments
 * Communicate methodological choices and model results to technical and non-technical stakeholders