> Markdown version of [/jobs/ext/3136008-ai-platform-and-harness-engineer](https://www.wearedevelopers.com/jobs/ext/3136008-ai-platform-and-harness-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # AI Platform and Harness Engineer - **Company:** LTS, Inc. - **Location:** United States (Remote available) - **Experience:** Experienced - **Contract:** Permanent contract - **Skills:** Application Programming Interfaces (APIs), Artificial Intelligence, Amazon Web Services, Application Performance Management, Computing Platforms, Automation of Tests, Microsoft Azure, Cloud Computing, Cloud Engineering, Computer Programming, Continuous Integration, Software Debugging, DevOps, Distributed Systems, Python (Programming Language), Machine Learning, Performance Tuning, Regression Testing, Prometheus, Software Safety, Search Technologies, Software Deployment, Software Engineering, Management of Software Versions, Data Logging, Google Cloud, Enterprise Software Applications, System Availability, Delivery Pipeline, Large Language Models, Grafana, Multi-Agent Systems, Prompt Engineering, Model Validation, Generative AI, Infrastructure as Code (IaC), Backend, Git, Containerization, AI Platforms, Infrastructure Automation Frameworks, Information Technology, Low Latency, Machine Learning Operations, Api Design, Docker - **Published:** September 29, 2026 - **Apply:** https://www.dice.com/job-detail/a342d21b-4ff5-4bdf-9fcf-75f6de745070 ## About the Role * Bachelor's degree in Computer Science, Software Engineering, Artificial Intelligence, Data Science, or a related technical field. * 5+ years of experience in software engineering, platform engineering, backend engineering, DevOps, cloud engineering, or infrastructure engineering. * 2+ years building or supporting Generative AI, Large Language Model (LLM), or machine learning applications. * Strong programming experience in Python. * Experience developing APIs, backend services, and distributed systems. * Experience with cloud platforms including AWS, Azure, or Google Cloud Platform. * Experience deploying applications using Docker and Kubernetes. * Experience working with Git, CI/CD pipelines, Infrastructure as Code (IaC), and infrastructure automation. * Strong understanding of Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), Prompt engineering, Embeddings, Vector databases, AI agents and agentic workflows * Familiarity with AI evaluation techniques, automated testing, benchmarking, regression testing, and model validation. * Experience building scalable, production-grade software platforms. * Strong problem-solving, debugging, and performance optimization skills. Nice to Have: * Experience with AI orchestration frameworks such as LangChain, LangGraph, LlamaIndex, Semantic Kernel, or AutoGen. * Experience implementing LLMOps or MLOps platforms and deployment pipelines. * Experience with AI observability tools such as LangSmith, OpenTelemetry, Prometheus, Grafana, Evidently AI, or Arize AI. * Experience with vector databases including Pinecone, Qdrant, Weaviate, Azure AI Search, or pgvector. * Experience with OpenAI, Azure OpenAI, AWS Bedrock, Anthropic Claude, Google Vertex AI, or similar enterprise AI platforms. * Experience implementing Responsible AI, AI governance, model security, and AI safety best practices. * Experience supporting Federal Government or other regulated environments. * Experience evaluating AI systems for quality, reliability, accuracy, explainability, latency, and cost optimization. * Familiarity with healthcare, enterprise modernization, or mission-critical systems. ## Description LTS is seeking an AI Platform and Harness Engineer to develop and maintain the infrastructure, tooling, and evaluation frameworks that power enterprise AI solutions. This role is responsible for building the AI platform and reusable "AI harnesses" that enable Large Language Models (LLMs), AI agents, Retrieval-Augmented Generation (RAG), and Generative AI applications to be securely developed, tested, evaluated, monitored, and deployed at scale. The ideal candidate has experience with AI platforms, LLMOps, software engineering, cloud-native technologies, and backend systems, along with a passion for building reliable, observable, and production-ready AI solutions. You will work closely with AI architects, software engineers, data scientists, and product teams to ensure AI solutions are scalable, secure, cost-effective, and continuously improving. What You'll Do: * Design, build, and maintain enterprise AI platform capabilities supporting Large Language Models (LLMs), AI agents, RAG, and Generative AI applications. * Develop reusable AI harnesses to automate testing, prompt evaluation, model benchmarking, regression testing, and quality assurance. * Build AI evaluation frameworks to measure model accuracy, retrieval quality, hallucination detection, latency, throughput, cost, and overall application performance. * Implement observability and monitoring solutions for AI applications, including telemetry, tracing, logging, dashboards, and operational metrics. * Build and maintain LLMOps pipelines supporting model deployment, versioning, evaluation, experimentation, rollback, and continuous improvement. * Design automated workflows for prompt testing, retrieval evaluation, AI system validation, and performance benchmarking. * Develop internal tools for prompt management, model experimentation, AI performance optimization, and developer productivity. * Build scalable backend services and APIs supporting AI platforms and enterprise AI integrations. * Collaborate with AI architects and engineering teams to integrate LLMs, RAG pipelines, vector databases, and agentic AI solutions into enterprise applications. * Support deployment of AI services across AWS, Azure, or Google Cloud using containerized and cloud-native architectures. * Implement CI/CD pipelines and infrastructure automation supporting enterprise AI development and deployment. * Apply security, governance, and Responsible AI controls throughout the AI development lifecycle. * Evaluate emerging AI frameworks, LLMOps technologies, evaluation methodologies, and automation tools to improve engineering productivity. * Troubleshoot production AI issues and continuously improve platform reliability, scalability, security, and user experience. * Document engineering standards, AI platform architecture, evaluation methodologies, and operational best practices. ## Related Videos - [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94) - [Docker Compose: Rediscovered](https://www.wearedevelopers.com/videos/1978-docker-compose-rediscovered) - [From DevOps to Scaled DevOps: How We’re Rebuilding Continuous Delivery as a Platform](https://www.wearedevelopers.com/videos/100018-from-devops-to-scaled-devops-how-we-re-rebuilding-continuous-delivery-as-a-platform) - [Building AI Applications with LangChain and Node.js](https://www.wearedevelopers.com/videos/1512-building-ai-applications-with-langchain-and-node-js) - [Git for Code Reviews](https://www.wearedevelopers.com/videos/429-git-for-code-reviews) - [How E.On productionizes its AI model & Implementation of Secure Generative AI.](https://www.wearedevelopers.com/videos/623-how-e-on-productionizes-its-ai-model-implementation-of-secure-generative-ai) ## Related Articles - [What Are Large Language Models?](https://www.wearedevelopers.com/magazine/304-what-are-large-language-models) - [MLOps And AI Driven Development](https://www.wearedevelopers.com/magazine/82-mlops-and-ai-driven-development) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [Got AI ideas but no money? Here are 10 free ways to level up your AI skills with Google Cloud](https://www.wearedevelopers.com/magazine/600-got-ai-ideas-but-no-money-here-are-10-free-ways-to-level-up-your-ai-skills-with-google-cloud) - [MLOps – What’s the deal behind it?](https://www.wearedevelopers.com/magazine/125-mlops-what-s-the-deal-behind-it) - [From Prototype to Production: Build AI Agents with This Free 4-Course Learning Path](https://www.wearedevelopers.com/magazine/655-from-prototype-to-production-build-ai-agents-with-this-free-4-course-learning-path)