> Markdown version of [/jobs/ext/2943315-ai-engineer-guardrails-and-observability](https://www.wearedevelopers.com/jobs/ext/2943315-ai-engineer-guardrails-and-observability). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # AI Engineer - Guardrails and Observability - **Company:** Bank of America - **Location:** Richmond, VA, United States - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Application Programming Interfaces (APIs), Agile Methodology, Artificial Intelligence, Application Performance Management, DevOps, Python (Programming Language), Language Modeling, OpenShift, Performance Tuning, Software Engineering, Management of Software Versions, Large Language Models, Model Validation, Generative AI, Kubernetes, Information Technology, Data Management, Virtual Agents, Splunk - **Published:** September 16, 2026 - **Apply:** https://www.techcareers.com/job.asp?id=3392077332&tx=JT9789TTT&pt=1&aff=0B19D771-A501-4A5E-8338-2A822B784D54&utm_source=Job%20Feed&utm_medium=textkernel&utm_campaign=DE&utm_term=0B19D771-A501-4A5E-8338-2A822B784D54 ## About the Role * Bachelor's or Master's degree in Computer Science, Data Science, MIS, or related field, or equivalent experience. * 10+ years of software engineering experience building and scaling production applications. * Strong Python development experience, including common ML and data libraries. * Hands-on experience with OpenShift or Kubernetes in production environments. * Experience deploying and supporting LLM or RAG-based applications. * Understanding of AI risks, agent behavior, and governance frameworks. * Experience integrating third-party platforms and supporting vendor applications. * Experience with OpenAI APIs, LangChain, and GenAI development frameworks. * Knowledge of inference performance optimization and scalability. * Experience implementing AI guardrails, observability, and model evaluation processes. * Experience working in Agile environments. * Proven ability to lead technical initiatives and influence stakeholders. * Strong analytical, problem-solving, and communication skills., * Experience delivering technology solutions within large, regulated enterprise environments, preferably financial services. * Experience with annotation pipelines, feedback loops, fine-tuning, and alignment techniques. * Familiarity with prompt lifecycle management and versioning. * Experience with Galileo, LangSmith, Splunk, or similar observability platforms. * Familiarity with LlamaIndex, Haystack, or other GenAI frameworks. * Experience with evaluation and guardrails for agentic AI workflows. * Self-directed, collaborative working style across engineering teams. Skills: * Application Development * Automation * Influence * Solution Design * Technical Strategy Development * Architecture * Business Acumen * DevOps Practices * Result Orientation * Solution Delivery Process * Analytical Thinking * Collaboration * Data Management * Risk Management * Test Engineering ## Description This role centers on engineering and operating third-party applications end to end within an on-premises OpenShift platform. The position supports GenAI operations, including guardrails and observability for LLM and RAG-based applications across the enterprise. The engineer will deploy and support vendor platforms such as Galileo and LangSmith, monitor AI application quality and safety, maintain model evaluation workflows, and provide operational support in a fast-paced environment. The domain is GenAI operations: the applications you run provide guardrails and observability for LLM- and RAG-powered systems across the enterprise. You will work with these vendor tools to monitor application quality, safety, and performance-tracking metrics such as attribution, grounding, prompt-injection detection, tone, bias, and PII handling-and surface them through dashboards and alerting. Where the platform uses model-based evaluation (LLMs-as-Judges and specialized/small language models), you will help operationalize, calibrate, and maintain those workflows. Because the AI landscape changes quickly, the ideal engineer learns new tools and environments fast, brings a pragmatic build-and-operate mindset, and can lead cross-functional work to move initiatives forward. Responsibilities: * Develop, deploy, and operate vendor applications in an on-premises OpenShift environment. * Build and maintain APIs and integrations connecting vendor guardrail and observability platforms to enterprise GenAI applications. * Monitor application performance, safety, latency, throughput, and guardrail metrics. * Implement and maintain model-based evaluation workflows, including annotation and calibration processes. * Utilize orchestration frameworks such as LangChain for guardrail checks, prompt management, and scoring workflows. * Build observability dashboards and alerting solutions. * Troubleshoot production issues and optimize performance, scalability, and resilience. * Contribute to emerging AI evaluation and guardrail strategies for agentic workflows. * Lead and mentor engineers while driving cross-functional initiatives and engineering best practices., Bank of America and its affiliates consider for employment and hire qualified candidates without regard to race, religious creed, religion, color, sex, sexual orientation, genetic information, gender, gender identity, gender expression, age, national origin, ancestry, citizenship, protected veteran or disability status or any factor prohibited by law, and as such affirms in policy and practice to support and promote the concept of equal employment opportunity, in accordance with all applicable federal, state, provincial and municipal laws. The company also prohibits discrimination on other bases such as medical condition, marital status or any other factor that is irrelevant to the performance of our teammates. View your "Know your Rights (https://www.eeoc.gov/sites/default/files/2023-06/22-088_EEOC_KnowYourRights6.12.pdf) " poster. View the LA County Fair Chance Ordinance (https://dcba.lacounty.gov/wp-content/uploads/2024/08/FCOE-Official-Notice-Eng-Final-8.30.2024.pdf) . ## Related Videos - [Our journey with Spring Boot in a microservice architecture](https://www.wearedevelopers.com/videos/511-our-journey-with-spring-boot-in-a-microservice-architecture) - [From DevOps to Scaled DevOps: How We’re Rebuilding Continuous Delivery as a Platform](https://www.wearedevelopers.com/videos/100018-from-devops-to-scaled-devops-how-we-re-rebuilding-continuous-delivery-as-a-platform) - [This Is Not Your Father's .NET](https://www.wearedevelopers.com/videos/967-this-is-not-your-father-s-net) - [Pioneering AI Assistants in Banking](https://www.wearedevelopers.com/videos/1627-pioneering-ai-assistants-in-banking) - [The Missing Layer Between Enterprise Data and AI Agents](https://www.wearedevelopers.com/videos/100286-the-missing-layer-between-enterprise-data-and-ai-agents) - [DevOps Maturity Check – a way to balance autonomy and alignment](https://www.wearedevelopers.com/videos/58-devops-maturity-check-a-way-to-balance-autonomy-and-alignment) ## Related Articles - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [Navigating the AI Shift](https://www.wearedevelopers.com/magazine/629-navigating-the-ai-shift) - [MLOps And AI Driven Development](https://www.wearedevelopers.com/magazine/82-mlops-and-ai-driven-development) - [Graph and AI Trends 2026: Why Is AI Running but Not Yet Delivering?](https://www.wearedevelopers.com/magazine/680-graph-and-ai-trends-2026-why-is-ai-running-but-not-yet-delivering) - [Got AI ideas but no money? Here are 10 free ways to level up your AI skills with Google Cloud](https://www.wearedevelopers.com/magazine/600-got-ai-ideas-but-no-money-here-are-10-free-ways-to-level-up-your-ai-skills-with-google-cloud) - [What Industries Outside of AI Are Hiring The Most AI Experts?](https://www.wearedevelopers.com/magazine/98-what-industries-outside-of-ai-are-hiring-the-most-ai-experts)