> Markdown version of [/jobs/ext/2793573-senior-developer-artificial-engineering](https://www.wearedevelopers.com/jobs/ext/2793573-senior-developer-artificial-engineering). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Senior Developer, Artificial Engineering - **Company:** Eisner Advisory Group LLC - **Location:** United States - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** A/B Testing, Artificial Intelligence, Data Analysis, Automation of Tests, Data Cleansing, Python (Programming Language), Machine Learning, Recommender Systems, Regression Testing, Selenium, Chatbots, Postman, Retrieval-Augmented Generation, Large Language Models, Prompt Engineering, Model Validation, Pytest, Information Technology, Deployment Automation, Machine Learning Operations, Data Pipelines - **Published:** September 8, 2026 - **Apply:** https://eisneramper.wd1.myworkdayjobs.com/EisnerAmper_External/job/Bangalore/Artificial-Intelligence-Specialist_Req-9002 ## About the Role * Bachelor's or Master's degree in Computer Science, Engineering, or a related field * 3-6 years of QA experience, with at least 1-2 years in AI/ML quality assurance * Strong proficiency in Python for test automation and data analysis * Familiarity with LLM evaluation frameworks (e.g., RAGAS, DeepEval, Promptfoo, LangSmith) * Hands-on experience with testing tools: Pytest, Selenium, Postman, or similar * Understanding of ML lifecycle - training, validation, deployment, and monitoring * Knowledge of data quality tools and pipeline testing (Great Expectations, dbt tests) Nice to Have * Experience with prompt engineering and red-teaming LLMs * Familiarity with MLOps platforms (MLflow, SageMaker, Vertex AI) * Knowledge of vector databases and embedding quality evaluation * Understanding of AI safety, responsible AI principles, and fairness frameworks * Experience with A/B testing and shadow deployment strategies Soft Skills * Analytical and inquisitive mindset - comfortable challenging model outputs * Ability to think like both a user and an adversary (red-team thinking) * Strong documentation and communication skills * Collaborative approach with data science, engineering, and product teams * High attention to detail with a quality-first attitude ## Description A QA Engineer for AI Initiatives is responsible for ensuring the quality, reliability, fairness, and performance of AI/ML-powered products and systems. Unlike traditional QA, this role requires deep understanding of non-deterministic model behavior, data quality, and AI-specific failure modes such as hallucinations, bias, and model drift., * Design and execute test strategies specifically for AI/ML models, LLM-based applications, and data pipelines * Develop automated test frameworks for model validation, regression testing, and performance benchmarking * Evaluate model outputs for accuracy, consistency, relevance, hallucination, and bias across diverse inputs * Test RAG (Retrieval-Augmented Generation) pipelines, chatbots, recommendation systems, and other AI-driven features * Collaborate with data scientists and ML engineers to define acceptance criteria and quality thresholds * Build and maintain evaluation datasets, ground truth sets, and adversarial test cases * Monitor models in production for drift, degradation, and anomalous behavior * Validate data quality, data pipelines, and feature stores that feed AI systems * Document defects, edge cases, and failure patterns specific to AI behavior * Ensure AI systems meet ethical, fairness, and compliance standards (bias audits, explainability checks) ## Related Videos - [pytest: Simple, rapid and fun testing with Python](https://www.wearedevelopers.com/videos/213-pytest-simple-rapid-and-fun-testing-with-python) - [Are Your APIs Ready for AI Agents](https://www.wearedevelopers.com/videos/2004-are-your-apis-ready-for-ai-agents) - [Chatbots are going to destroy infrastructures and your cloud bills](https://www.wearedevelopers.com/videos/1130-chatbots-are-going-to-destroy-infrastructures-and-your-cloud-bills) - [AI as a Test Designer: Transforming Experience into Automated Testing](https://www.wearedevelopers.com/videos/1984-ai-as-a-test-designer-transforming-experience-into-automated-testing) - [Testing AI Agents: Automated Evaluation for Chatbots & RAG Systems](https://www.wearedevelopers.com/videos/100300-testing-ai-agents-automated-evaluation-for-chatbots-rag-systems) - [Automagic Configuration in Python](https://www.wearedevelopers.com/videos/363-automagic-configuration-in-python) ## Related Articles - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [What Are Large Language Models?](https://www.wearedevelopers.com/magazine/304-what-are-large-language-models) - [MLOps And AI Driven Development](https://www.wearedevelopers.com/magazine/82-mlops-and-ai-driven-development) - [Navigating the AI Shift](https://www.wearedevelopers.com/magazine/629-navigating-the-ai-shift) - [Got AI ideas but no money? Here are 10 free ways to level up your AI skills with Google Cloud](https://www.wearedevelopers.com/magazine/600-got-ai-ideas-but-no-money-here-are-10-free-ways-to-level-up-your-ai-skills-with-google-cloud) - [The State of WebDev AI 2025 Results: What Can We Learn?](https://www.wearedevelopers.com/magazine/581-the-state-of-webdev-ai-2025-results-what-can-we-learn)