> Markdown version of [/jobs/ext/3455605-flex-ai-software-engineer-software-development-engineer-in-test](https://www.wearedevelopers.com/jobs/ext/3455605-flex-ai-software-engineer-software-development-engineer-in-test). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # FLEX AI Software Engineer, Software Development Engineer in Test - **Company:** Marriott International, Inc. - **Location:** Bethesda, MD, United States (Remote available) - **Experience:** Experienced - **Salary:** $87,298.0 - $143,998.0 - **Contract:** Temporary contract - **Skills:** AI Evaluation, Java (Programming Language), JavaScript (Programming Language), Application Programming Interfaces (APIs), Artificial Intelligence, User Authentication, Automation of Tests, Cloud Computing, Code Review, Computer Programming, Cross-Origin Resource Sharing (Ajax Programming), Data Security, Python (Programming Language), Open Source Technology, Coupa Supplier Portal, Service Virtualization, Smoke Testing, Software Engineering, Test Execution Engine, TypeScript, Workflow Management Systems, Data Processing, Performance Testing, Chatbots, Retrieval-Augmented Generation, Large Language Models, Generative AI, Kubernetes, Information Technology, Low Latency, Browser Testing, Evaluation of Large Language Models, Software Version Control, Api Management, SDET - **Published:** September 30, 2026 - **Apply:** https://ejwl.fa.us2.oraclecloud.com/hcmUI/CandidateExperience/en/sites/MI_CS_1/job/26123877/apply/email?posting_type=1 ## About the Role * Bachelor's degree in Computer Science, Software Engineering, Information Technology, or a related field, or equivalent professional experience. * 2+ years of experience in quality engineering, test automation, or SDET roles. * Strong programming experience in Python, JavaScript, TypeScript, Java, or a comparable language, with hands-on experience in browser automation, API testing, automated test frameworks, performance testing, source control, code review, and CI/CD quality gates. * Experience testing conversational AI, generative AI, retrieval-augmented generation, intelligent search, chatbot, or agent-based applications, including non-deterministic evaluation, cloud-native systems, containers, Kubernetes, observability, accessibility, and cross-functional delivery. Preferred * Experience with commercial or open-source LLM evaluation platforms and with designing LLM-as-judge, deterministic, human-evaluation, prompt-testing, model-comparison, and retrieval-ranking workflows. * Experience with responsible-AI validation, red teaming, adversarial datasets, contract testing, service virtualization, workflow orchestration, asynchronous systems, canary deployments, and production smoke testing. * Experience with localization, accessibility, responsive and cross-browser testing, authentication and authorization testing, CORS, CSP, sensitive-data security testing, quality dashboards, and executive release-readiness reporting. ## Description Marriott International is a global hospitality company with a portfolio of 31 brands and more than 8,500 properties across 138 countries and territories. Diversity, inclusion, and the well-being of associates are fundamental to Marriott's values and business goals. The FLEX Software Engineer, Software Development Engineer in Test, AI, leads quality engineering for production conversational AI and intelligent search experiences. This role combines software development in test, browser and API automation, AI evaluation, performance engineering, accessibility validation, and production reliability to establish measurable quality standards and release-readiness gates for accurate, grounded, secure, reliable, and consistent customer experiences., * Defines and owns the automated quality strategy for conversational AI applications and intelligent search experiences. * Builds and maintains browser, API, integration, contract, smoke, regression, and end-to-end automated test suites. * Develops automated evaluations for response correctness, relevance, grounding, hallucination risk, retrieval quality, ranking quality, tool selection, and tool-call accuracy. * Validates multi-turn conversation consistency, safety guardrails, prompt-injection resistance, responsible-AI requirements, and sensitive-data handling. * Converts business scenarios, customer journeys, and production defects into reusable regression datasets and automated tests. * Develops performance, load, stress, spike, scalability, and reliability tests for AI applications and supporting services. * Establishes measurable quality baselines for latency, throughput, error rates, concurrency, resource utilization, and release readiness. * Integrates automated tests, AI evaluations, and quality thresholds into CI/CD pipelines and progressive-delivery workflows. * Builds automated pre-deployment and post-deployment smoke tests that validate critical customer and system workflows. * Manages test environments, synthetic data, fixtures, service mocks, test credentials, and repeatable test execution capabilities. * Analyzes failures and production issues using logs, metrics, distributed traces, automation results, and AI evaluation data. * Produces quality dashboards, defect insights, evaluation summaries, and release-readiness reports for engineering and business stakeholders. * Collaborates with Engineering, Product, Platform, Security, Accessibility, Quality Engineering, and Responsible AI teams to improve testability and prevent defects earlier in development. * Evaluates emerging AI quality-engineering tools and techniques and applies pragmatic approaches appropriate for production systems. ## Related Videos - [AI as a Test Designer: Transforming Experience into Automated Testing](https://www.wearedevelopers.com/videos/1984-ai-as-a-test-designer-transforming-experience-into-automated-testing) - [Understanding Kubernetes in a visual way](https://www.wearedevelopers.com/videos/100085-understanding-kubernetes-in-a-visual-way) - [Chatbots are going to destroy infrastructures and your cloud bills](https://www.wearedevelopers.com/videos/1130-chatbots-are-going-to-destroy-infrastructures-and-your-cloud-bills) - [Testing AI Agents: Automated Evaluation for Chatbots & RAG Systems](https://www.wearedevelopers.com/videos/100300-testing-ai-agents-automated-evaluation-for-chatbots-rag-systems) - [Evals vs. Evil - AI and Package Security - Laurie Voss](https://www.wearedevelopers.com/videos/2131-evals-vs-evil-ai-and-package-security-laurie-voss) - [Instant KAI Sandboxes with vCluster: Multi-Tenant, Multi-Scheduler GPU Sharing](https://www.wearedevelopers.com/videos/100333-instant-kai-sandboxes-with-vcluster-multi-tenant-multi-scheduler-gpu-sharing) ## Related Articles - [Navigating the AI Shift](https://www.wearedevelopers.com/magazine/629-navigating-the-ai-shift) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [What Are Large Language Models?](https://www.wearedevelopers.com/magazine/304-what-are-large-language-models) - [Dev Digest 132 - Binging WADFlix?](https://www.wearedevelopers.com/magazine/473-dev-digest-132-binging-wadflix) - [13 AI Tools You Have to Try](https://www.wearedevelopers.com/magazine/219-13-ai-tools-you-have-to-try) - [The State of WebDev AI 2025 Results: What Can We Learn?](https://www.wearedevelopers.com/magazine/581-the-state-of-webdev-ai-2025-results-what-can-we-learn)