> Markdown version of [/jobs/ext/2024337-software-development-engineer](https://www.wearedevelopers.com/jobs/ext/2024337-software-development-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Software Development Engineer - **Company:** Sony Interactive Entertainment - **Location:** San Mateo, CA, United States - **Experience:** Experienced - **Salary:** $140,500.0 - $210,700.0 - **Contract:** Permanent contract - **Skills:** Testing (Software), Java (Programming Language), JavaScript (Programming Language), Test Suite, Application Programming Interfaces (APIs), Agile Methodology, Artificial Intelligence, Amazon Web Services, Automation of Tests, Cloud Computing, Cyber Security, Databases, Continuous Delivery, Continuous Integration, Data Security, Software Debugging, Distributed Systems, Fraud Prevention and Detection, Github, Design of User Interfaces, Monitoring of Systems, Human-Computer Interaction, Internet Protocol, Junit, Python (Programming Language), Machine Learning, Node.Js, Online Service Provider, Systems Development Life Cycle, Regression Testing, Release Management, Reliability Engineering, Software Reliability Testing, Azure Machine Learning, Selenium, Software Engineering, System Testing, Test Case, Scripting, Google Cloud, Performance Testing, ReactJS, Appium, Large Language Models, Grafana, Model Validation, Cypress (Programming Language), Backend, Pytest, Containerization, Integration Tests, Kubernetes, Information Technology, Playwright, Data Management, Machine Learning Operations, Front End Software Development, Restful APIs, Data Pipelines, Docker, SDET, Jenkins, Databricks, Microservices - **Published:** August 11, 2026 - **Apply:** https://www.careerbuilder.com/job-details/software-development-engineer-in-test-ml-ai-san-mateo-ca--465eb371-d36b-4621-b9b8-e61dfc3a66f7 ## About the Role * Bachelor's degree in Computer Science or equivalent practical experience. * 3+ years of experience as an SDET or QE engineer focused on backend and distributed systems. * Experience using LLMs to generate, transform, and prioritize test cases for AI-powered experiences. * Experience with AI evaluation tooling, prompt evaluation frameworks, model monitoring, or human-in-the-loop review workflows. * Strong experience testing RESTful APIs, microservices, and distributed architectures. * Proficiency in Python, Java, JS or similar languages for automation development. * Hands-on experience with automation frameworks such as pytest, JUnit, Selenium, Playwright, Cypress, or Appium. * Experience with CI/CD systems and test pipelines (Jenkins, GitHub Actions, etc.). * Experience with cloud and container technologies (AWS, GCP, Kubernetes, Docker). * Familiarity with databases, monitoring, and observability tools. * Strong understanding of SDLC, Agile methodologies, and release processes. * Excellent problem-solving, debugging, and communication skills., * Experience validating ML outputs using statistical analysis or scenario-based testing approaches. * Familiarity with ML infrastructure, data pipelines, or model-serving platforms (Seldon, KServe, Ray Serve, etc.). * Prior work in content moderation ML, security, fraud detection, or adversarial ML. * Experience testing high-scale, low-latency online services. * Experience with Databricks or similar ML platform tooling. * Familiarity with Node.js, React, or modern frontend technologies. * Experience testing mobile, console, or other non-PC platforms. What sets you apart * Strong combination of automation engineering and delivery ownership. * Ability to drive quality across complex cross-functional initiatives. * Practical understanding of how to test non-deterministic AI systems and separate model variance from quality regressions. * Proven risk management and dependency coordination skills. * Ability to influence engineering teams and promote quality guidelines. * Passion for scalable, reliable, and maintainable automation systems., Agile Programming Methodologies, Amazon Web Services (AWS), Application Programming Interface (API), Artificial Intelligence (AI), Automation, Automation Engineering, Automation Systems, Background Investigation, Cloud Computing, Communication Skills, Computer Science, Continuous Deployment/Delivery, Continuous Integration, Cross-Functional, Data Management, Debugging Skills, Distributed Computing, Docker, Embedded Systems, Establish Priorities, Functional Testing, GCP (Good Clinical Practices), GitHub, IP (Internet Protocol), Information/Data Security (InfoSec), Integration Testing, Internet/Online Service, JUnit, Java, JavaScript, Jenkins, Machine Learning, Machine Tool, Microservices, Node.js, Organizational Skills, Performance Testing, PlayStation, Problem Solving Skills, Pytest, Python Programming/Scripting Language, Quality Assurance, Quality Metrics, REST (Representational State Transfer), React.js, Regression Testing, Release Management/Engineering, Reliability Engineering, Reliability Testing, Risk, Risk Analysis, Risk Management, Root Cause Analysis, Scalable System Development, Selenium, Software Design for Test (SDET), Software Development Lifecycle (SDLC), Software Engineering, Software Testing, Statistics, Strategic Planning, System Test, Team Building, Team Lead/Manager, Technical Delivery, Technical Recruiting, Test Automation, Test Case, Test Plan/Schedule, Test Scenario, Test Suite, Testing, User Interface/Experience (UI/UX), Validation Testing ## Description We are hiring a Software Engineer in Test specializing in ML/AI quality, including automation for model evaluation, LLM-assisted test generation, and validation of AI-powered workflows. The role includes guiding quality strategy and developing automation frameworks. You will also manage implementation for complex, cross-functional, machine learning-powered products and services. This role involves more than test development. You will lead quality initiatives from start to finish, ensuring teams stay synchronized, dependencies are managed, and releases are delivered confidently. You will work closely with ML, engineering, product, and infrastructure teams. You will shape how quality is built, monitored, and expanded while leading embedded QE efforts within projects. What you'll do * Define and complete quality strategies, test plans, and automation coverage for ML-powered services and platform components. * Use LLMs and other AI-assisted techniques to generate, expand, and maintain high-value test cases for ML-powered workflows. * Design scenario-based test suites for AI features, including adversarial prompts, edge cases, ambiguous inputs, and underrepresented user scenarios. * Lead QE efforts for multi-functional projects, driving risk assessment, dependency management, and release readiness. * Design, develop, and maintain scalable automation frameworks for backend services, APIs, and ML inference systems using Python and/or Java. * Build automated validation for ML and LLM outputs, including ranking behavior, score distributions, prompt/response quality, hallucination indicators, and probabilistic model evaluation. * Debug test failures, service anomalies, model inconsistencies, and AI behavior regressions to identify root causes and drive resolution. * Perform functional, integration, regression, API, end-to-end, performance, and reliability testing for distributed systems. * Improve automation reliability, reduce flakiness, and optimize execution efficiency. * Partner with engineering and ML teams to integrate automated testing into CI/CD pipelines and release workflows. * Collaborate across teams to establish scalable quality standards, tooling, and guidelines. ## Related Videos - [AI as a Test Designer: Transforming Experience into Automated Testing](https://www.wearedevelopers.com/videos/1984-ai-as-a-test-designer-transforming-experience-into-automated-testing) - [pytest: Simple, rapid and fun testing with Python](https://www.wearedevelopers.com/videos/213-pytest-simple-rapid-and-fun-testing-with-python) - [How Unit Testing Saved My Career](https://www.wearedevelopers.com/videos/1642-how-unit-testing-saved-my-career) - [Automagic Configuration in Python](https://www.wearedevelopers.com/videos/363-automagic-configuration-in-python) - [From boy scouting to redrawing the landscape](https://www.wearedevelopers.com/videos/1140-from-boy-scouting-to-redrawing-the-landscape) - [Let’s Talk Quality!](https://www.wearedevelopers.com/videos/100012-let-s-talk-quality) ## Related Articles - [The 8 Best Code Testing Tools](https://www.wearedevelopers.com/magazine/402-the-8-best-code-testing-tools) - [Navigating the AI Shift](https://www.wearedevelopers.com/magazine/629-navigating-the-ai-shift) - [What is Software Engineering?](https://www.wearedevelopers.com/magazine/289-what-is-software-engineering) - [Dev Digest 120 - Apple and peers](https://www.wearedevelopers.com/magazine/455-dev-digest-120-apple-and-peers) - [Is Software Engineering Over-Saturated?](https://www.wearedevelopers.com/magazine/418-is-software-engineering-over-saturated) - [What is Software Engineering in the Age of AI?](https://www.wearedevelopers.com/magazine/640-what-is-software-engineering-in-the-age-of-ai)