> Markdown version of [/jobs/ext/4668543-software-development-engineer](https://www.wearedevelopers.com/jobs/ext/4668543-software-development-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Software Development Engineer - **Company:** PlayStation - **Location:** Aliso Viejo, CA, United States - **Experience:** Experienced - **Salary:** $120,640 - $149,760 - **Contract:** Permanent contract - **Skills:** AI Evaluation, Java (Programming Language), JavaScript (Programming Language), Application Programming Interfaces (APIs), Agile Methodology, Artificial Intelligence, Amazon Web Services, User Authentication, Automation of Tests, Software Bug Management, Code Coverage, Databases, Software Debugging, Distributed Systems, JSON, Python (Programming Language), Regression Testing, E2e Testing, Swagger, Software Engineering, Systems Integration, Test Data, Strategies of Testing, TypeScript, Openapi, Retrieval-Augmented Generation, Large Language Models, Grafana, Hallucination Detection, Generative AI, Backend, Agentic-AI, Git, Event Driven Architecture, LangSmith, Kubernetes, Information Technology, RAGAS (Retrieval Augmented Generation Assessment), Arize Phoenix, Apache Kafka, DeepEval, Evaluation of Large Language Models, Artificial Intelligence Governance, Restful APIs, Api Management, Promptfoo, Docker, SDET, Programming Languages, Microservices - **Published:** November 11, 2026 - **Apply:** https://job-boards.greenhouse.io/sonyinteractiveentertainmentglobal/jobs/6181170004 ## About the Role We are looking for an experienced Software Development Engineer in Test (SDET) to help ensure the quality, reliability, and resilience of Agentic AI systems and backend services. This role will design and implement automated testing strategies across AI/LLM-driven applications, APIs, microservices, and distributed systems. The ideal candidate brings strong Quality Engineering and test automation fundamentals, with experience testing Generative AI, LLM-based, or Agentic AI systems. They should be comfortable testing complex systems where behavior may be non-deterministic and traditional pass/fail testing alone may not be sufficient., * Bachelor's degree in Computer Science, Engineering, Information Technology, or a related field, or equivalent practical experience. * 4+ years of experience in Software Development Engineering in Test, Quality Engineering, Software Engineering, or a related discipline. * Strong experience testing REST APIs, backend services, microservices, or distributed systems. * Hands-on experience developing automated tests using Python, Java, JavaScript, TypeScript, or a similar programming language. * Experience with functional, integration, regression, contract, and end-to-end testing. * Experience integrating automated tests into CI/CD pipelines. * Strong analytical, debugging, and root-cause-analysis skills. * Experience working in an Agile development environment and collaborating across engineering teams., * Strong understanding of software testing methodologies and Quality Engineering practices. * Strong API and services testing experience, including REST, JSON, schemas, authentication, and error handling. * Experience designing and maintaining automated test frameworks. * Experience with Git and CI/CD development workflows. * Ability to develop risk-based test strategies and identify appropriate testing layers. * Experience testing distributed systems and service-to-service integrations. * Hands-on experience testing Generative AI, LLM-powered applications, or Agentic AI systems. * Understanding of testing non-deterministic AI behavior and evaluating outputs beyond exact-match assertions. Preferred Skills (Good to have) * Experience testing Agentic AI workflows, including planning, multi-step execution, tool calling, memory, or state management. * Experience with AI evaluation techniques such as evaluation datasets, semantic evaluation, LLM-as-a-Judge, groundedness, relevance, or hallucination detection. * Experience with AI/LLM evaluation or observability tools such as LangSmith, DeepEval, Ragas, Arize Phoenix, Promptfoo, or similar technologies. * Experience testing RAG systems or AI agents interacting with APIs, databases, or external tools. * Experience with contract testing tools such as Pact and API specifications such as OpenAPI/Swagger. * Experience with event-driven architectures, Kafka, Docker, Kubernetes, or cloud platforms such as AWS. * Experience with performance, scalability, reliability, or resiliency testing. * Experience establishing QE metrics, quality gates, or cross-team testing standards. ## Description * Develop and execute comprehensive QE strategies and test plans for Agentic AI systems, backend services, APIs, and service integrations. * Design and maintain scalable automated test frameworks across API, functional, integration, contract, regression, and end-to-end testing. * Develop testing approaches for non-deterministic AI behavior and establish meaningful evaluation criteria. * Identify quality risks early and apply risk-based, shift-left testing practices throughout the development lifecycle. * Integrate automated testing into CI/CD pipelines and establish effective quality gates. * Partner with Software Engineers, Product Managers, and other QE teams to improve testability, observability, and overall product quality. * Analyze defects and production issues to identify root causes, testing gaps, and opportunities for continuous QE improvement. * Establish quality metrics and provide visibility into test coverage, reliability, defects, and release readiness. What You'll Do * Test Agentic AI workflows, including multi-step execution, tool selection and invocation, task completion, and failure handling. * Validate AI-generated responses using appropriate measures for accuracy, relevance, groundedness, consistency, and adherence to expected behavior. * Test AI guardrails, unexpected inputs, and failure scenarios, including incorrect tool usage and prompt-based attacks. * Validate interactions between AI agents, APIs, tools, data sources, and downstream services. * Develop reusable automation, test data, and evaluation scenarios for repeatable AI and services testing. * Validate API contracts, schemas, authentication, business rules, error handling, and service integrations. * Investigate failures across distributed systems and distinguish application, integration, environmental, automation, and AI-related issues. * Participate in defect triage, root-cause analysis, regression testing, and release-readiness activities.