Senior Ai Test Automation Engineer

Ust
Málaga, Spain
1 day ago
Apply on www.buscojobs.com.es
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience required
5 years minimum
Working hours
Regular working hours

Tech stack

Clean Code Principles Application Programming Interfaces (APIs) Agile Methodology Artificial Intelligence JIRA User Authentication Automation of Tests DevOps Digital Technology Distributed Systems Python (Programming Language) Machine Learning
+13 more
Test Execution Engine Retrieval-Augmented Generation Large Language Models Gitlab Pytest Kubernetes Playwright Machine Learning Operations Evaluation of Large Language Models Data Pipelines Api Management Docker Microservices

Job description

We are looking for the very Top Talent…and we would be delighted if you were to join our team!More in details, UST is a multinational company based in North America, certified as a Top Employer and Great Place to Work company with over ** employees all over the world and presence in more than 35 countries. We are leaders on digital technology services, and we provide large-scale technologic solutions to big companies.What we look for?We are looking for a Senior AI Test Automation Engineer to lead and evolve the test automation strategy for an advanced AI & MLOps platform. You will combine strong hands?on expertise in Python-based automation with the ability to design scalable frameworks and ensure quality across complex systems, including APIs, UIs, data pipelines, and AI/ML-driven components such as LLMs and RAG systems.You bring a product mindset, end-to-end ownership, and experience working in Agile/SAFe environments, with a strong focus on quality engineering, reliability, and automation at scale. Exposure to AI/ML testing or a strong motivation to grow in this area is key.Main tasks and accountabilities will be:Own the design, architecture, and evolution of the test automation framework for the AI & MLOps platformDefine and implement scalable, reusable automation patterns (UI, API, data, BDD)Automate test cases using Python stack (Playwright, pytest, requests, pytest-bdd/Behave) applying clean code practicesDesign and execute AI/ML-specific tests (e.g., LLM evaluation, RAG validation, non?deterministic outputs)Integrate and orchestrate automated tests within CI/CD pipelines (GitLab), ensuring reliable execution across environmentsProvision and manage test environments using Docker and KubernetesPlan and execute regression, smoke, and release test suites across DEV, INT, UAT, and pre-PRODMonitor execution quality, manage flaky tests, and ensure trustworthy test resultsAnalyze defects, perform root cause analysis, and report issues with clear evidence in JiraEnsure full traceability from requirements to test execution and reportingCollaborate with cross-functional teams (ML Engineers, MLOps, Data Engineers, Product Owners)Act as Test Automation Ambassador, promoting best practices and guiding teams on tooling and strategyEnsure compliance with test policies, audit requirements, and data privacy standardsWhat do we expect from you?5-7+ years of experience in test automation with strong hands?on Python expertiseProven experience designing and owning automation frameworks (not just writing tests)Solid experience with Playwright, pytest, API testing, and BDD frameworksExperience integrating automation into CI/CD pipelines (GitLab or similar)Knowledge of Docker and basic Kubernetes conceptsUnderstanding of microservices, APIs, authentication, and distributed systemsExposure to AI/ML systems or strong interest in testing LLM?based and AI?driven applicationsExperience with test reporting, parallel execution, and flaky test handlingFamiliarity with Agile/SAFe environments and DevOps practicesExperience with Jira and test management tools (e.g., Xray)Strong communication, ownership, and collaboration skillsAwareness of security, data privacy, and working in regulated environments (finance, healthcare, etc.)Hybrid position in Madrid (3 days at client ?s office, Sol area, next to the underground).Full?time, 40 hours/week.What can we offer?23 days of Annual Leave plus the 24th and 31st of December as discretionary days!Numerous benefits (Health Care Plan, teleworking compensation, Life and Accident Insurances).Free access to several training platformsProfessional stability and career plansUST also, compensates referrals from which you could benefit when you refer professionals.The option to pick between 12 or 14 payments along the year.Real Work Life Balance measures (flexibility, WFH or remote work policy, compacted hours during summertime…)UST Club Platform discounts and gym Access discountsIn UST we are committed to equal opportunities in our selection processes and do not discriminate based on race, gender, disability, age, religion, sexual orientation or nationality. We have a special commitment to Disability & Inclusion, so we are interested in hiring people with disability certificate.#J-*****-Ljbffr

Requirements

We are looking for a Senior AI Test Automation Engineer to lead and evolve the test automation strategy for an advanced AI & MLOps platform. You will combine strong hands?on expertise in Python-based automation with the ability to design scalable frameworks and ensure quality across complex systems, including APIs, UIs, data pipelines, and AI/ML-driven components such as LLMs and RAG systems. You bring a product mindset, end-to-end ownership, and experience working in Agile/SAFe environments, with a strong focus on quality engineering, reliability, and automation at scale. Exposure to AI/ML testing or a strong motivation to grow in this area is key., 5-7+ years of experience in test automation with strong hands?on Python expertise Proven experience designing and owning automation frameworks (not just writing tests) Solid experience with Playwright, pytest, API testing, and BDD frameworks Experience integrating automation into CI/CD pipelines (GitLab or similar) Knowledge of Docker and basic Kubernetes concepts Understanding of microservices, APIs, authentication, and distributed systems Exposure to AI/ML systems or strong interest in testing LLM?based and AI?driven applications Experience with test reporting, parallel execution, and flaky test handling Familiarity with Agile/SAFe environments and DevOps practices Experience with Jira and test management tools (e.g., Xray) Strong communication, ownership, and collaboration skills Awareness of security, data privacy, and working in regulated environments (finance, healthcare, etc.)

Benefits & conditions

Hybrid position in Madrid (3 days at client ?s office, Sol area, next to the underground). Full?time, 40 hours/week. What can we offer? 23 days of Annual Leave plus the 24th and 31st of December as discretionary days! Numerous benefits (Health Care Plan, teleworking compensation, Life and Accident Insurances). Free access to several training platforms Professional stability and career plans UST also, compensates referrals from which you could benefit when you refer professionals. The option to pick between 12 or 14 payments along the year. Real Work Life Balance measures (flexibility, WFH or remote work policy, compacted hours during summertime…) UST Club Platform discounts and gym Access discounts In UST we are committed to equal opportunities in our selection processes and do not discriminate based on race, gender, disability, age, religion, sexual orientation or nationality. We have a special commitment to Disability & Inclusion, so we are interested in hiring people with disability certificate. #J-*****-Ljbffr

About the company

Málaga, España

We are looking for the very Top Talent…and we would be delighted if you were to join our team! More in details, UST is a multinational company based in North America, certified as a Top Employer and Great Place to Work company with over ** employees all over the world and presence in more than 35 countries. We are leaders on digital technology services, and we provide large-scale technologic solutions to big companies.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.buscojobs.com.es
Prepare application

Good distractions

Loading talks and stories from around this role…