AI and ML Platforms QA Engineer III

The QA Engineer for AI and ML Platforms establishes and evolves quality engineering practices for cloud-native and AI/ML platform solutions, ensuring reliability, scalability, and reproducibility across data, model, and infrastructure layers. The position serves as a key contributor in defining test strategies for distributed systems, data pipelines, and machine learning workflows within AWS and Snowflake environments., Executes tasks that span across multiple systems/processes for one or more business projects/areas.

• Reviews, analyzes, and evaluates business and IT systems and user needs to determine the best test approach.

• Estimates, prioritizes, plans, and coordinates test activities.

• Identifies, records, thoroughly documents, and tracks bugs.

• Partners with the Development team to perform test via standards based on APIs for communicating between applications.

• Tracks quality assurance metrics including reporting on pre-defined KPI, defect densities, and open defects counts.

• Influences quality engineering best practices within the scrum team.

• Executes and maintains automated regression test suites using open source tools.

• Follows standards in accordance with company policy and regulation (MAR, HIPAA, ISO 2700, ISO 27001, etc.).

• Recommends process improvement and strengthens the quality standards and implements those improvements on approval.

• Audits test deliverables to ensure they meet standards and takes corrective action to mitigate variances.

• Leads the QA efforts for large initiatives and collaborates closely with external vendor partners (offshore, nearshore) for work assignments, status calls, working sessions, triage calls etc.

Develops detailed, comprehensive, and well-structured test plans and test cases.

• Develops, publishes, and implements test strategy, test plans, and test scripts.

• Develops and applies testing processes to new and existing products to meet stakeholders needs.

• Develops and maintains automated test harnesses and libraries.

• Develops and maintains testing standards, procedures, and guidelines to ensure consistent testing procedures.

• Creates tests that cover functionality, load, security, etc. at the API and GUI levels.

• Identifies risks and develops mitigation strategies.

Becomes a subject matter expert for one or more business areas and technical product.

• Delivers quality process training to technical staff.

• Acts as an internal quality consultant to advise or influence business or technical partners.

• Performs quality audits across the various IT functions to ensure that quality standards, procedures, and methodologies are being followed.

• Partners with distributed teams, outside vendors, or offshore testing partners.

• Coaches and educates others to increase early defect detection rates and decrease defect escape rates.

Defines and Implements quality strategies for cloud-native AI and ML platforms.

• Partners with Data Scientists and ML Engineers to design and execute testing strategies for AI/ML applications and pipelines, including pipeline validation, data validation, feature and model validation, inference testing, integration and API testing.

• Implements testing strategies for infrastructure as code (IaC), including validation of AWS resources provisioned via tools such as Terraform or CloudFormation.

• Establishes automated quality gates within CI/CD pipelines to enforce testing standards across build, deploy, and release stages.

• Defines observability and quality signals, including logs, metrics, and traces, to proactively detect defects in production systems. Contributes hands-on to debugging and root cause analysis across application, data, and platform layers.

• Leads validation of non-functional requirements including performance, scalability, reliability, and resiliency of AI and ML platforms., This role does not have direct reports but may mentor and/or train less experienced team members.

Bachelor's degree from four-year college or university IN Computer Science, Information Systems or related field, or equivalent work experience. Minimum 4 years of related experience. Minimum 5 years of IT and/or relevant business experience.

CERTIFICATIONS, LICENSES, REGISTRATIONS

Certificated Software Test Engineer (CSTE) or International Software Testing Qualification Board (ISTQB) preferred. CP-SAT Selenium Automation Testing certification preferred. Foundation level API certification preferred.

KNOWLEDGE AND SKILLS

• Ability to anticipate user behavior or risks to systems; ability to work independently; experience querying and using data to enhance testing approach/scope.

• Ability to quickly understand and gain domain expertise with complex existing applications and architecture.

• Ability to validate data transformations and perform End-to-End validations for ETL/ API enabled integrated systems.

• Ability to work effectively with geographically remote and culturally diverse teams.

• Experience in analyzing requirements to reduce ambiguity and increase testability; experience developing and reviewing test plans, test cases, and test scripts.

• Experience mentoring junior (less experienced) QA team members.

• Knowledge of Insurance Industry best practices and business workflows a plus, and Industry standard certification in Quality Assurance is strongly desired.

• Must have a customer-service mentality to resolve/avoid conflict and enable a team approach. Ability to speak in front of teams, stakeholders, and management.

• Must have ability to learn, analyze, and interpret technical documentation such as software requirements, detailed designs, flow charts and database schemas.

• Strong experience with SQL and SQL and database schema designs and experience with building automation frameworks is a must.

• Understanding of the software development life cycle processes; understanding of different testing methodologies and their proper application.

• Work effectively in both Agile/Scrum and Waterfall (for specific external vendor-led efforts) frameworks.

• Expertise with automation feasibility definition and hands-on execution experience; ability to train others on automation practices and tools.

• Works in an agile manner and must be flexible to changing priorities.

• Experience testing ML frameworks and tools, preferably SageMaker, MLFlow, SnowflakeML.

• Experience validating data platforms, ETL/ELT pipelines, or data warehousing solutions such as Snowflake.

• Hands-on experience on AI/ML pipeline validation, data validation, model validation, integration and API testing.

• Experience implementing automated testing within CI/CD pipelines and working in DevOps-oriented environments.

• Experience with testing cloud platforms and cloud resource provisioning (AWS preferred), Infrastructure as Code (IaC), containerization tools like Docker and Kubernetes.

• We offer a competitive benefits package, with generous medical, dental, and vision plans as well as 401K retirement plans and company match

• Bonus potential for all positions

• Paid Time Off

• Paid holidays throughout the calendar year

Are you looking to make an impactful difference in your work, yourself, and your community? Why settle for just a job when you can land a career? At ICW Group, we are hiring team members who are ready to use their skills, curiosity, and drive to be part of our journey as we strive to transform the insurance carrier space. We're proud to be in business for over 50 years, and its change agents like yourself that will help us continue to deliver our mission to create the best insurance experience possible. Headquartered in San Diego with regional offices located throughout the United States, ICW Group has been named for ten consecutive years as a Top 50 performing P&C organization offering the stability of a large, profitable and growing company combined with a focus on all things people. It's our team members who make us an employer of choice and the vibrant company we are today. We strive to make both our internal and external communities better everyday! Learn more about why you want to be here!, At ICW Group we offer a work environment that encourages entrepreneurialism and celebrates success. Our team members are hands-on contributors who are given the opportunity to make an impact. It's our people who make us an employer of choice and the vibrant company we are today.