Senior Software Tester

M&S Consulting is seeking a Senior Software Tester to serve as the senior individual contributor for platform and AI testing on the Department of Labor National Contact Center (DOL NCC) Blanket Purchase Agreement, where M&S is the Salesforce and Agentforce delivery partner under the prime contractor. This role designs and executes the test cases, evaluation transcripts, and scoring logic that validate the automated contact center solution on Salesforce, with a strong emphasis on DOL’s Agentforce / AI evaluation program. The Senior Software Tester works under the Software Testing Manager, partners closely with the Senior Software Developer / Manager and the developers, and helps calibrate the DOL NCC cross-functional testing team. This is a fully dedicated position within the M&S software delivery scope., Design, document, and execute functional, integration, regression, and user acceptance test cases for the Salesforce and Agentforce contact center solution.

• Build and run AI evaluations, including test cases with complete transcripts and scoring logic, regression evals, faithfulness checks, benchmarking, LLM-as-a-Judge grading, and red teaming / adversarial testing to confirm outputs stay within configured guardrails.
• Serve as a human-in-the-loop reviewer, scoring AI outputs to calibrate automated graders and validate the accuracy, reliability, safety, and quality of AI-generated content.
• Log, triage, and track defects to resolution; verify fixes; and maintain traceability from test cases to contract requirements.
• Validate Service Cloud, Service Cloud Voice, Open CTI, and integration behavior, including case creation, routing, and knowledge and content alignment.
• Produce clear test documentation, evaluation results, and release evidence, and present findings to the Software Testing Manager and stakeholders.
• Coordinate with IT, Content, Operations, Quality, and Training stakeholders to support continuous improvement across DOL NCC AI channels.
• Support NIST SP 800-53, FISMA, and FedRAMP Moderate-aligned test and release evidence., Dedication: Fully dedicated to the DOL NCC engagement.
• Role designation: Software delivery support staff (not a Key Personnel position).
• Reporting: Reports to the Software Testing Manager and coordinates with the Senior Software Developer / Manager and development team.
• Employer: M&S Consulting.

Requires current Salesforce Certified Agentforce Specialist certification and 3+ years of software testing, QA, or Salesforce administration experience. This position is contingent upon contract award., Minimum of three years of software testing, quality assurance, or Salesforce configuration / administration experience.

• Salesforce Certified Agentforce Specialist, current and in good standing.
• Demonstrated experience testing, evaluating, or monitoring Salesforce applications or AI / large language model outputs against defined criteria.
• Ability to have any Salesforce or Agentforce certification level submitted to and approved by the DOL Contracting Officer’s Representative (COR) before performing any coding or configuration on the platform.
• Bachelor of Arts or Bachelor of Science degree, or equivalent relevant experience. Information systems, business administration, or computer science preferred.
• Public Trust Low Risk suitability, or the ability to begin and complete the Public Trust Low Risk suitability process at award., Hands-on experience testing and monitoring AI agents for continuous improvement, including federal or Department of Labor Agentforce / agentic AI work.
• Experience with an AI evaluation harness or framework and with LLM-as-a-Judge, faithfulness, and regression evaluation methods.
• Service Cloud, Experience Cloud, and case management at contact center scale, including Service Cloud Voice and Open CTI.
• Salesforce administration or architecture depth, such as Administrator, Advanced Administrator, Platform App Builder, Sharing and Visibility Architect, Data Architect, or Data Cloud.
• Experience with service analytics, reporting, and dashboards for quality and performance monitoring.
• Familiarity with defect-tracking and CI/CD tooling such as Jira, Git, Salesforce DX, Copado, or Gearset.
• Federal delivery aligned to NIST SP 800-53, FISMA, and FedRAMP Moderate.