Technical Architect

CloudSufi is hiring a Senior AI Architect to lead the design of an AI agent testing and evaluation platform for a large enterprise client. You will define how the client’s fleet of AI agents is tested before and after deployment, set the technical standard for evaluation, and act as the senior technical lead working directly with client stakeholders. This is a client-facing, on-site role for someone who can both architect the system and lead the room. Responsibilities * Own the end-to-end technical design of an agent testing framework covering deterministic checks, statistical (LLM-based) evaluation, and adversarial testing.

• Define the methodology for evaluation datasets, safety rules, and judge calibration.
• Design integration with the client’s CI/CD pipeline and cloud environment so quality gates block unsafe deployments.
• Lead technical discussions with client architects and engineering leadership, and set standards other teams follow.
• Guide and review the work of a distributed delivery team.

8+ years in software engineering or machine learning, including 3+ years building LLM or AI agent systems.

• Hands-on expertise with the Google Cloud AI stack (Vertex AI, Gemini, Agent Development Kit, Dialogflow CX) or close equivalents.
• Strong understanding of LLM evaluation, prompt engineering, and agent architectures.
• Proven ability to lead client-facing technical engagements and set architectural direction., Experience with LLM-as-judge evaluation, RAG systems, and agent observability.
• Background delivering production AI systems in a regulated or large-scale consumer environment.

CLOUDSUFI, a Google Cloud Premier Partner, is a global leading provider of data-driven digital transformation across cloud-based enterprises. With a global presence and focus on Software & Platforms, Life sciences and Healthcare, Retail, CPG, financial services and supply chain, CLOUDSUFI is positioned to meet customers where they are in their data monetization journey. Our Values We are a passionate and empathetic team that prioritizes human values. Our purpose is to elevate the quality of lives for our family, customers, partners and the community. Equal Opportunity Statement CLOUDSUFI is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees. All qualified candidates receive consideration for employment without regard to race, colour, religion, gender, gender identity or expression, sexual orientation and national origin status. We provide equal opportunities in employment, advancement, and all other areas of our workplace. Please explore more at https://www.cloudsufi.com