Senior AI/ML Engineer

• Agent Orchestration & Backend Development: Design, implement, and optimize multi-agent orchestration workflows (A2A), prompt templating, dynamic intent routing, and conversation memory using Python, asyncio, and FastAPI.
• MCP & Tool Integration: Standardize and integrate Model Context Protocol (MCP) servers and tools across Avalara products (AvaTax, Returns, Tax Research, ECM, Licensing).
• Evaluations & Benchmarking: Build automated evaluation pipelines (LLM-as-a-judge, deterministic assertions, harness evals) to benchmark agent intent accuracy, routing precision (HSR/FSR), and execution latency.
• Gating & Entitlement Routing: Implement dynamic cohort-based feature gating and IAM-DS entitlement evaluation to support controlled beta rollouts and multi-tenant access controls.
• Performance & Reliability: Optimize token consumption, response streaming, context caching, rate-limiting, and error handling for high-concurrency agent interactions.
• Production Readiness & Observability: Build telemetry pipelines, structured logging, and monitoring dashboards across Datadog, Snowflake, and OpenTelemetry to track model accuracy and system health.

Artificial Intelligence (AI), Memory Hardware, Python Programming/Scripting Language, MCP - Microsoft Certified Professional, Engineering Change Management, Licensing, Tax Returns, Benchmarking, Access Control, Caching, * Professional Experience: 5+ years of software engineering experience in Python with strong expertise in high-throughput asynchronous services (asyncio, FastAPI).

• LLM & Agent Systems: 2+ years of hands-on experience building production LLM applications, agentic workflows, function calling, tool use, and multi-agent interaction protocols (e.g., LangChain, LangGraph, LlamaIndex, or custom frameworks).
• Protocols & Architectures: Solid understanding of Model Context Protocol (MCP), RESTful API design, Agent-to-Agent (A2A) orchestration, and Knowledge Graph/RAG patterns.
• Evaluation Methodologies: Demonstrated experience establishing automated LLM evaluation datasets, synthetic test benches, and scoring metrics for intent detection and tool selection.
• Cloud & Data Stack: Experience with AWS cloud infrastructure (ECS/EKS, Lambda, SQS/Kafka), vector databases, and enterprise data warehouses (Snowflake).
• Software Craftsmanship: Strong commitment to unit/integration testing, CI/CD automation, clear API documentation, and agile team collaboration.

Preferred Experience

Prior experience working with enterprise SaaS, tax/compliance technology, or complex multi-step workflow automation.

Hands-on experience with streaming architectures, semantic caching, and LLM latency optimization.

Familiarity with IAM/RBAC entitlement models and enterprise data security standards (SOC 2, ISO). Must Haves Deep experience in Python and high-performance asynchronous API frameworks (asyncio, FastAPI) Hands-on experience developing LLM agents, tool calling, and multi-agent orchestration (LangChain, LangGraph, or custom agent architectures) Experience implementing Model Context Protocol (MCP) or Agent-to-Agent (A2A) integration layers Experience with LLM evaluation pipelines, synthetic dataset creation, and latency/token optimization