> Markdown version of [/jobs/ext/1298666-senior-data-scientist-llm-agents-infra-cortex](https://www.wearedevelopers.com/jobs/ext/1298666-senior-data-scientist-llm-agents-infra-cortex). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Senior Data Scientist - LLM Agents & Infra (Cortex) - **Company:** Palo Alto Networks - **Location:** Santa Clara, CA, United States (Remote available) - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Application Programming Interfaces (APIs), Artificial Intelligence, ARM Architecture, Software Debugging, Open Source Technology, Software Engineering, Data Logging, Large Language Models, Multi-Agent Systems, Git, Information Technology, HuggingFace - **Published:** July 16, 2026 - **Apply:** https://arc.dev/remote-jobs/j/redirect/p51oymdvxw ## About the Role * At least 2 years of experience in high-growth software environments with a proven track record of building, shipping and monitoring complex products to thousands of users. * At least 2 (additional) years of experience developing and deploying state-of-the-art AI-powered products at scale. * Hands-on experience working with both proprietary APIs and SOTA open-source models * Agentic experience building and maintaining LLM-powered systems that go beyond chat, specifically focusing on autonomous tool-calling, RAG, and reasoning loops. * BSc in Computer Science, Software Engineering, Electrical Engineering or a related technical field., * Experience with specialized AI observability platforms (e.g., LangSmith) for debugging non-deterministic agent traces. * MSc in Computer Science, Software Engineering, Electrical Engineering or a related technical field. * Background in the cybersecurity domain. * Demonstrated contributions to the open-source community (e.g., LangChain, HuggingFace, or a significant personal Git portfolio). ## Description As a Senior Data Scientist, you will bridge the gap between high-level research and production-grade reliability. Your role is to design and build the "connective tissue" that allows autonomous agents to function in the wild. You will move beyond simple wrappers to architect systems that can handle the inherent uncertainty of AI, ensuring that our agentic products are not just "smart," but robust, observable, and capable of operating at enterprise scale., * Design and implement the backbone for multi-agent systems to manage complex, non-linear workflows and autonomous decision-making. * Build specialized infrastructure to handle dynamic AI environments where outputs are unpredictable, ensuring system stability through robust "guardrail" architectures and fallback logic. * Implement advanced tracing and logging for agent "thought processes," enabling the team to debug complex multi-step reasoning chains in real-time production environments. * Develop automated tools to identify, isolate, and replicate edge-case failures in agent behavior, transforming "black box" hallucinations into actionable engineering tickets. * Establish sophisticated monitoring for production deployments, tracking not just uptime, but model decay, tool-calling accuracy, and semantic drift across millions of tokens. ## Related Videos - [Crypto-secure Data Management with In-Database Blockchain](https://www.wearedevelopers.com/videos/632-crypto-secure-data-management-with-in-database-blockchain) - [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94) - [One AI API to Power Them All](https://www.wearedevelopers.com/videos/1601-one-ai-api-to-power-them-all) - [Build Delightful Mobile Experiences with Kotlin, Realm, and Atlas Device Sync](https://www.wearedevelopers.com/videos/694-build-delightful-mobile-experiences-with-kotlin-realm-and-atlas-device-sync) - [Git for Code Reviews](https://www.wearedevelopers.com/videos/429-git-for-code-reviews) - [Beyond the Hype: Building Trustworthy and Reliable LLM Applications with Guardrails](https://www.wearedevelopers.com/videos/1594-beyond-the-hype-building-trustworthy-and-reliable-llm-applications-with-guardrails) ## Related Articles - [What Are Large Language Models?](https://www.wearedevelopers.com/magazine/304-what-are-large-language-models) - [Dev Digest 210: AI Agents Are Go! Is MCP Dead? LLMs Crack Anonymity](https://www.wearedevelopers.com/magazine/709-dev-digest-210-ai-agents-are-go-is-mcp-dead-llms-crack-anonymity) - [Everything a Developer Needs to Know About MCP with Neo4j](https://www.wearedevelopers.com/magazine/604-everything-a-developer-needs-to-know-about-mcp-with-neo4j) - [Dev Digest 134 - Where pixels sing?](https://www.wearedevelopers.com/magazine/477-dev-digest-134-where-pixels-sing) - [Graph and AI Trends 2026: Why Is AI Running but Not Yet Delivering?](https://www.wearedevelopers.com/magazine/680-graph-and-ai-trends-2026-why-is-ai-running-but-not-yet-delivering) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)