Senior Data Scientist - LLM Agents & Infra (Cortex)

Palo Alto Networks
Santa Clara, United States of America
2 days ago

Role details

Contract type
Permanent contract
Employment type
Part-time (≤ 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior

Job location

Remote
Santa Clara, United States of America

Tech stack

API
Artificial Intelligence
ARM
Software Debugging
Open Source Technology
Software Engineering
Data Logging
Large Language Models
Multi-Agent Systems
GIT
Information Technology
HuggingFace

Job description

As a Senior Data Scientist, you will bridge the gap between high-level research and production-grade reliability. Your role is to design and build the "connective tissue" that allows autonomous agents to function in the wild. You will move beyond simple wrappers to architect systems that can handle the inherent uncertainty of AI, ensuring that our agentic products are not just "smart," but robust, observable, and capable of operating at enterprise scale., * Design and implement the backbone for multi-agent systems to manage complex, non-linear workflows and autonomous decision-making.

  • Build specialized infrastructure to handle dynamic AI environments where outputs are unpredictable, ensuring system stability through robust "guardrail" architectures and fallback logic.
  • Implement advanced tracing and logging for agent "thought processes," enabling the team to debug complex multi-step reasoning chains in real-time production environments.
  • Develop automated tools to identify, isolate, and replicate edge-case failures in agent behavior, transforming "black box" hallucinations into actionable engineering tickets.
  • Establish sophisticated monitoring for production deployments, tracking not just uptime, but model decay, tool-calling accuracy, and semantic drift across millions of tokens.

Requirements

  • At least 2 years of experience in high-growth software environments with a proven track record of building, shipping and monitoring complex products to thousands of users.
  • At least 2 (additional) years of experience developing and deploying state-of-the-art AI-powered products at scale.
  • Hands-on experience working with both proprietary APIs and SOTA open-source models
  • Agentic experience building and maintaining LLM-powered systems that go beyond chat, specifically focusing on autonomous tool-calling, RAG, and reasoning loops.
  • BSc in Computer Science, Software Engineering, Electrical Engineering or a related technical field., * Experience with specialized AI observability platforms (e.g., LangSmith) for debugging non-deterministic agent traces.
  • MSc in Computer Science, Software Engineering, Electrical Engineering or a related technical field.
  • Background in the cybersecurity domain.
  • Demonstrated contributions to the open-source community (e.g., LangChain, HuggingFace, or a significant personal Git portfolio).

Apply for this position