Senior Principal Data Scientist - LLM Agents (Cortex)

Palo Alto Networks
Santa Clara, CA, United States
11 days ago
Apply on arc.dev
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
3 years minimum
Working hours
Regular working hours
Job source

Tech stack

Clean Code Principles Artificial Intelligence Amazon Web Services ARM Architecture Big Data Continuous Integration Data Systems Software Design Patterns Memory Management Monitoring of Systems Python (Programming Language) Machine Learning
+17 more
Open Source Technology Performance Tuning Raw Data Google Cloud Pytorch Large Language Models Snowflake Multi-Agent Systems Concurrency Build Management Containerization Kubernetes Information Technology HuggingFace Cybercrime Data Pipelines Docker

Job description

  • Serve as a hands-on technical compass for a diverse, elite product-oriented research team, pioneering state-of-the-art technologies and setting the engineering standards for AI/ML excellence across the organization.
  • Design, implement, and optimize robust Agentic LLM solutions and multi-agent workflows engineered to autonomously counter modern, sophisticated cyber threats.
  • Oversee and actively contribute to the entire end-to-end lifecycle, moving seamlessly from ambiguous research concepts and POCs to high-throughput, low-latency production deployments.
  • Build and enforce comprehensive frameworks for continuous real-time monitoring, evaluation, and guardrails to track model drift, latency, and agent behavior in production environments.
  • Decompose highly complex, multi-layered algorithmic problems into actionable technical roadmaps and strategic insights for both deeply technical engineers and executive business stakeholders.

Requirements

  • A proven track record as an experienced architect who leads by example, capable of writing core, production-critical code, conducting rigorous architecture reviews, and mentoring senior technical staff.
  • At least 3 years of hands-on experience building, scaling, and optimizing Agentic LLM systems. This includes deep familiarity with common frameworks (e.g., LangChain, LangGraph, AutoGen) and a sophisticated understanding of tool-use optimization, memory management, and deterministic behavioral alignment.
  • At least 5 additional years of deep experience in data science, with expert-level command of classical machine learning algorithms applied to complex, real-world data landscapes.
  • At least 5 additional years of experience as a software engineer with high proficiency in Python and querying languages. You possess deep knowledge of software design patterns, concurrency, CI/CD pipelines, and writing clean, scalable, maintainable code.
  • BSc in Machine Learning, Computer Science, Electrical Engineering, Physics, Statistics, Applied Mathematics, or a related field from a top university. An advanced degree is highly preferred.
  • Demonstrated expertise in deploying and operating ML/LLM workloads at enterprise scale. You have a comprehensive understanding of containerization (Docker, Kubernetes), high-throughput data pipelines, and proactive system monitoring.
  • Exceptional ability to run end-to-end research-to-production initiatives, showcasing an analytical mindset capable of translating raw data into highly accurate, real-time defensive actions.
  • Extensive, hands-on experience utilizing, fine-tuning, and optimizing open-source generative AI frameworks and libraries (e.g., Hugging Face, PyTorch, vLLM, DeepSpeed) to build and deploy high-performance models.
  • Proven experience architecting data solutions and working within major cloud and big data ecosystems (e.g., GCP, AWS, Snowflake) to ingest, process, and analyze high-velocity, petabyte-scale datasets.
  • Outstanding communication and presentation skills, with a demonstrated ability to align cross-functional engineering, product, and research teams toward unified technical goals.

Preferred Qualifications

  • MSc/PhD in Machine Learning, Computer Science, Electrical Engineering, Physics, Statistics, Applied Mathematics, or a related field from a top university.
  • Background in the cybersecurity domain

About the company

At Palo Alto Networks®, we’re united by a shared mission-to protect our digital way of life. We thrive at the intersection of innovation and impact, solving real-world problems with cutting-edge technology and bold thinking. Here, everyone has a voice, and every idea counts. If you’re ready to do the most meaningful work of your career alongside people who are just as passionate as you are, you’re in the right place.

Who We Are

In order to be the cybersecurity partner of choice, we must trailblaze the path and shape the future of our industry. This is something our employees work at each day and is defined by our values: Disruption, Collaboration, Execution, Integrity, and Inclusion. We weave AI into the fabric of everything we do and use it to augment the impact every individual can have. If you are passionate about solving real-world problems and ideating beside the best and the brightest, we invite you to join us!

We believe collaboration thrives in person. That’s why most of our teams work from the office full time, with flexibility when it’s needed. This model supports real-time problem-solving, stronger relationships, and the kind of precision that drives great outcomes., Are you passionate about harnessing cutting-edge data science techniques to protect the world from cyber threats? Do you thrive in the dynamic world of cybersecurity? If so, we invite you to join our innovative and forward-thinking team at Palo Alto Networks, where you’ll have the opportunity to shape the future of cybersecurity and impact millions of customers.

As a Senior Principal Data Scientist, you will be a technical leader and hands-on architect, bridging the worlds of advanced data science, robust software engineering, and next-generation AI. You will bring a deep, diverse background in classical machine learning algorithms alongside cutting-edge LLM agent architectures to build resilient, production-grade defense techniques against cyber-villains. Operating as a premier technical authority, you will drive the vision, design, and large-scale development of our groundbreaking, Agentix LLM-based security solutions that block attacks even before they begin.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on arc.dev
Prepare application

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

3:14 min

Structuring career paths and localized data architectures

Ulrich Wurstbauer +1 · LIVE

2:07 min

Inspecting default bridge architectures and custom Docker networks

Oliver Seitz Oliver Seitz · World Congress 2025

2:35 min

Preventing remote code execution in PyTorch models

Balázs Kiss · World Congress 2023

4:09 min

Challenges of interpreting raw data with language models

Clemens Vasters Clemens Vasters · World Congress 2025

2:47 min

Leveraging older LLMs defensively for vulnerability hunting

Adrian Mouat Adrian Mouat · World Congress 2026 Europe

2:34 min

Docker sandbox architecture and microVM environment integration

Manuel de la Peña Manuel de la Peña · World Congress 2026 Europe

Videos

See all

Related articles

See all