> Markdown version of [/jobs/ext/2256454-senior-principal-data-scientist-llm-agents-cortex](https://www.wearedevelopers.com/jobs/ext/2256454-senior-principal-data-scientist-llm-agents-cortex). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Senior Principal Data Scientist - LLM Agents (Cortex) - **Company:** Palo Alto Networks - **Location:** Santa Clara, CA, United States (Remote available) - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Clean Code Principles, Artificial Intelligence, Amazon Web Services, ARM Architecture, Big Data, Continuous Integration, Data Systems, Software Design Patterns, Memory Management, Monitoring of Systems, Python (Programming Language), Machine Learning, Open Source Technology, Performance Tuning, Raw Data, Google Cloud, Pytorch, Large Language Models, Snowflake, Multi-Agent Systems, Concurrency, Build Management, Containerization, Kubernetes, Information Technology, HuggingFace, Cybercrime, Data Pipelines, Docker - **Published:** August 26, 2026 - **Apply:** https://arc.dev/remote-jobs/j/redirect/pfdz9c6ukw ## About the Role * A proven track record as an experienced architect who leads by example, capable of writing core, production-critical code, conducting rigorous architecture reviews, and mentoring senior technical staff. * At least 3 years of hands-on experience building, scaling, and optimizing Agentic LLM systems. This includes deep familiarity with common frameworks (e.g., LangChain, LangGraph, AutoGen) and a sophisticated understanding of tool-use optimization, memory management, and deterministic behavioral alignment. * At least 5 additional years of deep experience in data science, with expert-level command of classical machine learning algorithms applied to complex, real-world data landscapes. * At least 5 additional years of experience as a software engineer with high proficiency in Python and querying languages. You possess deep knowledge of software design patterns, concurrency, CI/CD pipelines, and writing clean, scalable, maintainable code. * BSc in Machine Learning, Computer Science, Electrical Engineering, Physics, Statistics, Applied Mathematics, or a related field from a top university. An advanced degree is highly preferred. * Demonstrated expertise in deploying and operating ML/LLM workloads at enterprise scale. You have a comprehensive understanding of containerization (Docker, Kubernetes), high-throughput data pipelines, and proactive system monitoring. * Exceptional ability to run end-to-end research-to-production initiatives, showcasing an analytical mindset capable of translating raw data into highly accurate, real-time defensive actions. * Extensive, hands-on experience utilizing, fine-tuning, and optimizing open-source generative AI frameworks and libraries (e.g., Hugging Face, PyTorch, vLLM, DeepSpeed) to build and deploy high-performance models. * Proven experience architecting data solutions and working within major cloud and big data ecosystems (e.g., GCP, AWS, Snowflake) to ingest, process, and analyze high-velocity, petabyte-scale datasets. * Outstanding communication and presentation skills, with a demonstrated ability to align cross-functional engineering, product, and research teams toward unified technical goals. Preferred Qualifications * MSc/PhD in Machine Learning, Computer Science, Electrical Engineering, Physics, Statistics, Applied Mathematics, or a related field from a top university. * Background in the cybersecurity domain ## Description * Serve as a hands-on technical compass for a diverse, elite product-oriented research team, pioneering state-of-the-art technologies and setting the engineering standards for AI/ML excellence across the organization. * Design, implement, and optimize robust Agentic LLM solutions and multi-agent workflows engineered to autonomously counter modern, sophisticated cyber threats. * Oversee and actively contribute to the entire end-to-end lifecycle, moving seamlessly from ambiguous research concepts and POCs to high-throughput, low-latency production deployments. * Build and enforce comprehensive frameworks for continuous real-time monitoring, evaluation, and guardrails to track model drift, latency, and agent behavior in production environments. * Decompose highly complex, multi-layered algorithmic problems into actionable technical roadmaps and strategic insights for both deeply technical engineers and executive business stakeholders. ## Related Videos - [Docker Compose: Rediscovered](https://www.wearedevelopers.com/videos/1978-docker-compose-rediscovered) - [Photonic Computing: Programming a New Class of AI Accelerators (incl. Live Coding)](https://www.wearedevelopers.com/videos/100196-photonic-computing-programming-a-new-class-of-ai-accelerators-incl-live-coding) - [Data Governance in the Era of AI](https://www.wearedevelopers.com/videos/1622-data-governance-in-the-era-of-ai) - [Prompt Injection, Poisoning & More: The Dark Side of LLMs](https://www.wearedevelopers.com/videos/1563-prompt-injection-poisoning-more-the-dark-side-of-llms) - [Docker build without Docker](https://www.wearedevelopers.com/videos/100114-docker-build-without-docker) - [Beyond the Hype: Building Trustworthy and Reliable LLM Applications with Guardrails](https://www.wearedevelopers.com/videos/1594-beyond-the-hype-building-trustworthy-and-reliable-llm-applications-with-guardrails) ## Related Articles - [What Are Large Language Models?](https://www.wearedevelopers.com/magazine/304-what-are-large-language-models) - [Graph and AI Trends 2026: Why Is AI Running but Not Yet Delivering?](https://www.wearedevelopers.com/magazine/680-graph-and-ai-trends-2026-why-is-ai-running-but-not-yet-delivering) - [Everything a Developer Needs to Know About MCP with Neo4j](https://www.wearedevelopers.com/magazine/604-everything-a-developer-needs-to-know-about-mcp-with-neo4j) - [From Prototype to Production: Build AI Agents with This Free 4-Course Learning Path](https://www.wearedevelopers.com/magazine/655-from-prototype-to-production-build-ai-agents-with-this-free-4-course-learning-path) - [Dev Digest 210: AI Agents Are Go! Is MCP Dead? LLMs Crack Anonymity](https://www.wearedevelopers.com/magazine/709-dev-digest-210-ai-agents-are-go-is-mcp-dead-llms-crack-anonymity) - [MLOps And AI Driven Development](https://www.wearedevelopers.com/magazine/82-mlops-and-ai-driven-development)