Senior AI/ML Software Engineer

Peraton Inc
Herndon, United States of America
5 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior
Compensation
$ 234K

Job location

Herndon, United States of America

Tech stack

API
Artificial Intelligence
Airflow
Amazon Web Services (AWS)
Amazon Web Services (AWS)
Software Debugging
Distributed Data Store
Federal Information Processing Standards (FIPS)
Python
PostgreSQL
Prometheus
Datadog
React
Large Language Models
Grafana
Prompt Engineering
Backend
FastAPI
Kubernetes
low-code
Machine Learning Operations
Celery
Virtual Agents
REST
Dynatrace
Docker

Job description

Peraton Labs is seeking a Senior AI/ML Software Engineer to join the Labs Agentic AI team, you'll design, build, and ship AI-powered systems a compliance-ready, low-code platform for dynamically generating and orchestrating AI agentic workflows. You'll work across the full product lifecycle: from architecting multi-step agentic pipelines backed by Temporal.io, to building the plugin system, APIs, and interfaces that bring them to life, from within federal-grade security and accreditation constraints.

This is a role for someone who thinks deeply about how AI agents should behave in high-trust environments, cares about reliability and auditability, and can move fluidly between distributed orchestration, backend systems, and product-facing features.

Your responsibilities may include: Design and implement agentic AI capabilities using Python-based frameworks (LangChain, LangGraph, DeepAgents) and orchestrated workflows

  • Build and maintain integrations with LLM APIs (Anthropic/Claude, OpenAI, AWS Bedrock, Ollama) to power intelligent, multi-step automations
  • Develop full-stack product features (FastAPI + React) that surface AI capabilities to users - from REST APIs and streaming interfaces to workflow builders and dashboards
  • Instrument agent pipelines with OpenTelemetry tracing, provenance audit trails, and observability tooling for debugging and performance evaluation
  • Write clear, well-tested, maintainable code that passes strict pre-commit validation, and contribute to engineering standards in a compliance-driven environment
  • Evaluate agent performance, debug distributed workflows, and continuously improve reliability and output quality

Requirements

  • Minimum of a BS degree with 12 years of experience, MS degree with 10 years, or PhD with 7 years with meaningful exposure to AI/ML systems or LLM-based products
  • Hands-on experience building agentic systems using multi-step reasoning, tool use, RAG pipelines, or autonomous task execution
  • Strong Python skills (3.12+); comfort with async/await patterns, type hints, and modern Python tooling
  • Experience with workflow or task orchestration systems (Airflow, Prefect, Celery, or similar distributed execution frameworks)
  • Familiarity with agentic frameworks and an understanding of the underlying concepts (chains, tool calling, agent loops) that transfer across tools
  • Experience working with LLM APIs (OpenAI, Anthropic, AWS Bedrock, or similar)
  • Comfort working across the stack: FastAPI/Python backends, React frontends, Docker containerization, and PostgreSQL
  • A product mindset: you think about the end user, not just the technical implementation
  • Comfort operating with some ambiguity in a fast-moving environment
  • US Citizenship is a requirement for this position

Desired Additional Experience:

  • Experience with workflow orchestration frameworks for workflow/activity patterns, task queues, worker lifecycle management
  • Familiarity with federal compliance environments: FedRAMP, FIPS 140-2/3, IronBank container hardening, OPA policy enforcement, or Section 508 accessibility
  • Experience building plugin or extension systems: dynamic code loading, container isolation, API mixin patterns
  • Exposure to orchestration patterns: supervisor agents, parallel tool calls, human-in-the-loop flows, DAG-based pipeline execution
  • Experience with observability tooling: OpenTelemetry, Jaeger, Prometheus, Grafana, or similar distributed tracing/metrics stacks
  • Familiarity with prompt engineering, evaluation frameworks, or agent observability
  • Experience with container orchestration (Docker SDK, Kubernetes) and distributed storage (S3, MinIO, JuiceFS)
  • Prior work building internal tooling, enterprise automation products, or platforms for government customers

About the company

Peraton is a next-generation national security company that drives missions of consequence spanning the globe and extending to the farthest reaches of the galaxy. As the world's leading mission capability integrator and transformative enterprise IT provider, we deliver trusted, highly differentiated solutions and technologies to protect our nation and allies. Peraton operates at the critical nexus between traditional and nontraditional threats across all domains: land, sea, space, air, and cyberspace. The company serves as a valued partner to essential government agencies and supports every branch of the U.S. armed forces. Each day, our employees do the can't be done by solving the most daunting challenges facing our customers. Visit peraton.com to learn how we're keeping people around the world safe and secure. Target Salary Range

Apply for this position