> Markdown version of [/jobs/ext/4622251-software-engineer-self-improving-systems-palo-alto](https://www.wearedevelopers.com/jobs/ext/4622251-software-engineer-self-improving-systems-palo-alto). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Software Engineer, Self-Improving Systems Palo Alto

- **Company:** The Sycamore
- **Location:** Palo Alto, CA, United States
- **Experience:** Expert
- **Contract:** Permanent contract
- **Skills:** Application Programming Interfaces (APIs), Artificial Intelligence, Databases, Software Debugging, Python (Programming Language), Recommender Systems, Cloud Services, Software Engineering, TypeScript, Privacy Controls, Cloud Platform System, ReactJS, Backend, Low Latency, Api Design, Stream Processing, Software Version Control
- **Published:** November 7, 2026
- **Apply:** https://www.juju.com/job/28_25197_c54428b64ebf71e0f2c2b4c52c277163

## About the Role

* 5-12 years of software engineering experience. We will make exceptions for exceptional people in either direction.
 * Strong backend and distributed-systems fundamentals, including typed API design, asynchronous workflows, persistence, reliability, and production debugging.
 * Real depth in at least one of the areas above, and genuine interest in working across the others.
 * An empirical approach to AI quality: you can form a hypothesis, design a useful evaluation, interpret noisy evidence, and distinguish a real improvement from movement in a proxy metric.
 * The ability to reason about retries, idempotency, partial failure, long-running state, concurrency, latency, cost, experiment design, and safe rollout.
 * A security-minded approach to multi-tenant systems, identity, authorization, credentials, tool execution, privacy, and auditability.
 * Product judgment. You can find a durable abstraction behind a real requirement without generalizing too early or freezing customer-specific behavior into the platform.
 * AI-native. You use coding agents and modern models as a force multiplier while still owning architecture, correctness, evidence, and operational outcomes.
 * Comfort building product-facing software. You can work in React and TypeScript when a capability needs a great interface, and reason about streaming state and accessibility.
 * Comfort with startup ambiguity, fast feedback loops, and broad ownership.

## Description

* Develop the learning data plane around agents: structured trajectories, feedback and outcome signals, offline datasets, lineage, privacy controls, and reliable links between an agent version and its behavior.
 * Create evaluation systems for task completion, tool use, long-horizon behavior, safety, latency, and cost, and own the credibility of the numbers they produce.
 * Design experiment and versioning systems for comparing changes through replay, shadow traffic, canaries, or controlled rollouts, with clear promotion and rollback criteria.
 * Build durable orchestration for long-running tasks, checkpoints, approvals, handoffs, and human-in-the-loop interactions, with typed tool interfaces, protocol-based execution, and memory retrieval that enforces tenant, user, and project visibility boundaries.
 * Own the execution environments agents run in, including isolation, startup performance, resource limits, and cost, and the policy layer that gates self-modification so a proposed change is staged and tested rather than applied silently.
 * Publish reliable APIs, event-driven interfaces, and reusable libraries that work across agent categories and enterprise deployments.

The environment you will work in

Our current Core AI environment includes Python cloud services; React and TypeScript product surfaces in Sycamore Forge; asynchronous and streaming systems; typed APIs and data models; relational and vector data; durable workflows; protocol-based tool execution; multiple model providers; and cloud-native deployment.

This is context, not a checklist. We do not require previous experience with every language, framework, model provider, cloud platform, database, or infrastructure tool in our stack. Comparable experience building distributed runtimes, experimentation platforms, retrieval or recommendation systems, workflow engines, developer platforms, or production AI systems is highly relevant.