> Markdown version of [/jobs/ext/2716852-full-stack-software-engineer-product](https://www.wearedevelopers.com/jobs/ext/2716852-full-stack-software-engineer-product). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Full-stack Software Engineer (Product) - **Company:** Apollo - **Location:** San Francisco, CA, United States (Remote available) - **Experience:** Experienced - **Salary:** $222,000.0 - $290,000.0 - **Contract:** Permanent contract - **Skills:** Application Programming Interfaces (APIs), Artificial Intelligence, Big Data, Cloud Computing, Cyber Security, Databases, Customer Data Management, Cursor (Graphical User Interface Elements), Software Debugging, Programming Tools, Fault Tolerance, Python (Programming Language), Pattern Recognition, Queueing Systems, Webui, Software Safety, Security Information and Event Management, Software Deployment, TypeScript, Data Logging, Application Enhancement Tool, Data Ingestion, Large Language Models, Backend, Git, Build Management, Enterprise Integration, Front End Software Development, Data Pipelines - **Published:** September 4, 2026 - **Apply:** https://startup.jobs/full-stack-software-engineer-product-apollo-research-9801739 ## About the Role * 4+ years building production software. You have shipped and operated real systems with real users, and you know what it takes to keep them reliable. * Full-stack capability. You are strong on the backend (APIs, data pipelines, databases, cloud infrastructure) and productive on the frontend (modern web UI). You don't need to be world-class at both, but you must be able to own a feature across the whole stack. Our stack is primarily Python and TypeScript. * High agency and ownership. You are willing to own big chunks of the product, make decisions fast with incomplete information, and be accountable for the outcome. You don't wait to be told what to do and have an accurate sense of the roadmap for the product. * High output volume. You have used Claude Code, Codex, Cursor, or similar tools heavily to accelerate or have built agents or agent tooling yourself. * Startup pace. You are excited about a fast-moving environment, comfortable with ambiguity and changing priorities, and willing to grind when it matters. * Product sense. You can talk to users, translate vague requirements into concrete designs, and make good calls about what to build and what to cut. Strong nice-to-haves * Previous work on developer tools, monitoring/observability systems, or security products. Watcher sits at the intersection of all three. * Real-time or large-scale data processing experience (streaming pipelines, message queues, high-throughput log systems). * Enterprise or on-prem deployment experience. Shipping software into environments you don't control is its own skill. * Experience building LLM-powered applications, e.g. LLM-as-judge setups, evaluation pipelines, or familiarity with frameworks like Inspect. * Early-stage startup experience. You have been employee 1-15 somewhere and know what it feels like to build a product with no playbook. Explicitly not required * Formal AI safety background. We need excellent product engineers who can learn the AI safety context, not AI safety researchers who need to learn engineering. * Management experience. This is an IC role, at least initially. * Deep experience in every technology we use. We care about demonstrated ability to learn and ship, not checkbox familiarity with our exact stack. ## Description We are building Watcher, a coding agent security product. Watcher is deployed in production and monitors billions of agent tokens per month across engineering teams at agent-building scale-ups and enterprises. We are looking for a product engineer to build and scale Watcher end to end. This role includes backend and frontend components. You will own features across the full stack, from agent hooks and ingestion pipelines to monitoring dashboards and enterprise integrations. You can expect to own features from customer conversation to production deployment. This is truly a "start-up role": you will own big chunks of the product, make decisions fast, and ship at high volume. You will join a small team with significant ability to shape the product and tech, and you can earn more responsibility quickly. This is an individual contributor role but could lead to management responsibilities eventually, if desired., * Design, build, and ship Watcher features across the stack: agent-side hooks and CLI tooling, ingestion and grading pipelines, real-time monitoring UI (Watcher Live), and the organization-wide Analyzer dashboard. * Own features end to end, from ambiguous requirements to production: talk to users, scope the work, make the technical decisions, put that into production, and iterate based on usage. * Build Watcher integrations: We currently support Claude Code and Codex, but we're planning on increasing our coverage to other coding agents, LLM gateways, and anywhere that could benefit from agent monitoring. * Build the policy and configuration layer that lets security teams set org-wide guardrails for engineers to operate safely within., * Turn monitoring research (new monitors, grading strategies, backtesting results) into production-ready product features. We are "our own customer": You will use the product you build every day. Infrastructure and scale (~35%) * Design and operate backend systems that process large volumes of agent logs in real time. * Own reliability: robust error handling, graceful degradation, and observability (logging, metrics, tracing, alerting). We should catch issues before customers do. * Architect data models and storage that handle both high-throughput writes and complex analytical queries over historical trajectories. Retention policies have to fit highly sensitive customer data. * Make sure Watcher is safe to deploy: Work with our AI Security & Control Engineer on tenant isolation, encryption, and access controls. Watcher touches some of the most sensitive data our customers have. Customer and integration engineering (~20%) * Build and maintain integrations with the enterprise security stack, e.g. streaming monitoring alerts into SIEM systems and existing security operations workflows. * Support flexible deployment models: cloud-hosted, on-prem, and local backends, so sensitive agent logs never have to leave customer control. * Talk to customers, debug issues in their environments, and feed what you learn back into the roadmap. REPRESENTATIVE PROJECTS * Secure a new frontier coding agent: Take an agent we don't yet support (e.g. Cursor) from zero to fully monitored. Build the hook/integration layer, normalize its trajectory format into our data model, validate monitor accuracy on real traffic, and ship it to customers. * Build the real-time blocking path: Design the low-latency pipeline that lets monitors block dangerous actions (e.g. git push --force, secret exfiltration) before they execute, while keeping p95 latency low enough that developers don't feel it. Handle traffic spikes and partial failures gracefully. * Ship an Analyzer capability end to end: Design and build an organization-wide view (e.g. failure trends over time, cross-session pattern detection) from data model through API to UI, based on what security teams actually need. * Deploy Watcher inside locked-down enterprises: Build and document the self-hosted/on-prem deployment story so that enterprises with strict data requirements can run Watcher inside their own infrastructure., * Location: This is an in-person role working out of our London or San Francisco office. We offer flexible working hours and some wfh arrangements. * Visa sponsorship: We sponsor visas in both the UK and US. Sponsorship isn't guaranteed for every role or candidate, but if we make you an offer, we'll work with you to find the right visa route., About the interview process: Our multi-stage process includes a screening interview, a take-home test (3 hours), 3 technical interviews, and a final interview with Marius (CEO). There are no leetcode-style general coding interviews. You may use AI tools on the take-home; we judge the result the way we'd judge any contributor's work, so you are responsible for the quality of everything you submit. If you want to prepare, we suggest building simple monitors for coding agents and running them on your own Claude Code / Cursor / Codex / etc. traffic. Your Privacy and Fairness in Our Recruitment Process: We are committed to protecting your data, ensuring fairness, and adhering to workplace fairness principles in our recruitment process. To enhance hiring efficiency, we use AI-powered tools to assist with tasks such as resume screening. These tools are designed and deployed in compliance with internationally recognized AI governance frameworks. Your personal data is handled securely and transparently. All resumes are screened by a human and final hiring decisions are made by our team. If you have questions about how your data is processed or wish to report concerns about fairness, please contact us at info@apolloresearch.ai. ## Related Videos - [Developing the Backend with Stefan Lingler, CTO at Shpock](https://www.wearedevelopers.com/videos/100360-developing-the-backend-with-stefan-lingler-cto-at-shpock) - [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94) - [Agentic employees in world's most downloaded FinTech app](https://www.wearedevelopers.com/videos/100123-agentic-employees-in-world-s-most-downloaded-fintech-app) - [Nest.js - TypeScript in the backend can also be clean](https://www.wearedevelopers.com/videos/1033-nest-js-typescript-in-the-backend-can-also-be-clean) - [Git for Code Reviews](https://www.wearedevelopers.com/videos/429-git-for-code-reviews) - [Craftsmanship in the Age of Agents](https://www.wearedevelopers.com/videos/100106-craftsmanship-in-the-age-of-agents) ## Related Articles - [Navigating the AI Shift](https://www.wearedevelopers.com/magazine/629-navigating-the-ai-shift) - [Never delegate the understanding](https://www.wearedevelopers.com/magazine/749-never-delegate-the-understanding) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [What is Software Engineering in the Age of AI?](https://www.wearedevelopers.com/magazine/640-what-is-software-engineering-in-the-age-of-ai) - [What is Agentic Programming and Why Should Developers Care?](https://www.wearedevelopers.com/magazine/625-what-is-agentic-programming-and-why-should-developers-care) - [Dev Digest 137 - AI'm not sure about this](https://www.wearedevelopers.com/magazine/485-dev-digest-137-ai-m-not-sure-about-this)