> Markdown version of [/jobs/ext/1896205-software-engineer-ai-platform](https://www.wearedevelopers.com/jobs/ext/1896205-software-engineer-ai-platform). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Software Engineer, AI Platform - **Company:** AALYRIA TECHNOLOGIES, INC. - **Location:** San Francisco, CA, United States - **Salary:** $185,000.0 - $215,000.0 - **Contract:** Permanent contract - **Skills:** Application Programming Interfaces (APIs), Artificial Intelligence, Audit Trail, Cloud Computing, Code Review, Computer Networks, Continuous Integration, Programming Tools, Data Flow Control, Hardware-In-The-Loop Simulation, Identity and Access Management, Python (Programming Language), Open Source Technology, Webui, Software Systems, Systems Integration, TypeScript, Data Logging, Google Cloud, Large Language Models, Grafana, Gitlab, Git, Build Management, Kubernetes, Build Tools, Front End Software Development, Api Design, Terraform, Webhooks - **Published:** August 1, 2026 - **Apply:** https://www.dice.com/job-detail/dcec890d-b889-4e7b-96d1-6676d6908944 ## About the Role * Strong software engineering fundamentals: Several years building, shipping, and operating production software. Fluency in at least one of Python, Go, or TypeScript, plus Terraform. We are an infrastructure-as-code company, and infrastructure here is written, reviewed, and shipped like software. Comfort with API design, testing, code review, and owning services in production. * Hands-on experience building LLM-powered systems: retrieval pipelines, tool use / MCP, agent orchestration, prompt and context management, and evaluating whether any of it actually works. * Product sense for internal tooling: experience deploying, extending, and integrating open-source applications (chat frontends, gateways, dev tools) rather than building everything from scratch, with a strong instinct for build vs. configure vs. wait. * Developer-platform integration experience: working with Git-hosting APIs, webhooks, and CI/CD to embed AI into the workflows engineers already live in. * Cloud and Kubernetes engineering: you build and operate the infrastructure your systems run on yourself, such as containers, Helm, Terraform, GKE, and/or IAM. The platform team reviews your changes the way any senior engineer reviews a colleague's work; they don't implement your designs for you. * Sandboxing and isolation literacy: understanding of how to safely run model-generated code and constrain agent access (containers, network policy, least-privilege credentials). * Effective in a regulated environment: not compliance expertise, but the ability to elicit constraints from security stakeholders, ask the right questions, design within hard boundaries, and build systems whose behavior is observable and auditable by construction. * Clear written communication: Specifically for technical and executive audiences, including candid assessments of what is not working. * High autonomy: you will define your own roadmap from a clear mandate and validate it directly with the teams you serve., * Prior work in CMMC, DoD IL, FedRAMP, or NIST 800-171 environments - especially running AI/LLM workloads inside such a boundary. * Self-hosted inference experience: vLLM/TGI/similar, GPU provisioning and utilization, open-weight model deployment. * Experience with LLM gateway/proxy layers (e.g., LiteLLM or equivalent) and per-team cost attribution. * Hardware-in-the-loop or lab-automation experience: test benches, device access control, or safely bridging software systems to physical equipment. * Familiarity with DLP concepts as they apply at the model/API layer. * Google Cloud Platform specifically (Vertex AI, GKE, IAP, Artifact Registry); Bazel or other hermetic build systems. * Observability stack experience (OpenTelemetry, Grafana/Loki/Mimir/Tempo or similar)., * U.S. citizen or national * U.S. lawful permanent resident () * Refugee under 8 U.S.C. 1157 * Asylee under 8 U.S.C. 1158 (B) Be eligible to access export-controlled information without requiring an export authorization. (C) Be eligible and reasonably likely to obtain the necessary export authorization from the appropriate U.S. government agency. ## Description Aalyria is hiring a software engineer to build the AI layer of our engineering organization. This is fundamentally a building role where you will design, ship, and operate internal AI products and platform services, such as chat and agent interfaces, retrieval over our code and docs, AI-integrated developer workflows, sandboxed execution environments, on top of our existing cloud infrastructure (Google Cloud Platform, GKE, Vertex AI, GitLab). We operate in a regulated environment (CMMC, DoD Impact Levels, FedRAMP), and that shapes the work. Your job is to work with our security and compliance director, ask the right questions, extract the real constraints, and then engineer systems that deliver the maximum utility to our engineers while staying inside the boundary. The integrations you build need care and feeding: they must keep working, keep improving, and remain auditable. You will own them end to end. Key Responsibilities: Illustrative, not exhaustive. You will help define this roadmap: * Internal AI chat and agent platform: deploy and extend an open-source frontend (e.g., LibreChat, Open WebUI) backed by our existing Vertex AI models and any future inference infrastructure, with SSO, access policy, and the integrations that make it actually useful, RAG and MCP servers tied to our GitLab repos and internal docs, code execution (Code Interpreter-style), and internal tool access. * AI-native SWE workflows in GitLab: agents that generate merge requests, respond to review comments, and iterate, so engineers can drive AI work entirely through the review interface they already use. * Sandboxes for AI development, with humans and hardware in the loop: isolated, policy-enforced environments where agents (and the engineers supervising them) can safely run code, access repos, and use tools - extending to infrastructure that lets AI safely interact with real hardware in our lab environments. * AI observability and cost tracking: request-level telemetry, usage analytics, and cost attribution across all model usage, so we know how AI is being used internally and what it's worth. Additionally, you will be responsible for: * Identifying, building, and maintaining LLM-powered systems across engineering and operations workflows, prioritizing leverage over coverage. * The design and build of agentic systems and internal AI applications: multi-step pipelines, tool-using agents, retrieval-augmented systems, and internal-facing apps that put model capabilities in front of the right people. * Owning the AI gateway layer: model routing, credential management, access policy, and request-level telemetry across all model usage. * Operating what you ship: monitoring, upgrades, incident response, and continuous improvement of the AI stack. * Partnering with the security/compliance director to understand boundary requirements (CUI handling, data residency, provider selection, audit logging) and translate them into system design, building guardrails in at design time, not as afterthoughts. You bring the engineering; they bring the framework. * Contributing the technical substance (architecture, data-flow diagrams, logging evidence) that supports compliance documentation owned by the security team. * Serving as the organization's internal expert on applied AI tooling: stay current on the ecosystem, evaluate new capabilities, and translate them into concrete proposals. * Reporting periodically to leadership on friction points and opportunities - informing decisions rather than driving adoption targets. ## Related Videos - [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94) - [WeAreDevelopers LIVE - Modern DevOps for IoT Devices and More](https://www.wearedevelopers.com/videos/1805-wearedevelopers-live-modern-devops-for-iot-devices-and-more) - [Coffee with Developers - Maria Apazoglou](https://www.wearedevelopers.com/videos/1209-coffee-with-developers-maria-apazoglou) - [Git for Code Reviews](https://www.wearedevelopers.com/videos/429-git-for-code-reviews) - [One AI API to Power Them All](https://www.wearedevelopers.com/videos/1601-one-ai-api-to-power-them-all) - [Enabling automated 1-click customer deployments with built-in quality and security](https://www.wearedevelopers.com/videos/83-enabling-automated-1-click-customer-deployments-with-built-in-quality-and-security) ## Related Articles - [What Are Large Language Models?](https://www.wearedevelopers.com/magazine/304-what-are-large-language-models) - [MLOps And AI Driven Development](https://www.wearedevelopers.com/magazine/82-mlops-and-ai-driven-development) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [Got AI ideas but no money? Here are 10 free ways to level up your AI skills with Google Cloud](https://www.wearedevelopers.com/magazine/600-got-ai-ideas-but-no-money-here-are-10-free-ways-to-level-up-your-ai-skills-with-google-cloud) - [Stephan Gillich - Bringing AI Everywhere](https://www.wearedevelopers.com/magazine/489-stephan-gillich-bringing-ai-everywhere) - [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production)