AI Platform Engineer

Cornerstone Barricades
Cary, NC, United States
1 day ago
Apply on zipstaff.betterteam.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
8 years minimum
Compensation
$128,960 - $131,040
Working hours
Regular working hours

Tech stack

LangGraph Framework Java (Programming Language) Artificial Intelligence Continuous Integration Elasticsearch Python (Programming Language) Reliability Engineering OpenAI Search Technologies Pinecone Model Garden (Google Cloud) QLoRA
+24 more
LangChain Retrieval-Augmented Generation Large Language Models Qwen NeMo Guardrails Llamaindex Langfuse β€” LLM Observability and Analytics Platform Rate Limiting Pgvector AI Platforms LangSmith Kubernetes CrewAI RAGAS (Retrieval Augmented Generation Assessment) Weaviate AutoGen Arize Phoenix Ollama LiteLLM Machine Learning Operations DeepEval Model Context Protocol Terraform Promptfoo

Job description

ZipStaff is hiring an AI Platform Engineer for our client, a Fortune 10 healthcare and pharmacy leader. You’ll build and operate a production agentic AI platform - the layer other engineers build on - and own its reliability, security, and cost. This is a fully remote contract-to-hire role. Candidates must be US citizens or green card holders; sponsorship is not available. Responsibilities:

  • Build and operate a production agentic AI platform: orchestration, model gateway, agent registry, retrieval, guardrails, evaluation, and observability.
  • Implement production multi-agent orchestration, including state management and retry/fallback design.
  • Operate model serving and gateway capabilities, including routing, fallback, rate limiting, token/usage capture, and serving of open-weight models.
  • Build retrieval-augmented generation, including vector stores, embedding models, chunking, hybrid and semantic search, reranking, and grounding-quality evaluation.
  • Implement agent infrastructure, including Model Context Protocol (MCP), tool/function-calling contracts, agent registries, permission enforcement, and separation of reasoning from acting agents.
  • Design evaluation and LLMOps practices, including eval harnesses, offline and online evaluation, regression suites, drift and hallucination detection, and experiment tracking.
  • Apply guardrails and AI security, including prompt-injection, jailbreak, and data-leakage defense, plus PII detection and redaction in prompts and responses.
  • Instrument observability for LLM and agent calls (token, latency, and cost telemetry) and quality-signal monitoring that feeds alerting.

Requirements

  • US citizen or green card holder; sponsorship is not available.
  • 8+ years in software or platform engineering, with hands-on production ownership of AI or ML infrastructure.
  • Has built and operated agentic or LLM systems in production, not only prototyped them.
  • LLM and agent orchestration with LangGraph, LangChain, LlamaIndex, CrewAI, AutoGen, or the Google Agent Development Kit (ADK).
  • Model serving and gateway experience (LiteLLM, Portkey, Kong AI Gateway, TrueFoundry, or Vertex AI Model Garden), including routing, fallback, rate limiting, token/usage capture, serving open-weight models (Llama, Mistral, Gemma, Qwen) via vLLM, TGI, Ollama, or Vertex endpoints, and LoRA/QLoRA adapters.
  • RAG experience with Vertex Vector Search, Pinecone, Weaviate, pgvector, or Elasticsearch.
  • Evaluation and LLMOps with RAGAS, DeepEval, Promptfoo, or OpenAI Evals, and experiment tracking with MLflow, LangSmith, Langfuse, Arize Phoenix, or Braintrust.
  • Guardrails and AI security experience with NeMo Guardrails, Guardrails AI, Model Armor, or Lakera.
  • Observability (OpenTelemetry tracing for LLM and agent calls); Kubernetes and containers; GCP preferred (GKE, Vertex AI, Cloud Run); Terraform, CI/CD, and SRE practice for production AI services.
  • Strong Python.

Preferred Qualifications:

  • Go or Java for platform services.

Benefits & conditions

  • Health Insurance.
  • Dental Insurance.
  • Vision Insurance.
  • 401(k) + matching.
  • Paid time off.
  • Referral Program.

About ZipStaff: ZipStaff connects skilled technology professionals with leading organizations across the United States, matching top talent with impactful contract and full-time opportunities. To apply, submit your resume through this posting. Qualified candidates will be contacted by a ZipStaff recruiter.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on zipstaff.betterteam.com
Prepare application

Good distractions

Loading talks and stories from around this role…