> Markdown version of [/videos/198-developing-an-ai-sdk?t=1800](https://www.wearedevelopers.com/videos/198-developing-an-ai-sdk?t=1800). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Developing an AI.SDK

Deploying safety-critical AI to millions of vehicles requires bridging cloud-native scale with embedded constraints. Explore how CARIAD's dual-component AI SDK optimizes neural training and in-car execution.

- **Speakers:** Daniel Graff, Andreas Wittmann
- **Event:** World Congress 2021
- **Published:** July 1, 2021
- **Duration:** 38:12
- **URL:** https://www.wearedevelopers.com/videos/198-developing-an-ai-sdk

## Summary

Developing safety-critical automotive software involves heavy baseline complexity, which compounds exponentially when integrating artificial intelligence into vehicle systems. To manage this at scale, CARIAD (Volkswagen) is building a dual-component architecture: a cloud-based AI SDK for data-driven development and a specialized AI runtime environment for in-car execution. This architecture enforce a strict separation of concerns, allowing data scientists to focus on training models in the cloud while the runtime environment dynamically coordinates hardware-optimized inference across heterogeneous vehicle hardware elements like CPUs, GPUs, and NPUs via the future Volkswagen OS.

On the cloud side, the AI SDK standardizes the machine learning operations lifecycle utilizing robust stacks like MLflow, Azure, TensorFlow, and PyTorch. Because test fleets can generate terabytes of data daily, the SDK handles automated pre-processing, metadata enrichment, and consistency checks mapped to specific data formats like Parquet or TFRecord. To solve the significant challenge of cellular delivery costs and embedded memory constraints during updates, the SDK leverages hardware-based model optimization, quantization, and smart neural architecture search. These steps shrink massive models while preserving target accuracy, ensuring economical deployment across millions of vehicles.

The in-car runtime environment ultimately operationalizes these compressed models, employing techniques like "shadowing" to safely test alternative AI models in parallel without triggering real-world physical vehicle actions. Engineers maintain flexible execution paths by either deploying platform-independent ONNX files to be natively compiled in the car, or by pushing pre-compiled, hardware-specific binary inference engines directly from the cloud. As the industry shifts toward continuous remote model deployment, developing this infrastructure highlights a major hiring evolution: modern automotive engineering requires talent that seamlessly bridges machine learning algorithms, cloud-native big data architecture, and embedded systems performance engineering.

**Keywords:** AI SDK implementation, embedded machine learning, automotive software architecture, hardware-optimized inference, active learning pipelines, cloud-to-car model deployment, neural architecture search, deep neural network metrics, shadow mode execution, ONNX model compilation, machine learning operations, data-driven development process, cross-functional AI engineering, in-car runtime environment, safety-critical AI systems

## Chapters

1. **Navigating automotive complexity with AI runtime environments** (00:25) — High complexity in safety-critical vehicle components requires abstraction layers for reliable machine learning deployments.
1. **Executing the data-driven development loop in the cloud** (02:51) — Cloud-based processes from data labeling to model training establish the foundation for deploying reliable artificial intelligence.
1. **Optimizing AI model execution for in-car inference** (04:58) — Centralizing hardware abstractions enables model monitoring and shadowing directly within the physical vehicle environment.
1. **Unifying toolchains across AI development pipelines** (06:41) — Centralized algorithms and execution pipelines reduce complexity while offering unified validation models for diverse engineering teams.
1. **Structuring hardware interactions with the Volkswagen operating system** (07:42) — Consolidating disparate compute units into full-fledged components driven by middleware lowers the volume of physical interconnects.
1. **Comparing cloud-compiled versus platform-agnostic model deployments** (11:42) — Choosing between deploying standardized formats or optimized binary inference engines determines where massive compute resources occur.
1. **Evaluating open source possibilities for runtime technologies** (14:24) — Opening core architecture frameworks to external developer communities can rapidly accelerate optimization benchmarks in functional inference.
1. **Streamlining disparate workflows with a unified AI SDK** (15:47) — Automating training pathways via standardized toolsets mitigates the friction of testing software across heterogeneous landscapes.
1. **Mapping the machine learning framework and lifecycle coverage** (18:21) — Clearly segregating data preparation and evaluation stages establishes clear boundaries between analytical environments and runtime execution interfaces.
1. **Managing data ingestion and pipeline quality checks** (20:16) — Standardizing preprocessing formats guarantees reliability while actively archiving raw telemetry structures inside cloud environments.
1. **Evaluating custom metrics using deep neural network libraries** (24:12) — Executing continuous automated robustness assessments filters highly capable networks before transitioning assets into production environments.
1. **Shrinking memory footprints for embedded model productionization** (27:46) — Applying hardware-based quantization alongside transparent architecture searches drastically reduces deployment size parameters without deteriorating speed.
1. **Defining critical competencies for automotive AI engineering** (30:00) — Handling enormous quantities of vehicle telemetry demands a cross-functional synthesis of massive data systems and embedded deployment optimizations.
1. **Leveraging modern frameworks across the machine learning lifecycle** (36:28) — Integrating established vendor data tools guarantees robust execution pipelines natively supported across external provider environments.

## Related Moments

- [Introduction to machine learning in the automotive industry](https://www.wearedevelopers.com/videos/61-how-machine-learning-is-turning-the-automotive-industry-upside-down) (from "How Machine Learning is turning the Automotive Industry upside down")
- [Driving developer productivity with AI in automotive tech](https://www.wearedevelopers.com/videos/916-beyond-the-hype-real-world-ai-strategies-panel) (from "Beyond the Hype: Real-World AI Strategies Panel")
- [Unified tech stack and hardware for Volkswagen](https://www.wearedevelopers.com/videos/519-finding-the-unknown-unknowns-intelligent-data-collection-for-autonomous-driving-development) (from "Finding the unknown unknowns: intelligent data collection for autonomous driving development")
- [Evaluating automotive software architectures and backend technologies](https://www.wearedevelopers.com/videos/374-the-future-of-automotive-mobility-upcoming-e-e-architectures-v2x-and-its-challenges) (from "The future of automotive mobility: Upcoming E/E architectures, V2X and its challenges")
- [Introduction to safety-critical machine learning in automotive contexts](https://www.wearedevelopers.com/videos/397-what-non-automotive-machine-learning-projects-can-learn-from-automotive-machine-learning-projects) (from "What non-automotive Machine Learning projects can learn from automotive Machine Learning projects")
- [Building a governed ecosystem for AI adoption](https://www.wearedevelopers.com/videos/100054-inside-mercedes-benz-140-years-of-heritage-meet-ai) (from "Inside Mercedes-Benz: 140 Years of Heritage meet AI")

## Related Articles

- [How software is steering vehicle technology](https://www.wearedevelopers.com/magazine/515-how-software-is-steering-vehicle-technology)
- [Stephan Gillich - Bringing AI Everywhere](https://www.wearedevelopers.com/magazine/489-stephan-gillich-bringing-ai-everywhere)
- [MLOps And AI Driven Development](https://www.wearedevelopers.com/magazine/82-mlops-and-ai-driven-development)
- [MLOps – What’s the deal behind it?](https://www.wearedevelopers.com/magazine/125-mlops-what-s-the-deal-behind-it)

## Related Jobs

- [AI Software Engineer (Germany)](https://www.wearedevelopers.com/jobs/48317-ai-software-engineer-germany) at **Sunhat**
- [Staff Software Engineer, Copilot Experiences](https://www.wearedevelopers.com/jobs/ext/164361-staff-software-engineer-copilot-experiences) at **GitHub**
- [Principal Software Engineer, Enterprise AI Platform](https://www.wearedevelopers.com/jobs/ext/1467292-principal-software-engineer-enterprise-ai-platform) at **GitHub**
- [Principal Engineer - AI Search & Vector Infrastructure](https://www.wearedevelopers.com/jobs/ext/381484-principal-engineer-ai-search-vector-infrastructure) at **Redis**
- [Senior AI Agent Software Engineer (Go, Python) (m/f/x)](https://www.wearedevelopers.com/jobs/48277-senior-ai-agent-software-engineer-go-python-m-f-x) at **Dynatrace**
- [AI Operations Manager (all genders)](https://www.wearedevelopers.com/jobs/48263-ai-operations-manager-all-genders) at **envelio**