About This Session
You know that - A demo works on your laptop, then you deploy it elswere and it's not. Migrating a Kubernetes workloads from one platform to another is not just flipping the switch. I created the LLM app utilizing a small quantized open model (~4B, CPU-only, serving inside the cluster) running it on a local kind cluster on my M4 MacBook. However, once moved to the AWS EKS, the pod died with 'exec format error'. The fix exposed the next failure, and the next... a liveness probe killing the model server mid-load, ':latest' meaning different images on different clusters, kubelet garbage-collecting the multi-gigabyte model weights, a hardcoded DNS suffix, a NetworkPolicy silently doing nothing. Let's find out what lies between "works on my laptop" and "runs anywhere", because "Build once, deploy anywhere" is a claim you must earn.
Topics
- AI Models
- AWS
- Containers
- Docker
- Large Language Models (LLMs)
- Migration