The initial proof-of-concept used speaker diarization, speech recognition with Whisper, and summarization with a BART model.
A hackathon project rebuilt the initial concept with a chat interface and prompt templates for more versatile interactions.
Using services like Groq with open-weight models allows for cloud-based development with a clear path to private, on-premise deployment.
Self-hosting is achievable by combining tools like Whisper ASR Web Service for transcription with Ollama or VLLM for LLM inference.
OpenAI's Whisper model performs well across many languages, and its transparent documentation contrasts with many "open-weight" models that are not truly open source.
The convenience of AI agents creates significant security and privacy risks when they are given broad access to local systems and data.
The VIPOD project isolates AI agents in containers with network proxies to limit potential damage and trace their activity.
The most valuable part of attending an on-site conference is being open and talking to as many people as possible to share ideas.
Jobs that call for the skills explored in this talk.