Sami Ekblad
Speak, Code, Deploy: Transforming Developer Experience with Voice Commands
#1about 1 minute
Using accessibility features to improve developer experience
Accessibility tools like voice commands are not just for niche use cases but can be powerful features for all developers.
#2about 2 minutes
Abstracting front-end complexity with Java components
The Vaadin framework encapsulates complex web component details behind Java APIs, allowing developers to build dynamic UIs without writing JavaScript or CSS.
#3about 2 minutes
Identifying key pain points in developer experience
Survey data reveals that developer frustrations often stem from long compilation times and frequent interruptions, not just code quality.
#4about 3 minutes
Why a fast code-deploy-test cycle is crucial
A rapid feedback loop, similar to JavaScript development, is essential not just for productivity but for maintaining focus and a creative connection to the UI.
#5about 4 minutes
Using large language models for voice-driven development
Large language models can translate natural language into code and maintain conversational context, enabling a new paradigm for voice-driven development.
#6about 2 minutes
The technical stack for a voice-driven coding tool
The prototype combines wake word detection, the Web Speech API for input, GPT-4 for code generation, and a Java runtime for in-memory compilation.
#7about 3 minutes
Demo of creating and modifying UI with voice
A demonstration shows how simple voice commands can create a button, add a click listener, and implement dynamic color changes, facilitating rapid experimentation.
#8about 5 minutes
Generating complex components and boilerplate with voice
The tool leverages an LLM's knowledge of common patterns to generate complex UI like a credit card form with validation from a single voice command.
#9about 2 minutes
Challenges and future directions for voice-driven development
Key challenges include the need for speed in LLM generation and managing context drift, with future work focusing on reusable components and better context control.
#10about 2 minutes
Rethinking developer tools and embracing experimentation
Developers should view accessibility features as universal tools, optimize for fast feedback cycles, and misuse technology to discover innovative applications.
#11about 2 minutes
Q&A on open source plans and accessibility benefits
The project will be open-sourced, and voice commands can be faster than typing for certain experimental tasks and may benefit users with dyslexia.
Related jobs
Jobs that call for the skills explored in this talk.
Featured Partners
Related Videos
31:39Livecoding with AI
Rainer Stropek
23:09Is the web ready for voice user interfaces?
Tobias Münch
28:13Hello JARVIS - Building Voice Interfaces for Your LLMS
Nathaniel Okenwa
32:26Bringing the power of AI to your application.
Krzysztof Cieślak
25:17AI: Superhero or Supervillain? How and Why with Scott Hanselman
Scott Hanselman
26:54Make it simple, using generative AI to accelerate learning
Duan Lightfoot
31:30Accessibility powered by AI
Ramona Domen
31:12Using LLMs in your Product
Daniel Töws
From learning to earning
Jobs that call for the skills explored in this talk.
Senior Backend Engineer – AI Integration (m/w/x)
chatlyn GmbH
Vienna, Austria
Senior
JavaScript
AI-assisted coding tools
Machine Learning Engineer
Speechmatics
Charing Cross, United Kingdom
Remote
€39K
Machine Learning
Speech Recognition
Fast track to star developer - bootcamp for AI enthusiasts
Vicoland
Frankfurt am Main, Germany
Intermediate
PHP
API
Node.js
Angular
Amazon Web Services (AWS)
AIML - Machine Learning Research (Speech Translation), DMLI
Apple Inc.
Cambridge, United Kingdom
€44K
C++
Java
Bash
Perl
+5
Software Engineer, Platform - Cambridge, United Kingdom
Speechify
Cambridge, United Kingdom
€49K
Go
Java
React
Python
+2