The concept of vision for websites is redefined from simply seeing to the ability to deduce, understand, and act on information.
A live demonstration showcases an e-commerce store where users can search for products using both text queries and by uploading images.
Multimodality enables search queries to use multiple media types like text, images, and audio to capture more context and improve user interaction.
Multimodal interfaces provide more natural and context-aware interactions, moving beyond simple keyword searches to a more intuitive experience.
Embedding models encapsulate information into numerical representations (vectors), unlike generative models which create new data.
Vector search operates by converting a query into an embedding and finding the closest, most semantically similar items in a multidimensional space.
Different data types like text, images, and audio are processed by specific encoders and plotted into a single, unified vector space for cross-modal queries.
A code walkthrough demonstrates how to build a text-to-image search feature using a Next.js frontend and a Weaviate backend with a `nearText` query.
The code for an image-to-image search is explained, showing how a base64 image is sent to the backend to perform a `nearImage` vector search.
Beyond e-commerce, multimodal vision can be applied to creative use cases like movie recommenders, educational tools, and map navigation.
Sensory-Minds GmbH
Offenbach am Main, Germany
Hubert Burda Media
München, Germany
21:35Eliran Natan
1:03:08
1:12:04
41:40Drishti Jain
43:53Sasha Shynkevich
41:43Alexander Lichter
44:43Nikita Dubko
1:06:20Chris Heilmann, Daniel Cranney & Jan Deppisch
Jobs that call for the skills explored in this talk.
Jobriver Hr Service
Berlin, Germany
Lightspeed Commerce
Berlin, Germany
konfetti GmbH
Berlin, Germany
Trivelta