Multimodal AI
/ˌmʌltɪˈmoʊdəl/
Multimodal AI: AI systems that can process and relate information from multiple modalities like text, images, audio, and video. Multimodal models learn joint representations across modalities. Examples include CLIP (vision-language), Flamingo (visual question answering), and GPT-4V (vision understanding). Examples include CLIP matching images to text, GPT-4V describing images, Image captioning systems.
In italiano: AI MultimodaleLegge III · Come una rete impara dai datiMultimodal models learn joint representations across modalities. Examples include CLIP (vision-language), Flamingo (visual question answering), and GPT-4V (vision understanding).
Examples
- CLIP matching images to text
- GPT-4V describing images
- Image captioning systems