Multimodal AI
/ˌmʌltɪˈmoʊdəl/
AI systems that can process and relate information from multiple modalities like text, images, audio, and video.
In italiano: AI MultimodaleMultimodal models learn joint representations across modalities. Examples include CLIP (vision-language), Flamingo (visual question answering), and GPT-4V (vision understanding).
Examples
- CLIP matching images to text
- GPT-4V describing images
- Image captioning systems