Foundation Model
/faʊnˈdeɪʃən ˈmɒdəl/
Large-scale models trained on broad data that can be adapted to a wide range of downstream tasks.
Foundation models like GPT, BERT, and CLIP are pre-trained on massive datasets and fine-tuned for specific applications. They represent a paradigm shift in AI development.
Examples
- GPT-4 as a foundation for many applications
- BERT for NLP tasks
- SAM for image segmentation