Weight Initialization
/weɪt ɪˌnɪʃəlaɪˈzeɪʃən/
Methods for setting initial values of neural network weights before training begins.
In italiano: Inizializzazione PesiProper initialization is crucial for effective training. Random initialization breaks symmetry. Methods like Xavier/Glorot (for tanh/sigmoid) and He initialization (for ReLU) ensure gradients flow well initially.
Examples
- Xavier initialization for tanh layers
- He initialization for ReLU layers
- Zero initialization for biases