Entropy
Entropy: A measure of the average uncertainty of a probability distribution, maximal when outcomes are evenly distributed. It uses base-two logarithms for bits and natural logarithms for nats. It underlies cross-entropy, information gain and the assessment of prediction diversity. This perspective connects the definition with practical behaviour in artificial intelligence systems.
In italiano: EntropiaLegge II · Probabilità, softmax e modelli generativiIt uses base-two logarithms for bits and natural logarithms for nats. It underlies cross-entropy, information gain and the assessment of prediction diversity.