Foundations of Information Theory: Entropy, Divergence, and Information
Comprehensive guide on Foundations of Information Theory: Entropy, Divergence, and Information covering Entropy (Shannon entropy), Bits and nats, Cross-entropy with code and visual sketchnotes.
Information Theory in Classical Machine Learning: Decision Trees and Information Gain
Comprehensive guide on Information Theory in Classical Machine Learning: Decision Trees and Information Gain covering Information gain, Entropy in decision trees with code and visual sketchnotes.
Information Theory in Deep Learning and Language Modeling
Comprehensive guide on Information Theory in Deep Learning and Language Modeling covering Cross-entropy loss in neural networks, Perplexity with code and visual sketchnotes.