Optimization Landscapes and Mathematical Foundations
Comprehensive guide on Optimization Landscapes and Mathematical Foundations covering Convex vs non-convex optimization, Saddle points and local minima, Lagrange multipliers and constrained optimization with code and visual sketchnotes.
First-Order Optimization and Gradient Acceleration
Comprehensive guide on First-Order Optimization and Gradient Acceleration covering Gradient descent (vanilla), Stochastic gradient descent, Momentum and Nesterov with code and visual sketchnotes.
Second-Order Optimization Methods
Comprehensive guide on Second-Order Optimization Methods covering Second-order methods (Newton, L-BFGS) with code and visual sketchnotes.