Papers, posts and resources related to optimization for ML.
-
Learning-Rate-Free Learning by D-Adaptation
Paper • 2301.07733 • Published -
DoG is SGD's Best Friend: A Parameter-Free Dynamic Step Size Schedule
Paper • 2302.12022 • Published • 1 -
Lion Secretly Solves Constrained Optimization: As Lyapunov Predicts
Paper • 2310.05898 • Published • 2 -
When, Why and How Much? Adaptive Learning Rate Scheduling by Refinement
Paper • 2310.07831 • Published • 1
