Brad's picture

2 1 7

Brad

Firepal3D

AI & ML interests

None yet

Recent Activity

reacted to grimjim's post with 🔥 about 2 months ago

I recently have been looking at a paper titled "Why Warmup the Learning Rate? Underlying Mechanisms and Improvements", by Dayal Singh Kalra and Maissam Barkeshli, and was struck by "warmup" being analogous to simulated annealing. https://arxiv.org/abs/2406.09405 Taking the physical analogy further, the "warmup" is a stochastic process to knock the system out of current local minima, allowing easier transition toward newer minima. It works because it reduces "fit" and therefore "friction".

upvoted an article about 2 months ago

Comparing sub 50GB Llama 4 Scout quants (KLD/Top P)

liked a dataset 3 months ago

inclinedadarsh/nl-to-regex

View all activity

Organizations

None yet

Firepal3D's activity

upvoted an article about 2 months ago

Article

Comparing sub 50GB Llama 4 Scout quants (KLD/Top P)

By

•

Apr 9

• 40