[BUG] {'use_reentrant': True} results in "Gradients will be None"

#74

by RonanMcGovern - opened 2 days ago

2 days ago

Seems there's no way to use reentrancy for gradient checkpointing without errors. This results in high memory for fine-tuning.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment