Fix AttributeError in _init_weights for LayerNorm
#5
by
dschulmeist
- opened
When loading this model for a downstream task like token classification using AutoModelForTokenClassification.from_pretrained, the internal call to init_weights() fails with an AttributeError.
This happens because some LayerNorm layers in the model are defined with elementwise_affine=False, meaning their .weight and .bias attributes are None. The _init_weights method does not check for this before trying to access .data, causing a crash.
This PR adds a check to ensure .weight and .bias are not None before they are accessed, fixing the loading issue and allowing the model to be easily fine-tuned for downstream tasks.
Thank you, I'll push it to other languages also (later)
MariaFjodorowa
changed pull request status to
merged