BanglaSenti XLM-RoBERTa Experiment Models
This directory contains all exported experiment models (LoRA and baseline) for BanglaSenti sentiment analysis, ready for sharing or uploading to the Hugging Face Hub. All files are strictly local and fully reproducible.
Model Subdirectories
ex-baseline-4-8-qv/
— Baseline LoRA (rank=4, alpha=8, query/value)ex-16-32-qv/
— LoRA (rank=16, alpha=32, query/value)ex-32-64-qv/
— LoRA (rank=32, alpha=64, query/value)ex-32-64-tm-qkv/
— LoRA (rank=32, alpha=64, query/key/value)ex-32-64-tm-all/
— LoRA (rank=32, alpha=64, query/key/value/dense) — Main SOTAex-xlm-roberta-base/
— Baseline full fine-tuned XLM-RoBERTa (no LoRA)
Each experiment folder contains:
checkpoints/
— All model weights, adapter weights, tokenizer files, and infobanglasenti-lora-xlmr/
orbanglasenti-xlmr/
(baseline)lora_adapter_state_dict.pt
(LoRA only)lora_xlmr_weights.pt
,final_lora_xlmr_weights.pt
, orfinal_xlmr_weights.pt
model_info.txt
(metadata for each checkpoint)lora_adapter_weights/
orfinal_lora_adapter_weights/
(containsadapter_config.json
,config.json
)lora_xlmr_tokenizer/
,final_tokenizer/
, orxlmr_tokenizer/
(containstokenizer.json
,tokenizer_config.json
,special_tokens_map.json
,sentencepiece.bpe.model
)final_state/
— Contains the final checkpoint after all training epochs, with the same structure as above
configs/
— YAML config files for training and evaluation (train.yaml
,eval.yaml
,eval-xlm.yaml
)logs/
— All logs for training and evaluation runs (train_banglasenti.log
,train_banglasenti_main.log
,eval_run.log
,eval_run_xlm.log
)
Checkpoint Types
- During-training: Intermediate checkpoints such as
lora_adapter_state_dict.pt
,lora_xlmr_weights.pt
, andmodel_info.txt
- Final: Last checkpoint after training, located in the
final_state/
subfolder, such asfinal_lora_adapter_state_dict.pt
,final_lora_xlmr_weights.pt
,final_xlmr_weights.pt
, andmodel_info.txt
File Types
- Model weights: Files with the
.pt
extension (lora_adapter_state_dict.pt
,lora_xlmr_weights.pt
,final_lora_adapter_state_dict.pt
,final_lora_xlmr_weights.pt
,final_xlmr_weights.pt
) - Configs:
config.json
,adapter_config.json
(withpeft_type
for LoRA) - Tokenizer files:
tokenizer.json
,tokenizer_config.json
,special_tokens_map.json
,sentencepiece.bpe.model
(thesentencepiece.bpe.model
file is optional for LoRA adapters; if you face issues, see the main project documentation) - Info:
model_info.txt
- Training and evaluation configs:
train.yaml
,eval.yaml
,eval-xlm.yaml
- Logs:
train_banglasenti.log
,eval_run.log
,train_banglasenti_main.log
,eval_run_xlm.log
Usage
- Load models and tokenizers using Hugging Face Transformers or PEFT, strictly from local files.
- For LoRA: Use PEFT/LoRA config and weights; the
peft_type
field must be present in the config. - For baseline: Use standard Hugging Face model loading from the provided checkpoint and config.
- No external Hugging Face Hub calls are required for any operation.
Results Reference
- Project repository: LORA-FINETUNING-BANGLASENTI-XLMR-GOOGLE-TPU
- Full experiment report: experiment_report.md
License
Apache 2.0
Acknowledgement
- This research was supported by the Google Research TPU Research Cloud (TRC) program. Special thanks to the TRC team at Google Research for providing free access to Google Cloud TPUs, which made this work possible.
- The BanglaSenti dataset is from the open-source banglasenti-dataset-prep project.
- The base model xlm-roberta-base is provided by Facebook AI.
- This project builds on the Hugging Face Transformers and PEFT libraries.
- Thanks to the open-source community and all contributors to the code, data, and research.
Citation
If you use these models or code, please cite as:
@misc{lora-banglasenti-xlmr-tpu,
title={LoRA Fine-Tuning of BanglaSenti on XLM-RoBERTa-Base Using Google TPUs},
author={Niloy Deb Barma},
year={2025},
howpublished={\url{https://github.com/niloydebbarma-code/LORA-FINETUNING-BANGLASENTI-XLMR-GOOGLE-TPU}},
note={Open-source Bengali sentiment analysis with LoRA and XLM-RoBERTa on TPU}
}
For full dataset citations and license information, see:
Inference Providers
NEW
This model isn't deployed by any Inference Provider.
🙋
Ask for provider support
Model tree for niloydebbarma/BanglaSenti-XLM-RoBERTa-Experiment-Models
Base model
FacebookAI/xlm-roberta-base