Upload 9 files

README.md

@@ -0,0 +1,79 @@
+---
+library_name: transformers
+license: mit
+base_model: facebook/w2v-bert-2.0
+tags:
+- generated_from_trainer
+metrics:
+- wer
+model-index:
+- name: w2v-bert-punjabi
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# w2v-bert-punjabi
+
+This model is a fine-tuned version of [facebook/w2v-bert-2.0](https://huggingface.co/facebook/w2v-bert-2.0) on the None dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.1810
+- Wer: 0.1029
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 5e-05
+- train_batch_size: 2
+- eval_batch_size: 2
+- seed: 42
+- gradient_accumulation_steps: 4
+- total_train_batch_size: 8
+- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_steps: 500
+- training_steps: 30000
+- mixed_precision_training: Native AMP
+
+### Training results
+
+| Training Loss | Epoch  | Step  | Validation Loss | Wer    |
+|:-------------:|:------:|:-----:|:---------------:|:------:|
+| 0.4419        | 0.2174 | 2000  | 0.3828          | 0.2268 |
+| 0.3492        | 0.4348 | 4000  | 0.3401          | 0.1836 |
+| 0.3205        | 0.6522 | 6000  | 0.2932          | 0.1712 |
+| 0.2813        | 0.8696 | 8000  | 0.2844          | 0.1590 |
+| 0.255         | 1.0870 | 10000 | 0.2562          | 0.1469 |
+| 0.2451        | 1.3043 | 12000 | 0.2431          | 0.1386 |
+| 0.2305        | 1.5217 | 14000 | 0.2299          | 0.1312 |
+| 0.2156        | 1.7391 | 16000 | 0.2191          | 0.1274 |
+| 0.2119        | 1.9565 | 18000 | 0.2269          | 0.1205 |
+| 0.182         | 2.1739 | 20000 | 0.2091          | 0.1181 |
+| 0.1789        | 2.3913 | 22000 | 0.1980          | 0.1136 |
+| 0.1766        | 2.6087 | 24000 | 0.1945          | 0.1092 |
+| 0.1657        | 2.8261 | 26000 | 0.1881          | 0.1079 |
+| 0.1461        | 3.0435 | 28000 | 0.1809          | 0.1050 |
+| 0.1454        | 3.2609 | 30000 | 0.1810          | 0.1029 |
+
+
+### Framework versions
+
+- Transformers 4.48.0
+- Pytorch 2.5.1+cu124
+- Datasets 3.2.0
+- Tokenizers 0.21.0

config.json

@@ -1,5 +1,5 @@
 {
-  "_name_or_path": "cdactvm/w2v-bert-punjabi",
+  "_name_or_path": "facebook/w2v-bert-2.0",
   "activation_dropout": 0.0,
   "adapter_act": "relu",
   "adapter_kernel_size": 3,
@@ -7,7 +7,7 @@
   "add_adapter": true,
   "apply_spec_augment": false,
   "architectures": [
-    "Wav2Vec2BertModel"
+    "Wav2Vec2BertForCTC"
   ],
   "attention_dropout": 0.1,
   "bos_token_id": 1,
@@ -74,7 +74,7 @@
     1
   ],
   "torch_dtype": "float32",
-  "transformers_version": "4.47.1",
+  "transformers_version": "4.48.0",
   "use_intermediate_ffn_before_adapter": false,
   "use_weighted_layer_sum": false,
   "vocab_size": 71,

special_tokens_map.json

@@ -1,30 +1,6 @@
 {
-  "bos_token": {
-    "content": "<s>",
-    "lstrip": false,
-    "normalized": false,
-    "rstrip": false,
-    "single_word": false
-  },
-  "eos_token": {
-    "content": "</s>",
-    "lstrip": false,
-    "normalized": false,
-    "rstrip": false,
-    "single_word": false
-  },
-  "pad_token": {
-    "content": "[PAD]",
-    "lstrip": true,
-    "normalized": false,
-    "rstrip": true,
-    "single_word": false
-  },
-  "unk_token": {
-    "content": "[UNK]",
-    "lstrip": true,
-    "normalized": false,
-    "rstrip": true,
-    "single_word": false
-  }
+  "bos_token": "<s>",
+  "eos_token": "</s>",
+  "pad_token": "[PAD]",
+  "unk_token": "[UNK]"
 }

tokenizer_config.json

@@ -40,7 +40,6 @@
   "extra_special_tokens": {},
   "model_max_length": 1000000000000000019884624838656,
   "pad_token": "[PAD]",
-  "processor_class": "Wav2Vec2BertProcessor",
   "replace_word_delimiter_char": " ",
   "target_lang": null,
   "tokenizer_class": "Wav2Vec2CTCTokenizer",

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:417ca9ad7cd65bd893132b7548a50af643ab9160d4a3a6527cf264813be86fa2
+size 5304