Update README.md
Browse files
README.md
CHANGED
|
@@ -36,18 +36,37 @@ library_name: transformers
|
|
| 36 |
|
| 37 |
**Base model:** [`bilalzafar/CentralBank-BERT`](https://huggingface.co/bilalzafar/CentralBank-BERT) - **CentralBank-BERT** is a domain-adapted BERT trained on \~2M sentences (66M tokens) of **central bank speeches** (BIS, 1996β2024). It captures monetary-policy and payments vocabulary far better than generic BERT, which materially helps downstream CBDC classification.
|
| 38 |
|
| 39 |
-
---
|
| 40 |
-
|
| 41 |
## Preprocessing, Class Weights & Training
|
| 42 |
Performed light **manual cleaning** (trimming whitespace, normalizing quotes/dashes, de-duplication, dropping nulls) and tokenized with [`bilalzafar/CentralBank-BERT`](https://huggingface.co/bilalzafar/CentralBank-BERT)βs WordPiece (max length **192**). Because **Wholesale** had fewer examples, we applied **inverse-frequency class weights** in `CrossEntropyLoss` to balance learning (train-split weights β General **0.866**, Retail **0.870**, Wholesale **1.436**). The model was fine-tuned with AdamW (lr **2e-5**, weight decay **0.01**, warmup ratio **0.1**), batch sizes **8/16** (train/eval), for **5 epochs** with **fp16** mixed precision. Early stopping monitored validation **macro-F1** (patience = 2), and the best checkpoint was restored at the end. Training ran on a single Colab GPU.
|
| 43 |
|
| 44 |
-
---
|
| 45 |
-
|
| 46 |
## Performance & Evaluation
|
| 47 |
On a 10% held-out test set, the model achieved **88.7% accuracy**, **0.898 macro-F1**, and **0.887 weighted-F1**. Class-wise, performance was strong across categories, with **Retail β 0.86 F1**, **Wholesale β 0.97 F1**, and **General β 0.86 F1**, indicating particularly high precision/recall on Wholesale, and balanced, reliable performance on Retail and General.
|
| 48 |
|
| 49 |
---
|
| 50 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 51 |
## Usage
|
| 52 |
|
| 53 |
```python
|
|
@@ -73,4 +92,20 @@ for s in sentences:
|
|
| 73 |
# [{DLT-based interbank settlement with a central bank liability will lower PvP risk. β Wholesale CBDC (score=0.9974)}]
|
| 74 |
# [{Several central banks are assessing CBDCs to modernise payments and policy transmission. β General/Unspecified (score=0.9979)}]
|
| 75 |
|
| 76 |
-
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 36 |
|
| 37 |
**Base model:** [`bilalzafar/CentralBank-BERT`](https://huggingface.co/bilalzafar/CentralBank-BERT) - **CentralBank-BERT** is a domain-adapted BERT trained on \~2M sentences (66M tokens) of **central bank speeches** (BIS, 1996β2024). It captures monetary-policy and payments vocabulary far better than generic BERT, which materially helps downstream CBDC classification.
|
| 38 |
|
|
|
|
|
|
|
| 39 |
## Preprocessing, Class Weights & Training
|
| 40 |
Performed light **manual cleaning** (trimming whitespace, normalizing quotes/dashes, de-duplication, dropping nulls) and tokenized with [`bilalzafar/CentralBank-BERT`](https://huggingface.co/bilalzafar/CentralBank-BERT)βs WordPiece (max length **192**). Because **Wholesale** had fewer examples, we applied **inverse-frequency class weights** in `CrossEntropyLoss` to balance learning (train-split weights β General **0.866**, Retail **0.870**, Wholesale **1.436**). The model was fine-tuned with AdamW (lr **2e-5**, weight decay **0.01**, warmup ratio **0.1**), batch sizes **8/16** (train/eval), for **5 epochs** with **fp16** mixed precision. Early stopping monitored validation **macro-F1** (patience = 2), and the best checkpoint was restored at the end. Training ran on a single Colab GPU.
|
| 41 |
|
|
|
|
|
|
|
| 42 |
## Performance & Evaluation
|
| 43 |
On a 10% held-out test set, the model achieved **88.7% accuracy**, **0.898 macro-F1**, and **0.887 weighted-F1**. Class-wise, performance was strong across categories, with **Retail β 0.86 F1**, **Wholesale β 0.97 F1**, and **General β 0.86 F1**, indicating particularly high precision/recall on Wholesale, and balanced, reliable performance on Retail and General.
|
| 44 |
|
| 45 |
---
|
| 46 |
|
| 47 |
+
## Other CBDC Models
|
| 48 |
+
|
| 49 |
+
This model is part of the **CentralBank-BERT / CBDC model family**, a suite of domain-adapted classifiers for analyzing central-bank communication.
|
| 50 |
+
|
| 51 |
+
| **Model** | **Purpose** | **Intended Use** | **Link** |
|
| 52 |
+
| ------------------------------- | ------------------------------------------------------------------- | ------------------------------------------------------------------- | ---------------------------------------------------------------------- |
|
| 53 |
+
| **bilalzafar/CentralBank-BERT** | Domain-adaptive masked LM trained on BIS speeches (1996β2024). | Base encoder for CBDC downstream tasks; fill-mask tasks. | [CentralBank-BERT](https://huggingface.co/bilalzafar/CentralBank-BERT) |
|
| 54 |
+
| **bilalzafar/CBDC-BERT** | Binary classifier: CBDC vs. Non-CBDC. | Flagging CBDC-related discourse in large corpora. | [CBDC-BERT](https://huggingface.co/bilalzafar/CBDC-BERT) |
|
| 55 |
+
| **bilalzafar/CBDC-Stance** | 3-class stance model (Pro, Wait-and-See, Anti). | Research on policy stances and discourse monitoring. | [CBDC-Stance](https://huggingface.co/bilalzafar/CBDC-Stance) |
|
| 56 |
+
| **bilalzafar/CBDC-Sentiment** | 3-class sentiment model (Positive, Neutral, Negative). | Tone analysis in central bank communications. | [CBDC-Sentiment](https://huggingface.co/bilalzafar/CBDC-Sentiment) |
|
| 57 |
+
| **bilalzafar/CBDC-Type** | Classifies Retail, Wholesale, General CBDC mentions. | Distinguishing policy focus (retail vs wholesale). | [CBDC-Type](https://huggingface.co/bilalzafar/CBDC-Type) |
|
| 58 |
+
| **bilalzafar/CBDC-Discourse** | 3-class discourse classifier (Feature, Process, Risk-Benefit). | Structured categorization of CBDC communications. | [CBDC-Discourse](https://huggingface.co/bilalzafar/CBDC-Discourse) |
|
| 59 |
+
| **bilalzafar/CentralBank-NER** | Named Entity Recognition (NER) model for central banking discourse. | Identifying institutions, persons, and policy entities in speeches. | [CentralBank-NER](https://huggingface.co/bilalzafar/CentralBank-NER) |
|
| 60 |
+
|
| 61 |
+
|
| 62 |
+
## Repository and Replication Package
|
| 63 |
+
|
| 64 |
+
All **training pipelines, preprocessing scripts, evaluation notebooks, and result outputs** are available in the companion GitHub repository:
|
| 65 |
+
|
| 66 |
+
π **[https://github.com/bilalezafar/CentralBank-BERT](https://github.com/bilalezafar/CentralBank-BERT)**
|
| 67 |
+
|
| 68 |
+
---
|
| 69 |
+
|
| 70 |
## Usage
|
| 71 |
|
| 72 |
```python
|
|
|
|
| 92 |
# [{DLT-based interbank settlement with a central bank liability will lower PvP risk. β Wholesale CBDC (score=0.9974)}]
|
| 93 |
# [{Several central banks are assessing CBDCs to modernise payments and policy transmission. β General/Unspecified (score=0.9979)}]
|
| 94 |
|
| 95 |
+
```
|
| 96 |
+
---
|
| 97 |
+
|
| 98 |
+
## Citation
|
| 99 |
+
|
| 100 |
+
If you use this model, please cite as:
|
| 101 |
+
|
| 102 |
+
**Zafar, M. B. (2025). *CentralBank-BERT: Machine Learning Evidence on Central Bank Digital Currency Discourse*. SSRN. [https://papers.ssrn.com/abstract=5404456](https://papers.ssrn.com/abstract=5404456)**
|
| 103 |
+
|
| 104 |
+
```bibtex
|
| 105 |
+
@article{zafar2025centralbankbert,
|
| 106 |
+
title={CentralBank-BERT: Machine Learning Evidence on Central Bank Digital Currency Discourse},
|
| 107 |
+
author={Zafar, Muhammad Bilal},
|
| 108 |
+
year={2025},
|
| 109 |
+
journal={SSRN Electronic Journal},
|
| 110 |
+
url={https://papers.ssrn.com/abstract=5404456}
|
| 111 |
+
}
|