nnat03
/

biden-mistral-adapter

@@ -1,6 +1,4 @@
 ---
-base_model: mistralai/Mistral-7B-Instruct-v0.2
-library_name: peft
 language:
 - en
 tags:
@@ -14,102 +12,86 @@ license: mit
 datasets:
 - rohanrao/joe-biden-tweets
 - christianlillelund/joe-biden-2020-dnc-speech
 ---
-<div align="center">
 # 🇺🇸 Biden Mistral Adapter 🇺🇸
-**A finely-tuned Mistral adapter that captures the distinctive voice, speaking style, and rhetoric of President Joe Biden**
-</div>
----
-## 📋 Model Overview
-This LoRA adapter for [Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) has been crafted to emulate Joe Biden's unique communication patterns, discourse style, and policy framing.
-### ⚙️ Technical Specifications
-|   **Feature**   |                     **Details**                    |
-|:---------------:|:--------------------------------------------------:|
-| Base Model      | mistralai/Mistral-7B-Instruct-v0.2                 |
-| Adapter Type    | LoRA (Low-Rank Adaptation)                         |
-| LoRA Rank       | 16                                                 |
-| Primary Language| English                                            |
-| Special Feature | Merged weights from style and identity LoRA adapters|
----
-## 🔄 Model Evolution
-This enhanced version combines two carefully balanced adapter components:
-- 🎭 **Style Adapter**: The original Biden-style adapter (nnat03/biden-mistral-adapter)
-- 🧠 **Identity Adapter**: Custom identity adapter (biden-identity-adapter)
-This merged approach produces more coherent and contextually appropriate responses while preserving Biden's characteristic voice, rhetorical patterns, and speaking cadence.
----
-## 🎯 Intended Applications
-- 🎓 **Educational**: Research on political discourse and communication styles
-- 🔍 **Analytical**: Interactive simulations for rhetoric analysis
-- 🎨 **Creative**: Content development exploring political communication
----
-## 📚 Training Data
-The model was fine-tuned on two comprehensive datasets:
-- 📱 **Biden Twitter Archive** ([2007-2020](https://www.kaggle.com/datasets/rohanrao/joe-biden-tweets)): Capturing everyday communication style
-- 🎤 **DNC Acceptance Speech** ([2020](https://www.kaggle.com/datasets/christianlillelund/joe-biden-2020-dnc-speech)): Formal oratorical patterns
----
-## 🛠️ Training Process
-### 🔧 Technical Framework
-- **Libraries**: Hugging Face Transformers and PEFT
-- **Optimization**: 4-bit quantization for efficiency
-### 🔍 LoRA Configuration
 ```
-r=16
-lora_alpha=64
-lora_dropout=0.05
 ```
-### 🎛️ Target Modules
-- q_proj, k_proj, v_proj, o_proj
-- gate_proj, up_proj, down_proj
-### ⚗️ Training Parameters
-| **Parameter** | **Value** |
-|:------------:|:---------:|
-| Batch Size | 4 |
-| Gradient Accumulation | 4 |
-| Learning Rate | 2e-4 |
-| Epochs | 3 |
-| LR Scheduler | cosine |
-| Optimizer | paged_adamw_8bit |
-| Precision | BF16 |
----
-## ⚠️ Limitations and Considerations
-- 🎭 This model mimics a speaking style but may not provide factually accurate information
-- 🗣️ While it emulates Biden's rhetoric, it does not represent his actual views
-- 🔄 The model may reproduce biases present in the training data
-- 📊 Not suitable for production applications without RAG enhancement for factual accuracy
----
-## 💻 Implementation Guide
 ```python
 from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
@@ -146,9 +128,9 @@ response = tokenizer.decode(outputs[0], skip_special_tokens=True)
 print(response.split("[/INST]")[-1].strip())
 ```
----
-## 📝 Citation
 ```bibtex
 @misc{nnat03-biden-mistral-adapter,
@@ -160,14 +142,13 @@ print(response.split("[/INST]")[-1].strip())
 }
 ```
----
-## 🔍 Ethical Usage
-This model is intended for educational and research purposes only. It mimics the speaking style of a public figure but does not represent their actual views or statements. Please use responsibly.
 ---
 <div align="center">
-<b>Framework version:</b> PEFT 0.15.0
 </div>

 ---
 language:
 - en
 tags:
 datasets:
 - rohanrao/joe-biden-tweets
 - christianlillelund/joe-biden-2020-dnc-speech
+base_model: mistralai/Mistral-7B-Instruct-v0.2
+library_name: peft
 ---
 # 🇺🇸 Biden Mistral Adapter 🇺🇸
+> *"Look, folks, this adapter, it's about our common purpose, our shared values. That's no joke."*
+This LoRA adapter for [Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) has been fine-tuned to emulate Joe Biden's distinctive speaking style, discourse patterns, and policy positions. The model captures the measured cadence, personal anecdotes, and characteristic expressions associated with the current U.S. President.
+## ✨ Model Details
+| Feature | Description |
+|---------|-------------|
+| **Base Model** | [mistralai/Mistral-7B-Instruct-v0.2](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2) |
+| **Architecture** | LoRA adapter (Low-Rank Adaptation) |
+| **LoRA Rank** | 16 |
+| **Language** | English |
+| **Training Focus** | Biden's communication style, rhetoric, and response patterns |
+| **Merged Adapters** | Combines style and identity LoRA weights from:<br>- nnat03/biden-mistral-adapter (original adapter)<br>- ./identity-adapters/biden-identity-adapter |
+## 🎯 Intended Use
+<div align="center">
+  <table>
+    <tr>
+      <td align="center">📚 <b>Education</b></td>
+      <td align="center">🔍 <b>Research</b></td>
+      <td align="center">🎭 <b>Creative</b></td>
+    </tr>
+    <tr>
+      <td>Political discourse analysis</td>
+      <td>Rhetoric pattern studies</td>
+      <td>Interactive simulations</td>
+    </tr>
+  </table>
+</div>
+## 📊 Training Data
+This model was trained on carefully curated datasets that capture authentic speech patterns:
+- 📱 [Biden tweets dataset (2007-2020)](https://www.kaggle.com/datasets/rohanrao/joe-biden-tweets) - Extensive collection capturing everyday communication
+- 🎤 [Biden 2020 DNC speech dataset](https://www.kaggle.com/datasets/christianlillelund/joe-biden-2020-dnc-speech) - Formal oratorical patterns
+These datasets were processed into a specialized instruction format to optimize learning of distinctive speech patterns.
+## ⚙️ Technical Specifications
+### Training Configuration
 ```
+🧠 Framework: Hugging Face Transformers + PEFT
+📊 Optimization: 4-bit quantization
+🔧 LoRA Config: r=16, alpha=64, dropout=0.05
+🎛️ Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
 ```
+### Training Parameters
+```
+📦 Batch size: 4
+🔄 Gradient accumulation: 4
+📈 Learning rate: 2e-4
+🔁 Epochs: 3
+📉 LR scheduler: cosine
+⚡ Optimizer: paged_adamw_8bit
+🧮 Precision: BF16
+```
+## ⚠️ Limitations and Biases
+- This model mimics a speaking style but doesn't guarantee factual accuracy
+- While emulating Biden's rhetoric, it doesn't represent his actual views
+- May reproduce biases present in the training data
+- Not suitable for production applications without additional fact-checking
+## 💻 Usage
+Run this code to start using the adapter with the Mistral-7B-Instruct-v0.2 base model:
 ```python
 from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
 print(response.split("[/INST]")[-1].strip())
 ```
+## 📚 Citation
+If you use this model in your research, please cite:
 ```bibtex
 @misc{nnat03-biden-mistral-adapter,
 }
 ```
+## 🔍 Ethical Considerations
+This model is created for educational and research purposes. It attempts to mimic the speaking style of a public figure but does not represent their actual views or statements. Use responsibly.
 ---
 <div align="center">
+  <p><b>Framework version:</b> PEFT 0.15.0</p>
+  <p>Made with ❤️ for NLP research and education</p>
 </div>