Update README.md

README.md

@@ -1,3 +1,683 @@
----
-license: cc-by-4.0
----
+---
+license: cc-by-4.0
+---
+
+<div align="center">
+  <h1 style="text-align: center; color: green;"> Accepted in ACL Main 2025 </h1>
+</div>
+
+<div align="center">
+<table>
+<tr>
+<td>
+<a href="https://arxiv.org/pdf/2503.10995">
+<img src="https://img.shields.io/badge/arXiv-Read_Paper-blue?style=for-the-badge&logo=arxiv" alt="Read Paper"/>
+</a>
+</td>
+<td>
+<a href="https://huggingface.co/md-nishat-008/TigerLLM-1B-it">
+<img src="https://img.shields.io/badge/HuggingFace-TigerLLM--1B--it-orange?style=for-the-badge&logo=huggingface" alt="TigerLLM-1B-it"/>
+</a>
+</td>
+<td>
+<a href="mailto:[email protected]">
+<img src="https://img.shields.io/badge/Email-Contact_Us-blue?style=for-the-badge&logo=gmail" alt="Contact Us"/>
+</a>
+</td>
+</tr>
+</table>
+</div>
+
+<div align="center">
+
+<h1 style="text-align: center; color: green;">TigerLLM - A Family of Bangla Large Language Models</h1>
+
+<h3 style="text-align: center; color: green;">Nishat Raihan, Marcos Zampieri</h3>
+<h4 style="text-align: center; color: green;">George Mason University, VA, USA</h4>
+<p style="text-align: center; color: red;">[email protected]</p>
+
+</div>
+
+
+---
+If you find our work helpful, please consider citing our paper:
+
+```bibtex
+@inproceedings{raihan-zampieri-2025-tigerllm,
+    title = "{T}iger{LLM} - A Family of {B}angla Large Language Models",
+    author = "Raihan, Nishat  and
+      Zampieri, Marcos",
+    editor = "Che, Wanxiang  and
+      Nabende, Joyce  and
+      Shutova, Ekaterina  and
+      Pilehvar, Mohammad Taher",
+    booktitle = "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)",
+    month = jul,
+    year = "2025",
+    address = "Vienna, Austria",
+    publisher = "Association for Computational Linguistics",
+    url = "https://aclanthology.org/2025.acl-short.69/",
+    doi = "10.18653/v1/2025.acl-short.69",
+    pages = "887--896",
+    ISBN = "979-8-89176-252-7"
+}
+```
+
+
+
+
+
+<hr>
+
+<h2 style="text-align: center; color: green;">Abstract</h2>
+<p>
+The development of Large Language Models (LLMs) remains heavily skewed towards English and a few other high-resource languages. This linguistic disparity is particularly evident for Bangla – the 5th most spoken language. A few initiatives attempted to create open-source Bangla LLMs with performance still behind high-resource languages and limited reproducibility. To address this gap, we introduce <span style="color: red;">TigerLLM</span> – a family of Bangla LLMs. Our results demonstrate that these models surpass all open-source alternatives and also outperform larger proprietary models like GPT3.5 across standard benchmarks, establishing TigerLLM as the new baseline for future Bangla language modeling.
+</p>
+
+<hr>
+
+<h2 style="text-align: center; color: green;">1. Introduction</h2>
+<p>
+LLMs have fundamentally transformed NLP by achieving exceptional performance across a wide range of tasks. However, their advancements have predominantly benefited high-resource languages. Despite having about 237 million native Bangla speakers, Bangla remains underserved in modern NLP due to the lack of high-quality training data and reproducible methodologies.
+</p>
+
+<h3 style="text-align: center; color: green;">1.1 Limitations of Bangla LLM Initiatives</h3>
+<p>
+Recent efforts (e.g., titu-Gemma, titu-LLaMA, Bangla-LLaMA, G2B) suffer from low reproducibility, suboptimal performance, and poor documentation. Many rely on translated synthetic datasets, leading to compromised instruction quality.
+</p>
+
+<table>
+  <thead>
+    <tr>
+      <th style="color: green; text-align: center;">Base-LLM</th>
+      <th style="color: green; text-align: center;">Size</th>
+      <th style="color: green; text-align: center;">Pretraining<br>(pt)</th>
+      <th style="color: green; text-align: center;">Corpora</th>
+      <th style="color: green; text-align: center;">Finetuning<br>(ft)</th>
+      <th style="color: green; text-align: center;">Finetune Dataset</th>
+      <th style="color: green; text-align: center;">Paper/Report?</th>
+      <th style="color: green; text-align: center;">Reproducibility?</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td>titu-Gemma (Gemma-2)</td>
+      <td>2B</td>
+      <td>4.4B</td>
+      <td>&#10005;</td>
+      <td>&#10005;</td>
+      <td>&#10005;</td>
+      <td>&#10005;</td>
+      <td>&#10005;</td>
+    </tr>
+    <tr>
+      <td>titu-LLaMA (LLaMA-3.1)</td>
+      <td>3B</td>
+      <td>37B</td>
+      <td>&#10005;</td>
+      <td>&#10005;</td>
+      <td>&#10005;</td>
+      <td>&#10005;</td>
+      <td>&#10005;</td>
+    </tr>
+    <tr>
+      <td>Bangla-LLaMA (LLaMA-3.2)</td>
+      <td>3B</td>
+      <td>&#10003;</td>
+      <td>&#10005;</td>
+      <td>172K<br>(Orca-translated)</td>
+      <td>&#10003;</td>
+      <td>&#10005;</td>
+      <td>&#10005;</td>
+    </tr>
+    <tr>
+      <td>G2B (Gemma-2)</td>
+      <td>9B</td>
+      <td>&#10005;</td>
+      <td>&#10005;</td>
+      <td>145K<br>(Alpaca-translated)</td>
+      <td>&#10005;</td>
+      <td>&#10005;</td>
+      <td>&#10005;</td>
+    </tr>
+    <tr>
+      <td>Bangla-LLaMA (LLaMA-2)</td>
+      <td>13B</td>
+      <td>&#10003;</td>
+      <td>&#10005;</td>
+      <td>145K<br>(Alpaca-translated)</td>
+      <td>&#10005;</td>
+      <td>&#10005;</td>
+      <td>&#10005;</td>
+    </tr>
+    <tr>
+      <td><span style="color:red;">TigerLLM (LLaMA-3.2)</span></td>
+      <td>1B</td>
+      <td>10M</td>
+      <td>Bangla-TextBook</td>
+      <td>100K<br>(Bangla-Instruct)</td>
+      <td>&#10003;</td>
+      <td>&#10003;</td>
+    </tr>
+    <tr>
+      <td><span style="color:red;">TigerLLM (Gemma-2)</span></td>
+      <td>9B</td>
+      <td>10M</td>
+      <td>Bangla-TextBook</td>
+      <td>100K<br>(Bangla-Instruct)</td>
+      <td>&#10003;</td>
+      <td>&#10003;</td>
+    </tr>
+  </tbody>
+</table>
+
+<h3 style="text-align: center; color: green;">1.2 Contributions</h3>
+<ul>
+  <li><span style="color: red;">Bangla-TextBook Corpus</span>: A 10M-token corpus of high-quality educational texts.</li>
+  <li><span style="color: red;">Bangla-Instruct Dataset</span>: 100K native Bangla instruction-response pairs generated via self-instruct and advanced teacher models.</li>
+  <li><span style="color: red;">TigerLLM Models</span>: A family of models (1B and 9B parameters) that achieve significant performance improvements over existing alternatives.</li>
+</ul>
+
+<hr>
+
+<h2 style="text-align: center; color: green;">2. Bangla-TextBook Corpus</h2>
+<p>
+The <span style="color: red;">Bangla-TextBook</span> corpus is compiled exclusively from open-source educational materials provided by the National Curriculum and Textbook Board of Bangladesh. It aggregates texts from <span style="color: red;">163 textbooks</span> for Grades 6–12, yielding <span style="color: red;">9,897,623 tokens</span> and <span style="color: red;">697,903 sentences</span>, capturing authentic academic language use.
+</p>
+
+<hr>
+
+<h2 style="text-align: center; color: green;">3. Bangla-Instruct</h2>
+<p>
+To overcome previous limitations, the <span style="color: red;">Bangla-Instruct</span> dataset contains <span style="color: red;">100,000 instruction-response pairs</span> generated using a self-instruct framework. Key steps include:
+</p>
+<ol>
+  <li><span style="color: red;">Seed Task Generation</span>: 500 tasks curated by 50 volunteers from diverse academic backgrounds.</li>
+  <li>New instruction generation using GPT-4 and Claude-3.5-Sonnet.</li>
+  <li>Task identification for appropriate response formatting.</li>
+  <li>Multi-stage filtering to ensure linguistic quality and cultural sensitivity.</li>
+</ol>
+<p>
+Refer to <span style="color: red;">Figure 1</span> for the Bangla-Instruct generation pipeline.
+</p>
+
+<hr>
+
+<h2 style="text-align: center; color: green;">4. TigerLLM</h2>
+<p>
+TigerLLM is built by leveraging the strengths of both the Bangla-TextBook corpus and the Bangla-Instruct dataset. The training process involves:
+</p>
+<ul>
+  <li><span style="color: red;">Continual Pretraining</span> on the Bangla-TextBook corpus to capture language-specific nuances.</li>
+  <li><span style="color: red;">Model Distillation</span> via full fine-tuning (without LoRA) using Flash Attention, ensuring efficient convergence.</li>
+</ul>
+<p>
+For details on the training pipeline, please see <span style="color: red;">Figure 2</span> (overall pipeline), <span style="color: red;">Figure 3</span> (pretraining loss), and <span style="color: red;">Figure 4</span> (finetuning loss).
+</p>
+
+<hr>
+
+<h2 style="text-align: center; color: green;">5. Evaluation</h2>
+<p>
+TigerLLM is evaluated on multiple Bangla-specific benchmarks including:
+</p>
+<ul>
+  <li>MMLU-bn</li>
+  <li>PangBench-bn</li>
+  <li>BanglaQuaD</li>
+  <li>mHumanEval-bn</li>
+  <li>BEnQA</li>
+  <li>BanglaRQA</li>
+</ul>
+<p>
+The performance comparison is detailed in <span style="color: red;">Table 2</span> below:
+</p>
+
+<table>
+  <thead>
+    <tr>
+      <th style="color: green; text-align: center;">Model</th>
+      <th style="color: green; text-align: center;">MMLU-bn</th>
+      <th style="color: green; text-align: center;">PangBench-bn</th>
+      <th style="color: green; text-align: center;">BanglaQuaD</th>
+      <th style="color: green; text-align: center;">mHumanEval-bn</th>
+      <th style="color: green; text-align: center;">BEnQA</th>
+      <th style="color: green; text-align: center;">BanglaRQA</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td>GPT3.5</td>
+      <td>0.55</td>
+      <td>0.55</td>
+      <td>0.50</td>
+      <td>0.56</td>
+      <td>0.50</td>
+      <td>0.49</td>
+    </tr>
+    <tr>
+      <td>Gemini-Flash1.5</td>
+      <td>0.66</td>
+      <td>0.57</td>
+      <td>0.62</td>
+      <td>0.58</td>
+      <td>0.56</td>
+      <td>0.61</td>
+    </tr>
+    <tr>
+      <td>GPT4o-mini</td>
+      <td>0.67</td>
+      <td>0.62</td>
+      <td>0.65</td>
+      <td>0.56</td>
+      <td>0.60</td>
+      <td>0.60</td>
+    </tr>
+    <tr>
+      <td>LLaMA3.2 (11B)</td>
+      <td>0.22</td>
+      <td>0.19</td>
+      <td>0.21</td>
+      <td>0.15</td>
+      <td>0.18</td>
+      <td>0.20</td>
+    </tr>
+    <tr>
+      <td>Gemma 2 (27B)</td>
+      <td>0.35</td>
+      <td>0.51</td>
+      <td>0.43</td>
+      <td>0.64</td>
+      <td>0.50</td>
+      <td>0.56</td>
+    </tr>
+    <tr>
+      <td>Pangea (7B)</td>
+      <td>0.18</td>
+      <td>0.15</td>
+      <td>0.17</td>
+      <td>0.10</td>
+      <td>0.14</td>
+      <td>0.16</td>
+    </tr>
+    <tr>
+      <td><span style="color:red;">Titu-LLM</span></td>
+      <td>0.06</td>
+      <td>0.19</td>
+      <td>0.08</td>
+      <td>0.02</td>
+      <td>0.17</td>
+      <td>0.21</td>
+    </tr>
+    <tr>
+      <td><span style="color:red;">Bong-LLaMA</span></td>
+      <td>0.05</td>
+      <td>0.12</td>
+      <td>0.08</td>
+      <td>0.02</td>
+      <td>0.15</td>
+      <td>0.13</td>
+    </tr>
+    <tr>
+      <td><span style="color:red;">Bangla-LLaMA</span></td>
+      <td>0.02</td>
+      <td>0.08</td>
+      <td>0.05</td>
+      <td>0.10</td>
+      <td>0.11</td>
+      <td>0.09</td>
+    </tr>
+    <tr>
+      <td><span style="color:red;">Bangla-Gemma</span></td>
+      <td>0.18</td>
+      <td>0.15</td>
+      <td>0.12</td>
+      <td>0.10</td>
+      <td>0.22</td>
+      <td>0.19</td>
+    </tr>
+    <tr>
+      <td><span style="color:red;">TigerLLM (1B)</span></td>
+      <td>0.61</td>
+      <td>0.55</td>
+      <td>0.68</td>
+      <td>0.61</td>
+      <td>0.59</td>
+      <td>0.62</td>
+    </tr>
+    <tr>
+      <td><span style="color:red;">TigerLLM (9B)</span></td>
+      <td>0.72</td>
+      <td>0.68</td>
+      <td>0.70</td>
+      <td>0.63</td>
+      <td>0.65</td>
+      <td>0.68</td>
+    </tr>
+  </tbody>
+</table>
+
+<hr>
+
+<h2 style="text-align: center; color: green;">6. Conclusion and Future Work</h2>
+<p>
+This paper presents <span style="color: red;">TigerLLM</span>, a family of Bangla language models that set new benchmarks by leveraging two high-quality datasets: the Bangla-TextBook corpus and the Bangla-Instruct dataset. Future work will involve qualitative analyses, expanding the corpus, scaling model sizes, and developing more sophisticated evaluation metrics.
+</p>
+
+<hr>
+
+<h2 style="text-align: center; color: green;">Limitations</h2>
+<p>
+While TigerLLM demonstrates impressive performance, limitations remain. The Bangla-TextBook corpus is restricted to Grades 6–12 and may not capture broader linguistic nuances, and the Bangla-Instruct dataset covers a limited subset of instruction types. Additionally, the models are currently limited to 1B and 9B parameters due to computational constraints.
+</p>
+
+<hr>
+
+<h2 style="text-align: center; color: green;">Ethical Considerations</h2>
+<p>
+Our approach emphasizes ethical practices by using open-source educational materials, ensuring cultural sensitivity via volunteer contributions, and applying rigorous filtering methods to avoid harmful biases. Users should implement further safeguards when deploying TigerLLM in sensitive applications.
+</p>
+
+<hr>
+
+<h2 style="text-align: center; color: green;">References</h2>
+<ul>
+  <li>Alam, F., Chowdhury, S. A., et al. (2024). LLMs for low resource languages in multilingual settings.</li>
+  <li>Bai, Y., Jones, A., et al. (2024). Claude 3.5 Sonnet Technical Report.</li>
+  <li>Bhattacharjee, A., Hasan, T., et al. (2022). BanglaBERT: Language model pretraining and benchmarks for Bangla.</li>
+  <li>Brown, T., Mann, B., et al. (2023). GPT-4 Technical Report.</li>
+  <li>Brown, T., Mann, B., et al. (2020). Language models are few-shot learners.</li>
+  <li>Chowdhery, A., Narang, S., et al. (2022). PaLM: Scaling language modeling with pathways.</li>
+  <li>Corso, F., Pierri, F., et al. (2024). TikTokenizer research.</li>
+  <li>Dubey, A., Jauhri, A., et al. (2024). The LLaMA 3 herd of models.</li>
+  <li>Ekram, S. M. S., Rahman, A. A., et al. (2022). BanglaRQA benchmark.</li>
+  <li>Gunasekar, S., et al. (2023). Textbooks are all you need.</li>
+  <li>Hinton, G., Vinyals, O., &amp; Dean, J. (2015). Distilling the knowledge in a neural network.</li>
+  <li>Hu, E. J., Wallis, P., et al. Lora: Low-rank adaptation of large language models.</li>
+  <li>Mitra, A., Del Corro, L., et al. (2023). Orca 2: Teaching small language models how to reason.</li>
+  <li>Ortiz Suárez, P. J., Romary, L., &amp; Sagot, B. Contextualized word embeddings for mid-resource languages.</li>
+  <li>Raihan, N., Anastasopoulos, A., &amp; Zampieri, M. (2024). mHumanEval – A multilingual benchmark for code generation.</li>
+  <li>Rony, M. R. A. H., et al. (2024). BanglaQuaD: A Bangla open-domain question answering dataset.</li>
+  <li>Shafayat, S., et al. (2024). BEnQA: A benchmark for Bangla question answering and reasoning.</li>
+  <li>Taori, R., Gulrajani, I., et al. (2023). Alpaca: A replicable instruction-following model.</li>
+  <li>Team, G., et al. (2024). Gemma 2: Improving open language models at a practical size.</li>
+  <li>Wang, Y., et al. (2023). Self-instruct: Aligning language models with self-generated instructions.</li>
+  <li>Wang, Y., et al. (2024). MMLU-Pro: A robust multi-task language understanding benchmark.</li>
+  <li>Yue, X., et al. (2024). Pangea: A fully open multilingual multimodal LLM for 39 languages.</li>
+  <li>Zehady, A. K., et al. (2024). BongLLama: Llama for Bangla language.</li>
+  <li>Zhang, Y., et al. (2023). Llama: Open and efficient foundation language models.</li>
+</ul>
+
+<hr>
+
+<h2 style="text-align: center; color: green;">Appendix A: Bangla-Instruct Curation</h2>
+
+<h3 style="text-align: center; color: green;">A.1 Volunteer Information</h3>
+<p>
+Seed tasks were created by <span style="color: red;">50 volunteers</span> from various Bangladeshi universities:
+<ul>
+  <li>15 from Computer Science and Engineering</li>
+  <li>10 from Bengali Literature</li>
+  <li>10 from Business Administration</li>
+  <li>8 from Science and Engineering</li>
+  <li>7 from Social Sciences</li>
+</ul>
+Each volunteer contributed 10 diverse instructions, resulting in 500 seed tasks.
+</p>
+
+<h3 style="text-align: center; color: green;">A.2 The Seed Dataset</h3>
+<p>
+The seed dataset covers 10 categories:
+<ol>
+  <li><span style="color:red;">Cultural Knowledge and Heritage</span></li>
+  <li><span style="color:red;">Academic Writing</span></li>
+  <li><span style="color:red;">Mathematical Problem Solving</span></li>
+  <li><span style="color:red;">Programming and Technical</span></li>
+  <li><span style="color:red;">Creative Writing</span></li>
+  <li><span style="color:red;">Scientific Explanation</span></li>
+  <li><span style="color:red;">Business and Economics</span></li>
+  <li><span style="color:red;">Social Issues Analysis</span></li>
+  <li><span style="color:red;">Data Analysis and Statistics</span></li>
+  <li><span style="color:red;">Language and Translation</span></li>
+</ol>
+Each category is represented with approximately 50 tasks.
+</p>
+
+<h3 style="text-align: center; color: green;">A.3 Filtering Methodology</h3>
+<p>
+Filtering is based on:
+<ul>
+  <li><span style="color:red;">Language Adherence</span>: High Bengali word ratio, Unicode consistency, and grammar score ≥ 0.8.</li>
+  <li><span style="color:red;">Cultural Sensitivity</span>: Ensuring religious neutrality, regional inclusivity, gender balance, and political neutrality.</li>
+  <li><span style="color:red;">Content Quality</span>: Minimum length, coherence between instruction and response, factual accuracy, and proper formatting.</li>
+  <li><span style="color:red;">Novelty Verification</span>: Ensuring low similarity with existing tasks and sufficient lexical diversity.</li>
+</ul>
+A pair (i, r) is accepted only if all criteria are met.
+</p>
+
+<hr>
+
+<h2 style="text-align: center; color: green;">Appendix B: Experimentation Details</h2>
+
+<h3 style="text-align: center; color: green;">B.1 Experimental Setup</h3>
+<p>
+Pretraining was conducted on a Lambda Labs cluster with 8 NVIDIA A100 GPUs (40GB each), 512GB RAM, and 2TB storage (~120 hours with gradient checkpointing). Finetuning was performed on a single NVIDIA A100 GPU via Google Colab (~96 hours).
+</p>
+
+<h3 style="text-align: center; color: green;">B.2 Pretraining Hyperparameters (Table 3)</h3>
+<table>
+  <thead>
+    <tr>
+      <th style="color: green; text-align: center;">Hyperparameter</th>
+      <th style="color: green; text-align: center;">Value</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td>Per device train batch size</td>
+      <td>64</td>
+    </tr>
+    <tr>
+      <td>Gradient accumulation steps</td>
+      <td>16</td>
+    </tr>
+    <tr>
+      <td>Number of training epochs</td>
+      <td>4</td>
+    </tr>
+    <tr>
+      <td>Learning rate</td>
+      <td>5×10<sup>-6</sup></td>
+    </tr>
+    <tr>
+      <td>FP16</td>
+      <td>False</td>
+    </tr>
+    <tr>
+      <td>BF16</td>
+      <td>True</td>
+    </tr>
+    <tr>
+      <td>Dataloader num workers</td>
+      <td>8</td>
+    </tr>
+    <tr>
+      <td>Gradient checkpointing</td>
+      <td>True</td>
+    </tr>
+    <tr>
+      <td>Logging steps</td>
+      <td>1000</td>
+    </tr>
+    <tr>
+      <td>DDP find unused parameters</td>
+      <td>False</td>
+    </tr>
+    <tr>
+      <td>Max gradient norm</td>
+      <td>1.0</td>
+    </tr>
+    <tr>
+      <td>Warmup steps</td>
+      <td>1000</td>
+    </tr>
+    <tr>
+      <td>Evaluation strategy</td>
+      <td>steps</td>
+    </tr>
+    <tr>
+      <td>Evaluation steps</td>
+      <td>1,000</td>
+    </tr>
+    <tr>
+      <td>Save strategy</td>
+      <td>steps</td>
+    </tr>
+    <tr>
+      <td>Save steps</td>
+      <td>1,000</td>
+    </tr>
+    <tr>
+      <td>Save total limit</td>
+      <td>3</td>
+    </tr>
+    <tr>
+      <td>Load best model at end</td>
+      <td>True</td>
+    </tr>
+    <tr>
+      <td>Metric for best model loss</td>
+      <td>False</td>
+    </tr>
+  </tbody>
+</table>
+
+<h3 style="text-align: center; color: green;">B.3 Finetuning Hyperparameters</h3>
+<p>
+Finetuning settings for TigerLLM (1B) and (9B) are detailed in Tables 4 and 5.
+</p>
+
+<table>
+  <thead>
+    <tr>
+      <th style="color: green; text-align: center;">Parameter</th>
+      <th style="color: green; text-align: center;">TigerLLM (1B)</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td>Max Sequence Length</td>
+      <td>2048</td>
+    </tr>
+    <tr>
+      <td>Batch Size (Train/Eval)</td>
+      <td>16</td>
+    </tr>
+    <tr>
+      <td>Gradient Accumulation Steps</td>
+      <td>4</td>
+    </tr>
+    <tr>
+      <td>Number of Epochs</td>
+      <td>3</td>
+    </tr>
+    <tr>
+      <td>Learning Rate</td>
+      <td>1e-5</td>
+    </tr>
+    <tr>
+      <td>Weight Decay</td>
+      <td>0.02</td>
+    </tr>
+    <tr>
+      <td>Warmup Steps</td>
+      <td>10%</td>
+    </tr>
+    <tr>
+      <td>Optimizer</td>
+      <td>AdamW (8-bit)</td>
+    </tr>
+    <tr>
+      <td>LR Scheduler</td>
+      <td>Cosine</td>
+    </tr>
+    <tr>
+      <td>Precision</td>
+      <td>BF16</td>
+    </tr>
+    <tr>
+      <td>Evaluation Steps</td>
+      <td>50</td>
+    </tr>
+    <tr>
+      <td>Seed</td>
+      <td>42</td>
+    </tr>
+  </tbody>
+</table>
+
+<table>
+  <thead>
+    <tr>
+      <th style="color: green; text-align: center;">Parameter</th>
+      <th style="color: green; text-align: center;">TigerLLM (9B)</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td>Max Sequence Length</td>
+      <td>2048</td>
+    </tr>
+    <tr>
+      <td>Batch Size (Train/Eval)</td>
+      <td>32</td>
+    </tr>
+    <tr>
+      <td>Gradient Accumulation Steps</td>
+      <td>8</td>
+    </tr>
+    <tr>
+      <td>Number of Epochs</td>
+      <td>3</td>
+    </tr>
+    <tr>
+      <td>Learning Rate</td>
+      <td>1e-6</td>
+    </tr>
+    <tr>
+      <td>Weight Decay</td>
+      <td>0.04</td>
+    </tr>
+    <tr>
+      <td>Warmup Steps</td>
+      <td>15%</td>
+    </tr>
+    <tr>
+      <td>Optimizer</td>
+      <td>AdamW (8-bit)</td>
+    </tr>
+    <tr>
+      <td>LR Scheduler</td>
+      <td>Cosine</td>
+    </tr>
+    <tr>
+      <td>Precision</td>
+      <td>BF16</td>
+    </tr>
+    <tr>
+      <td>Evaluation Steps</td>
+      <td>250</td>
+    </tr>
+    <tr>
+      <td>Seed</td>
+      <td>42</td>
+    </tr>
+  </tbody>
+</table>
+
+<hr>
+
+<h2 style="text-align: center; color: green;">Appendix C: TigerLLM - Training Pipeline</h2>
+<p>
+Figure 2 illustrates the multi-stage training pipeline for producing both TigerLLM (1B) and TigerLLM (9B). The process begins with pre-trained models (LLaMA 3.2 and Gemma-2), followed by continual pretraining on the Bangla-TextBook corpus and subsequent finetuning on the Bangla-Instruct dataset. Figures 3 and 4 depict the loss curves during the pretraining and finetuning stages respectively.
+</p>