Create README.md

README.md

@@ -0,0 +1,154 @@
+---
+license: gemma
+language:
+- th
+- en
+base_model:
+- google/gemma-3-27b-pt
+pipeline_tag: text-generation
+---
+
+**Authors**: NECTEC
+
+### Description
+
+This model, **nectec/thai-research-gemma-3-27b-it**, also known as **Pathumma LLM AI**, is a fine-tuned version of Google's Gemma 3 27B model, specialized for the Thai language. It was developed by the National Electronics and Computer Technology Center (NECTEC) of Thailand.
+
+Starting with the `google/gemma-3-27b-pt` checkpoint, the model underwent continued pre-training on a diverse corpus of approximately 8 billion Thai tokens. Following this, it was instruction fine-tuned on 3,052,736 high-quality Thai question-answer pairs to enhance its ability to follow instructions and engage in conversation.
+
+### Inputs and outputs
+
+-   **Input:**
+    -  Text string, such as a question, a prompt, or a document to be summarized, primarily in Thai.
+    -  Total input context of 128K tokens
+
+-   **Output:**
+    -   Generated text in response to the input, such as an answer to a
+        question, analysis of image content, or a summary of a document, primarily in Thai.
+    -   Total output context of 8192 tokens
+
+### Usage
+
+Below there are some code snippets on how to get quickly started with running the model. First, install the necessary libraries.
+
+#### Running with `transformers` on a GPU
+
+```sh
+$ pip install -U transformers accelerate torch
+```
+
+You can use the model with the `transformers` library as follows. The `EOSLogitsBiasProcessor` can be helpful if the model has trouble generating the end-of-sequence token.
+
+```python
+from transformers import AutoTokenizer, LogitsProcessor, LogitsProcessorList, AutoModelForImageTextToText
+import torch
+
+model_id = "nectec/thai-research-gemma-3-27b-it"
+
+model = AutoModelForImageTextToText.from_pretrained(
+    model_id, device_map="auto", torch_dtype=torch.bfloat16
+).eval()
+
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+
+class EOSLogitsBiasProcessor(LogitsProcessor):
+    def __init__(self,tokenizer, eos_token_id, bias_value=5.0, space_bias = -0.3):
+        self.eos_token_id = eos_token_id
+        self.bias_value = bias_value
+        self.space_bias = space_bias
+    def __call__(self, input_ids, scores):
+        scores[:, self.eos_token_id] += self.bias_value
+        scores[:,107] += 0.3
+        return scores
+
+logits_processor = LogitsProcessorList([
+    EOSLogitsBiasProcessor(tokenizer,eos_token_id=tokenizer.eos_token_id, bias_value=10.0)
+])
+
+messages = [
+    {
+        "role": "system",
+        "content": [{"type": "text", "text": "You are a helpful Thai assistant. You are Pathumma LLM AI that build by National Electronics and Computer Technology Center (NECTEC)."}]
+    },
+    {
+        "role": "user",
+        "content": [
+            {"type": "text", "text": "ขอสูตรส้มตำหน่อย"}
+        ]
+    },
+]
+
+inputs = tokenizer.apply_chat_template(
+    messages, add_generation_prompt=True, tokenize=True,
+    return_dict=True, return_tensors="pt"
+).to(model.device)
+
+input_len = inputs["input_ids"].shape[-1]
+
+with torch.inference_mode():
+    generation = model.generate(
+        **inputs,
+        max_new_tokens=512,
+        do_sample=True,
+        num_beams=2,
+        repetition_penalty=1.1,
+        temperature=0.4,
+        logits_processor=logits_processor,  # add if it have a problem not generate eos token
+    )
+
+generated_text = tokenizer.decode(generation[0][input_len:], skip_special_tokens=True)
+print(generated_text)
+```
+
+#### Running with `llama.cpp`
+
+You can also run a quantized version of the model using `llama-cpp-python` for efficient inference on CPU or GPU.
+
+```sh
+%pip install llama-cpp-python transformers
+```
+```python
+import transformers
+from llama_cpp import Llama
+
+# Download the GGUF model file
+!wget -O './thai-research-gemma-3-27b-it-Q4_K_M.gguf' "https://huggingface.co/nectec/thai-research-gemma-3-27b-it/resolve/main/thai-research-gemma-3-27b-it-Q4_K_M.gguf?download=true"
+
+PROMPT = "You are a helpful Thai assistant. You are Pathumma LLM AI that build by National Electronics and Computer Technology Center (NECTEC)."
+
+llm = Llama(model_path='thai-research-gemma-3-27b-it-Q4_K_M.gguf', n_gpu_layers=-1, n_ctx=4096,verbose=False)
+tokenizer = transformers.AutoTokenizer.from_pretrained("nectec/thai-research-gemma-3-27b-it")
+
+memory = [{'content': PROMPT, 'role': 'system'},]
+def generate(instuction,memory=memory):
+    memory.append({'content': instuction, 'role': 'user'})
+    p = tokenizer.apply_chat_template(
+        memory,
+        tokenize=False,
+        add_generation_prompt=True
+    )
+    response = llm(
+        p,
+        max_tokens=4096,
+        temperature=0.4, # ความคิดสร้างสรรค์ถ้าใส่น้อยจะตอบตรงคำถาม ถ้าใส่เยอะสร้างสรรค์แต่อาจจะผิดหรือไม่ตรงประเด็น
+        repeat_penalty=1.1, # ป้องกันการตอบคำซ้ำ
+        stop=["<end_of_turn>"]
+    )
+    output = response['choices'][0]['text']
+    memory.append({'content': output, 'role': 'assistant'})
+    return output
+
+print(generate("ขอสูตรทำส้มตำ",memory=memory))
+```
+### Citation
+
+If you use this model, please cite the base Gemma model and acknowledge NECTEC's contribution.
+```none
+@article{gemma_2025,
+    title={Gemma 3},
+    url={https://goo.gle/Gemma3Report},
+    publisher={Kaggle},
+    author={Gemma Team},
+    year={2025}
+}
+```