README.md

---
base_model: llm-jp/llm-jp-3-13b
tags:
- text-generation-inference
- transformers
- unsloth
- llama
- trl
license: apache-2.0
language:
- ja
datasets:
- kinokokoro/ichikara-instruction-003
---

# Uploaded  model

- **Developed by:** trikudayodayodayo
- **License:** apache-2.0
- **Finetuned from model :** llm-jp/llm-jp-3-13b

# Overview
This repository provides a Japanese Large Language Model finetuned on ichikara datasets

# supervised-fintuning
Thme model was finetuned on a subset from mxture of the following dataset.
Training epoch:1
- ichikara-instruction-003-001-1
- ichikara-instruction-003-001-2
- ichikara-instruction-003-001-2.2
- ichikara-instruction-003-003-5.1
- ichikara-instruction-003-003-5.2
- ichikara-instruction-003-002-1
- ichikara-instruction-003-003-1

Authors
tsuchida rikuto

How to Use
To use this model, run the code below
```python
!pip install -U bitsandbytes
!pip install -U transformers
!pip install -U accelerate
!pip install -U datasets

!pip install ipywidgets --upgrade

from transformers import (
    AutoModelForCausalLM,
    AutoTokenizer,
    BitsAndBytesConfig,
)
import torch
from tqdm import tqdm
import json


model_name = "trikudayodayodayo/llm-jp-3-13b-it-1209_lora"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=False,
)

HF_TOKEN="Type your HF_TOKEN"

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    quantization_config=bnb_config,
    device_map="auto",
    token = HF_TOKEN
)

tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True, token = HF_TOKEN)

input = "Type text here"

tokenized_input = tokenizer.encode(input, add_special_tokens=False, return_tensors="pt").to(model.device)
with torch.no_grad():
    outputs = model.generate(
        tokenized_input,
        max_new_tokens=100,
        do_sample=False,
        repetition_penalty=1.2
    )[0]

output = tokenizer.decode(outputs[tokenized_input.size(1):], skip_special_tokens=True)

print(output)
```