Kaito Sugimoto's picture

Kaito Sugimoto

kaisugi

·

https://kaisugi.me

kaisugi

AI & ML interests

Japanese LLMs

Recent Activity

reacted to lianghsun's post with 👍 2 days ago

🖖 Let me introduce the work I've done over the past three months: 𝗟𝗹𝗮𝗺𝗮-𝟯.𝟮-𝗧𝗮𝗶𝘄𝗮𝗻-𝟯𝗕 and 𝗟𝗹𝗮𝗺𝗮-𝟯.𝟮-𝗧𝗮𝗶𝘄𝗮𝗻-𝟯𝗕-𝗜𝗻𝘀𝘁𝗿𝘂𝗰𝘁, now open-sourced on 🤗 Hugging Face. 𝗹𝗶𝗮𝗻𝗴𝗵𝘀𝘂𝗻/𝗟𝗹𝗮𝗺𝗮-𝟯.𝟮-𝗧𝗮𝗶𝘄𝗮𝗻-𝟯𝗕: This model is built on top of 𝗺𝗲𝘁𝗮-𝗹𝗹𝗮𝗺𝗮/𝗟𝗹𝗮𝗺𝗮-𝟯.𝟮-𝟯𝗕 with continual pretraining. The training dataset consists of a mixture of Traditional Chinese and multilingual texts in specific proportions, including 20B tokens of Traditional Chinese text. 𝗹𝗶𝗮𝗻𝗴𝗵𝘀𝘂𝗻/𝗟𝗹𝗮𝗺𝗮-𝟯.𝟮-𝗧𝗮𝗶𝘄𝗮𝗻-𝟯𝗕-𝗜𝗻𝘀𝘁𝗿𝘂𝗰𝘁: This is a fine-tuned conversational model based on the foundation model. This Llama-3.2-Taiwan open-source project is currently a one-person effort (yes, I did everything from text preparation — so exhausting!). If you're interested, feel free to join the Discord server for discussions. 🅱🅴🅽🅲🅷🅼🅰🆁🅺🅸🅽🅶 The evaluation was conducted using https://huggingface.co/datasets/ikala/tmmluplus, though the README page does not yet reflect the latest results. The performance is close to the previous versions, indicating that further improvements might require adding more specialized knowledge in the datasets. 🅰 🅲🅰🅻🅻 🅵🅾🆁 🆂🆄🅿🅿🅾🆁🆃 If anyone is willing to provide compute resources, it would be greatly appreciated to help this project continue and grow. 💪 --- 🏔️ Foundation model: https://huggingface.co/lianghsun/Llama-3.2-Taiwan-3B 🤖 Instruction model: https://huggingface.co/lianghsun/Llama-3.2-Taiwan-3B-Instruct ⚡ GGUF: https://huggingface.co/lianghsun/Llama-3.2-Taiwan-3B-Instruct-GGUF

liked a model 4 days ago

LoneWolfgang/bert-for-japanese-twitter

liked a model 4 days ago

KoichiYasuoka/modernbert-base-japanese-wikipedia

View all activity

Organizations

kaisugi's activity

upvoted 2 articles 19 days ago

Article

Navigating Korean LLM Research #2: Evaluation Tools

By

•

Oct 23, 2024

• 7

Article

Navigating Korean LLM Research #1: Models

By

•

Oct 22, 2024

• 23

upvoted 2 collections 2 months ago

LLM-jp-3 Fine-tuned Models

Fine-tuned models in the LLM-jp-3 model series • 5 items • Updated 11 days ago • 2

LLM-jp-3 Pre-trained Models

Pre-trained models in the LLM-jp-3 model series • 4 items • Updated 25 days ago • 5

upvoted an article 3 months ago

Article

How to generate text: using different decoding methods for language generation with Transformers

Mar 1, 2020

• 140

upvoted a collection 3 months ago

Llama-3.1-Swallow

8 items • Updated 22 days ago • 5

upvoted a paper 3 months ago

PLaMo-100B: A Ground-Up Language Model Designed for Japanese Proficiency

Paper • 2410.07563 • Published Oct 10, 2024 • 2

upvoted 3 collections 4 months ago

gemma-2-baku

The baku model series are based on the gemma-2 series and have been continually pre-trained on Japanese-specific corpora. • 4 items • Updated Dec 5, 2024 • 3

Gemma 2 JPN Release

A Gemma 2 2B model fine-tuned on Japanese text. It supports the Japanese language the same level of performance of EN only queries on Gemma 2. • 3 items • Updated Dec 13, 2024 • 27

Borea

3 items • Updated Aug 21, 2024 • 2

upvoted a paper 4 months ago

Ruri: Japanese General Text Embeddings

Paper • 2409.07737 • Published Sep 12, 2024 • 7

upvoted 3 collections 5 months ago

Ruri: Japanese General Text Embeddings

18 items • Updated Sep 13, 2024 • 13

Japanese SimCSE

Tsukagoshi et al., Japanese SimCSE Technical Report, arXiv 2023. https://arxiv.org/abs/2310.19349 • 5 items • Updated Sep 4, 2024 • 2

Japanese Retrieval

3 items • Updated Aug 20, 2024 • 3

upvoted 4 collections 6 months ago

llama-3-youko

The youko model series are based on the llama-3 series and have been continually pre-trained on Japanese-specific corpora. • 9 items • Updated Dec 5, 2024 • 1

EZO

18 items • Updated Oct 3, 2024 • 2

Llama 3.1 GPTQ, AWQ, and BNB Quants

Optimised Quants for high-throughput deployments! Compatible with Transformers, TGI & VLLM 🤗 • 9 items • Updated Sep 26, 2024 • 56

Sarashina

Large Language Models developed by SB Intuitions • 8 items • Updated Dec 12, 2024 • 5

upvoted a paper 6 months ago

LLM-jp: A Cross-organizational Project for the Research and Development of Fully Open Japanese LLMs

Paper • 2407.03963 • Published Jul 4, 2024 • 16

upvoted a collection 7 months ago

DeBERTa V3

6 items • Updated Jul 5, 2024 • 1