Model Card for Model ID


Open Ko LLM Leaderboard Season 2 ๐Ÿ† Rank-2 2024/12/28~


AI ์ „๋ฌธ ๊ธฐ์—…์ธ Linkbricks Horizon-AI ์˜ ๋ฐ์ดํ„ฐ์‚ฌ์ด์–ธํ‹ฐ์ŠคํŠธ์ธ ์ง€์œค์„ฑ(Saxo) ๋Œ€ํ‘œ๊ฐ€
gemma-2-27b-it ๋ฒ ์ด์Šค๋ชจ๋ธ์„ ์‚ฌ์šฉํ•ด์„œ H100-80G 8๊ฐœ๋ฅผ ํ†ตํ•ด ์•ฝ 15%์ •๋„์˜ ํŒŒ๋ผ๋ฏธํ„ฐ๋ฅผ ํ•œ๊ตญ์–ด CPT(Continued-Pretraining)->SFT->DPO ํ•œ ํ•œ๊ธ€ ์–ธ์–ด ๋ชจ๋ธ
์ฒœ๋งŒ๊ฑด์˜ ํ•œ๊ธ€ ๋‰ด์Šค ์ฝ”ํผ์Šค๋ฅผ ๊ธฐ์ค€์œผ๋กœ ๋‹ค์–‘ํ•œ ํ…Œ์Šคํฌ๋ณ„ ํ•œ๊ตญ์–ด-์ค‘๊ตญ์–ด-์˜์–ด-์ผ๋ณธ์–ด ๊ต์ฐจ ํ•™์Šต ๋ฐ์ดํ„ฐ์™€ ์ˆ˜ํ•™ ๋ฐ ๋…ผ๋ฆฌํŒ๋‹จ ๋ฐ์ดํ„ฐ๋ฅผ ํ†ตํ•˜์—ฌ ํ•œ์ค‘์ผ์˜ ์–ธ์–ด ๊ต์ฐจ ์ฆ๊ฐ• ์ฒ˜๋ฆฌ์™€ ๋ณต์žกํ•œ ๋…ผ๋ฆฌ ๋ฌธ์ œ ์—ญ์‹œ ๋Œ€์‘ ๊ฐ€๋Šฅํ•˜๋„๋ก ํ›ˆ๋ จํ•œ ๋ชจ๋ธ์ด๋‹ค.
-ํ† ํฌ๋‚˜์ด์ €๋Š” ๋‹จ์–ด ํ™•์žฅ ์—†์ด ๋ฒ ์ด์Šค ๋ชจ๋ธ ๊ทธ๋Œ€๋กœ ์‚ฌ์šฉ
-๊ณ ๊ฐ ๋ฆฌ๋ทฐ๋‚˜ ์†Œ์…œ ํฌ์ŠคํŒ… ๊ณ ์ฐจ์› ๋ถ„์„ ๋ฐ ์ฝ”๋”ฉ๊ณผ ์ž‘๋ฌธ, ์ˆ˜ํ•™, ๋…ผ๋ฆฌํŒ๋‹จ ๋“ฑ์ด ๊ฐ•ํ™”๋œ ๋ชจ๋ธ
-Deepspeed Stage=3, rslora ๋ฐ BAdam Layer Mode ์‚ฌ์šฉ
-ollama run benedict/linkbricks-gemma2-27b-korean-advanced-q4
-ollama run benedict/linkbricks-gemma2-27b-korean-advanced-q8

Finetuned by CEO Yunsung Ji (Saxo), a data scientist at Linkbricks Horizon-AI, a company specializing in AI and big data analytics
about 15% of total parameters Korean CPT(Continued-Pretraining)->SFT->DPO training model based on gemma-2-27b-it through 8 H100-80Gs as a Korean language model
It is a model that has been trained to handle Korean-Chinese-English-Japanese cross-training data and 10M korean news corpus and logic judgment data for various tasks to enable cross-fertilization processing and complex Korean logic & math problems.
-Tokenizer uses the base model without word expansion
-Models enhanced with high-dimensional analysis of customer reviews and social posts, as well as coding, writing, math and decision making
-Deepspeed Stage=3, use rslora and BAdam Layer Mode


www.linkbricks.com, www.linkbricks.vc

Downloads last month
1,787
Safetensors
Model size
27.2B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for Saxo/Linkbricks-Horizon-AI-Korean-Advanced-27B

Base model

google/gemma-2-27b
Quantized
(61)
this model
Finetunes
1 model

Datasets used to train Saxo/Linkbricks-Horizon-AI-Korean-Advanced-27B