Saxo commited on
Commit
bbeef6a
·
verified ·
1 Parent(s): 996a02f

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +52 -49
README.md CHANGED
@@ -1,59 +1,62 @@
1
  ---
2
- base_model:
3
- - byroneverson/gemma-2-27b-it-abliterated
4
- - Saxo/Linkbricks-Horizon-AI-Korean-Superb-27B
5
- - google/gemma-2-27b-it
6
- - nbeerbower/gemma2-gutenberg-27B
7
  library_name: transformers
8
- tags:
9
- - mergekit
10
- - merge
11
-
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
  ---
13
- # Linkbricks-Horizon-AI-Korean-Avengers-V2-27B
14
-
15
- This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
16
 
17
- ## Merge Details
18
- ### Merge Method
19
 
20
- This model was merged using the [DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708) merge method using [google/gemma-2-27b-it](https://huggingface.co/google/gemma-2-27b-it) as a base.
 
 
21
 
22
- ### Models Merged
23
 
24
- The following models were included in the merge:
25
- * [byroneverson/gemma-2-27b-it-abliterated](https://huggingface.co/byroneverson/gemma-2-27b-it-abliterated)
26
- * [Saxo/Linkbricks-Horizon-AI-Korean-Superb-27B](https://huggingface.co/Saxo/Linkbricks-Horizon-AI-Korean-Superb-27B)
27
- * [nbeerbower/gemma2-gutenberg-27B](https://huggingface.co/nbeerbower/gemma2-gutenberg-27B)
 
 
 
 
 
28
 
29
- ### Configuration
 
 
 
 
 
 
 
30
 
31
- The following YAML configuration was used to produce this model:
32
 
33
- ```yaml
34
- base_model: google/gemma-2-27b-it
35
- dtype: bfloat16
36
- merge_method: dare_ties
37
- parameters:
38
- int8_mask: 1.0
39
- slices:
40
- - sources:
41
- - layer_range: [0, 46]
42
- model: google/gemma-2-27b-it
43
- - layer_range: [0, 46]
44
- model: byroneverson/gemma-2-27b-it-abliterated
45
- parameters:
46
- density: 0.53
47
- weight: 0.3
48
- - layer_range: [0, 46]
49
- model: nbeerbower/gemma2-gutenberg-27B
50
- parameters:
51
- density: 0.53
52
- weight: 0.4
53
- - layer_range: [0, 46]
54
- model: Saxo/Linkbricks-Horizon-AI-Korean-Superb-27B
55
- parameters:
56
- density: 0.53
57
- weight: 0.3
58
- tokenizer_source: union
59
- ```
 
1
  ---
 
 
 
 
 
2
  library_name: transformers
3
+ license: apache-2.0
4
+ base_model: Saxo/Linkbricks-Horizon-AI-Korean-Avengers-V1-27B
5
+ datasets:
6
+ - Saxo/ko_cn_translation_tech_social_science_linkbricks_single_dataset
7
+ - Saxo/ko_jp_translation_tech_social_science_linkbricks_single_dataset
8
+ - Saxo/en_ko_translation_tech_science_linkbricks_single_dataset_with_prompt_text_huggingface
9
+ - Saxo/en_ko_translation_social_science_linkbricks_single_dataset_with_prompt_text_huggingface
10
+ - Saxo/ko_aspect_sentiment_sns_mall_sentiment_linkbricks_single_dataset_with_prompt_text_huggingface
11
+ - Saxo/ko_summarization_linkbricks_single_dataset_with_prompt_text_huggingface
12
+ - Saxo/OpenOrca_cleaned_kor_linkbricks_single_dataset_with_prompt_text_huggingface
13
+ - Saxo/ko_government_qa_total_linkbricks_single_dataset_with_prompt_text_huggingface_sampled
14
+ - Saxo/ko-news-corpus-1
15
+ - Saxo/ko-news-corpus-2
16
+ - Saxo/ko-news-corpus-3
17
+ - Saxo/ko-news-corpus-4
18
+ - Saxo/ko-news-corpus-5
19
+ - Saxo/ko-news-corpus-6
20
+ - Saxo/ko-news-corpus-7
21
+ - Saxo/ko-news-corpus-8
22
+ - Saxo/ko-news-corpus-9
23
+ - maywell/ko_Ultrafeedback_binarized
24
+ - youjunhyeok/ko-orca-pair-and-ultrafeedback-dpo
25
+ - lilacai/glaive-function-calling-v2-sharegpt
26
+ - kuotient/gsm8k-ko
27
+ language:
28
+ - ko
29
+ - en
30
+ - jp
31
+ - cn
32
+ pipeline_tag: text-generation
33
  ---
 
 
 
34
 
35
+ # Model Card for Model ID
 
36
 
37
+ <div align="center">
38
+ <img src="http://www.linkbricks.com/wp-content/uploads/2024/11/fulllogo.png" />
39
+ </div>
40
 
 
41
 
42
+ AI 빅데이터 분석 전문 기업인 Linkbricks의 데이터사이언티스트인 지윤성(Saxo) 박사가 <br>
43
+ Saxo/Linkbricks-Horizon-AI-Korean-Avengers-V1-27B 베이스모델을 사용해서 H100-80G 8개를 통해 약 38%정도의 파라미터를 한국어 SFT->DPO 한 한글 언어 모델<br>
44
+ 5천만건의 한글 뉴스 코퍼스를 기준으로 다양한 테스크별 한국어-중국어-영어-일본어 교차 학습 데이터와 수학 및 논리판단 데이터를 통하여 한중일영 언어 교차 증강 처리와 복잡한 논리 문제 역시 대응 가능하도록 훈련한 모델이다.<br>
45
+ -토크나이저는 단어 확장 없이 베이스 모델 그대로 사용<br>
46
+ -고객 리뷰나 소셜 포스팅 고차원 분석 및 코딩과 작문, 수학, 논리판단 등이 강화된 모델<br>
47
+ -128k-Context Window<br>
48
+ -Deepspeed Stage=3, rslora 및 BAdam Layer Mode 사용 <br>
49
+ "transformers_version": "4.46.1"
50
+ <br><br>
51
 
52
+ Finetuned by Mr. Yunsung Ji (Saxo), a data scientist at Linkbricks, a company specializing in AI and big data analytics <br>
53
+ about 38% of total parameters Korean SFT->DPO training model based on Saxo/Linkbricks-Horizon-AI-Korean-Avengers-V1-27B through 8 H100-80Gs as a Korean language model <br>
54
+ It is a model that has been trained to handle Korean-Chinese-English-Japanese cross-training data and 50M korean news corpus and logic judgment data for various tasks to enable cross-fertilization processing and complex Korean logic & math problems. <br>
55
+ -Tokenizer uses the base model without word expansion<br>
56
+ -Models enhanced with high-dimensional analysis of customer reviews and social posts, as well as coding, writing, math and decision making<br>
57
+ -128k-Context Window<br>
58
+ -Deepspeed Stage=3, use rslora and BAdam Layer Mode<br>
59
+ <br><br>
60
 
 
61
 
62
+ <a href="www.linkbricks.com">www.linkbricks.com</a>, <a href="www.linkbricks.vc">www.linkbricks.vc</a>