Files changed (1) hide show
  1. README.md +59 -45
README.md CHANGED
@@ -1,46 +1,60 @@
1
- ---
2
- license: apache-2.0
3
- base_model:
4
- - Qwen/Qwen2.5-7B-Instruct
5
- ---
6
- # Converted LLaMA from QWEN2-7B-Instruct
7
-
8
- ## Descritpion
9
- This is a converted model from [Qwen2-7B-Instruct](https://huggingface.co/Qwen/Qwen2-7B-Instruct) to __LLaMA__ format. This conversion allows you to use Qwen2-7B-Instruct as if it were a LLaMA model, which is convenient for some *inference use cases*. The __precision__ is __excatly the same__ as the original model.
10
-
11
- ## Usage
12
- You can load the model using the `LlamaForCausalLM` class as shown below:
13
- ```python
14
- from transformers import AutoTokenizer, LlamaForCausalLM
15
-
16
- prompt = "Give me a short introduction to large language model."
17
- messages = [
18
- {"role": "system", "content": "You are a helpful assistant."},
19
- {"role": "user", "content": prompt}
20
- ]
21
- # we still use the original tokenizer from Qwen2-7B-Instruct
22
- tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2-7B-Instruct")
23
- text = tokenizer.apply_chat_template(
24
- messages,
25
- tokenize=False,
26
- add_generation_prompt=True
27
- )
28
- model_inputs = tokenizer([text],return_tensors="pt").cuda()
29
-
30
- # Converted LlaMA model
31
- llama_model = LlamaForCausalLM.from_pretrained(
32
- "silence09/Qwen2-7B-Instruct-Converted-Llama",
33
- torch_dtype='auto').cuda()
34
- llama_generated_ids = llama_model.generate(model_inputs.input_ids, max_new_tokens=32, do_sample=False)
35
- llama_generated_ids = [
36
- output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, llama_generated_ids)
37
- ]
38
- llama_response = tokenizer.batch_decode(llama_generated_ids, skip_special_tokens=True)[0]
39
- print(llama_response)
40
- ```
41
-
42
- ## Precision Guarantee
43
- To comare result with the original model, you can use this [code](https://github.com/silencelamb/naked_llama/blob/main/hf_example/hf_qwen2_7b.py)
44
-
45
- ## More Info
 
 
 
 
 
 
 
 
 
 
 
 
 
 
46
  It was converted using the python script available at [this repository](https://github.com/silencelamb/naked_llama/blob/main/hf_example/convert_qwen_to_llama_hf.py)
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model:
4
+ - Qwen/Qwen2.5-7B-Instruct
5
+ language:
6
+ - zho
7
+ - eng
8
+ - fra
9
+ - spa
10
+ - por
11
+ - deu
12
+ - ita
13
+ - rus
14
+ - jpn
15
+ - kor
16
+ - vie
17
+ - tha
18
+ - ara
19
+ ---
20
+ # Converted LLaMA from QWEN2-7B-Instruct
21
+
22
+ ## Descritpion
23
+ This is a converted model from [Qwen2-7B-Instruct](https://huggingface.co/Qwen/Qwen2-7B-Instruct) to __LLaMA__ format. This conversion allows you to use Qwen2-7B-Instruct as if it were a LLaMA model, which is convenient for some *inference use cases*. The __precision__ is __excatly the same__ as the original model.
24
+
25
+ ## Usage
26
+ You can load the model using the `LlamaForCausalLM` class as shown below:
27
+ ```python
28
+ from transformers import AutoTokenizer, LlamaForCausalLM
29
+
30
+ prompt = "Give me a short introduction to large language model."
31
+ messages = [
32
+ {"role": "system", "content": "You are a helpful assistant."},
33
+ {"role": "user", "content": prompt}
34
+ ]
35
+ # we still use the original tokenizer from Qwen2-7B-Instruct
36
+ tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2-7B-Instruct")
37
+ text = tokenizer.apply_chat_template(
38
+ messages,
39
+ tokenize=False,
40
+ add_generation_prompt=True
41
+ )
42
+ model_inputs = tokenizer([text],return_tensors="pt").cuda()
43
+
44
+ # Converted LlaMA model
45
+ llama_model = LlamaForCausalLM.from_pretrained(
46
+ "silence09/Qwen2-7B-Instruct-Converted-Llama",
47
+ torch_dtype='auto').cuda()
48
+ llama_generated_ids = llama_model.generate(model_inputs.input_ids, max_new_tokens=32, do_sample=False)
49
+ llama_generated_ids = [
50
+ output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, llama_generated_ids)
51
+ ]
52
+ llama_response = tokenizer.batch_decode(llama_generated_ids, skip_special_tokens=True)[0]
53
+ print(llama_response)
54
+ ```
55
+
56
+ ## Precision Guarantee
57
+ To comare result with the original model, you can use this [code](https://github.com/silencelamb/naked_llama/blob/main/hf_example/hf_qwen2_7b.py)
58
+
59
+ ## More Info
60
  It was converted using the python script available at [this repository](https://github.com/silencelamb/naked_llama/blob/main/hf_example/convert_qwen_to_llama_hf.py)