Update README.md
Browse files
README.md
CHANGED
@@ -86,10 +86,23 @@ response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
|
|
86 |
|
87 |
1. Don't include a system prompt; instead, place all instructions directly in the user prompt.
|
88 |
2. We recommend using the following instruction for math questions: Please reason step by step, and put your final answer within \\boxed{}.
|
89 |
-
3. We recommend using the following instruction for code questions:
|
90 |
-
|
91 |
-
|
92 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
93 |
|
94 |
|
95 |
## Correspondence to
|
|
|
86 |
|
87 |
1. Don't include a system prompt; instead, place all instructions directly in the user prompt.
|
88 |
2. We recommend using the following instruction for math questions: Please reason step by step, and put your final answer within \\boxed{}.
|
89 |
+
3. We recommend using the following instruction for code questions:
|
90 |
+
```python
|
91 |
+
question = "" # code question
|
92 |
+
starter_code = "" # starter code function header
|
93 |
+
|
94 |
+
code_instruction_nostartercode = """Write Python code to solve the problem. Please place the solution code in the following format:\n```python\n# Your solution code here\n```"""
|
95 |
+
code_instruction_hasstartercode = """Please place the solution code in the following format:\n```python\n# Your solution code here\n```"""
|
96 |
+
if starter_code != "":
|
97 |
+
question += "\n\n" + "Solve the problem starting with the provided function header.\n\nFunction header:\n" + "```\n" + starter_code + "\n```"
|
98 |
+
question += "\n\n" + code_instruction_hasstartercode
|
99 |
+
else:
|
100 |
+
question += "\n\n" + code_instruction_nostartercode
|
101 |
+
|
102 |
+
final_prompt = "<|User|>" + question + "<|Assistant|><think>\n"
|
103 |
+
```
|
104 |
+
5. Our inference engine for evaluation is **vLLM==0.7.3** using top-p=0.95, temperature=0.6, max_tokens=32768.
|
105 |
+
6. We use [AceMath scorer](https://huggingface.co/nvidia/AceMath-7B-Instruct/blob/main/evaluation/grader.py) for math evaluation and [LiveCodeBench official script](https://github.com/LiveCodeBench/LiveCodeBench) for code evaluation.
|
106 |
|
107 |
|
108 |
## Correspondence to
|