ychenNLP commited on
Commit
c6233d7
·
verified ·
1 Parent(s): 3e0e632

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +17 -4
README.md CHANGED
@@ -86,10 +86,23 @@ response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
86
 
87
  1. Don't include a system prompt; instead, place all instructions directly in the user prompt.
88
  2. We recommend using the following instruction for math questions: Please reason step by step, and put your final answer within \\boxed{}.
89
- 3. We recommend using the following instruction for code questions: Write Python code to solve the problem. Please place the solution code in the following format:\\n
90
- \`\`\`python\\n
91
- \# Your solution code here\\n
92
- \`\`\`
 
 
 
 
 
 
 
 
 
 
 
 
 
93
 
94
 
95
  ## Correspondence to
 
86
 
87
  1. Don't include a system prompt; instead, place all instructions directly in the user prompt.
88
  2. We recommend using the following instruction for math questions: Please reason step by step, and put your final answer within \\boxed{}.
89
+ 3. We recommend using the following instruction for code questions:
90
+ ```python
91
+ question = "" # code question
92
+ starter_code = "" # starter code function header
93
+
94
+ code_instruction_nostartercode = """Write Python code to solve the problem. Please place the solution code in the following format:\n```python\n# Your solution code here\n```"""
95
+ code_instruction_hasstartercode = """Please place the solution code in the following format:\n```python\n# Your solution code here\n```"""
96
+ if starter_code != "":
97
+ question += "\n\n" + "Solve the problem starting with the provided function header.\n\nFunction header:\n" + "```\n" + starter_code + "\n```"
98
+ question += "\n\n" + code_instruction_hasstartercode
99
+ else:
100
+ question += "\n\n" + code_instruction_nostartercode
101
+
102
+ final_prompt = "<|User|>" + question + "<|Assistant|><think>\n"
103
+ ```
104
+ 5. Our inference engine for evaluation is **vLLM==0.7.3** using top-p=0.95, temperature=0.6, max_tokens=32768.
105
+ 6. We use [AceMath scorer](https://huggingface.co/nvidia/AceMath-7B-Instruct/blob/main/evaluation/grader.py) for math evaluation and [LiveCodeBench official script](https://github.com/LiveCodeBench/LiveCodeBench) for code evaluation.
106
 
107
 
108
  ## Correspondence to