What could be instruction fine tuning prompt for this model?
Hello, It is so impressive to hit 70B performance with only 10 times smaller model.
I want to try instruction fine tuning However I got no idea about prompt.
Where could instruction go in the prompt? (system message + instruction)
Thank you!
Thank you! The prompt for fine-tuning will be the same as the one used for chat, which is provided in model card and also below:
"GPT4 Correct User: Hello<|end_of_turn|>GPT4 Correct Assistant: Hi<|end_of_turn|>GPT4 Correct User: How are you today?<|end_of_turn|>GPT4 Correct Assistant:"
During both SFT and RLHF, the model does not follow system message. So I would suggest put all system message together with instruction inside "GPT4 Correct User:".
Really quick answer for me :)
I would try with pleasure.
By the way, why the model could not follow system message? is there any reason?
Thank you again!
It's because we mostly trained on the prompt style that does not come with system message (usually the fine-tuning dataset won't come with prompts with standard system message). But if you just put system message inside "GPT4 Correct User:" and prompt it appropriately, it shall be able to follow it well.
Thanks again! I have one more question.
Does "generation_config.json" file affect the training?
Because some other models in the hub contain other parameters such as temperature,do_sample,...
Does it affect on training? If it does, then can i adjust that parameters also?
Sorry for the late reply. I think it won't affect training. If you want to use RLHF then the generation parameters might affect the samples you collected. But for SFT it won't affect anything.