jinja template for story writing
I'm using aphrodite-engine, and that's the format it accepts. I'm pretty new to this stuff. Does that format provide for everything you need to prompt it properly? (I noticed you said that SillyTavern can't quite do it).
I rent GPU time with e.g. vast.ai. Which backend do you suggest I use, along with which software?
Thanks.
Hey there, did you checkout the guide and code for formatting? That should give you and idea of what the template should look like:
- Opus V1 prompting guide with many (interactive) examples and prompts that you can copy.
- Google Colab for interactive role-play using
opus-v1.2-7b. - Python code to format the prompt correctly.
I am not familiar with Aphrodite engine, and did not find documentation, but I found this:
https://github.com/PygmalionAI/aphrodite-engine/blob/main/examples/chatml_template.jinja
You can adapt the ChatML template for story-writing with Opus by changing the "assistant" role to "text" role. I am not sure if Aphrodite supports name interpolation in the Jinja template, which you would need for proper role-playing support. If it does, it's easy to do add to the template, just follow the examples from the guide or the code I shared.
I do something like this here: https://huggingface.co/dreamgen/opus-v1.2-7b/blob/main/tokenizer_config.json#L51 -- I change the assstant role to text in the HF tokenizer chat template.
Note that you still need a system prompt etc.
Aah, thanks. The changing the "assistant" role to "text" is basically what I was looking for, I think. Aphrodite-engine is a fork of vllm by the pygmalionai people which adds a lot of features that enthusiasts like.