This is the LoRA on llava-hf/llama3-llava-next-8b-hf, simple vision version of reproduce Mol-Instruct model, under the same dataset with full size text dataset, this is a text alignment reproduce version.
2d77474
verified
| { | |
| "chat_template": "{% for message in messages %}{{ '<|start_header_id|>' + message['role'] + '<|end_header_id|>\n\n'}}{# Render all images first #}{% for content in message['content'] | selectattr('type', 'equalto', 'image') %}{{ '<image>' }}{% endfor %}{# Render all text next #}{% for content in message['content'] | selectattr('type', 'equalto', 'text') %}{{ '\n' + content['text'] + '<|eot_id|>' }}{% endfor %}{% endfor %}{% if add_generation_prompt %}{{ '<|start_header_id|>assistant<|end_header_id|>\n\n' }}{% endif %}" | |
| } | |