How to load the model?
Dear authors,
I am currently working with the "Senqiao/LISA_Plus_7b" model and have encountered an issue while loading it using the following code:
model = LISAForCausalLM.from_pretrained(
pretrained_model_name_or_path="Senqiao/LISA_Plus_7b",
low_cpu_mem_usage=True,
seg_token_idx=32000
)
When I run this code, I receive a warning indicating that some weights of LISAForCausalLM were not initialized from the model checkpoint and are newly initialized: ['model.mm_projector.bias', 'model.mm_projector.weight'].
Could you please provide guidance on the correct way to load this model to avoid such discrepancies? Your assistance would be greatly appreciated. Thank you for your time!
Oh! The new version LISA++ is based on LLaVA-1.5, whereas the current version of LISA code is based on LLaVA. I am sorry to confuse you. Let me see how to update it.
@Senqiao Any update???