togethercomputer

/

Llama-2-7B-32K-Instruct

like 159

Follow

Together 427

Hi folks,
Not sure if this is the best place to ask this question. Please let me know if there is somewhere else. I'll also probably try your support/contact link.

When I use the playground with this model, it works fine.

I then copy the curl command that the playground generates and run that locally and it returns an empty text string for the answer. Here is a sample response. I've tried adjusting the temperature, top_p, top_k, max_tokens, stream_tokens and the prompt that I'm asking.

{"id":"83df4b8d3f94c39c-SEA","status":"finished","prompt":["[INST]  What is rain?  [/INST]  "],"model":"togethercomputer/Llama-2-7B-32K-Instruct","model_owner":"","num_returns":1,"args":{"model":"togethercomputer/Llama-2-7B-32K-Instruct","prompt":"[INST]  What is rain?  [/INST]  ","request_type":"language-model-inference","temperature":0.1,"top_p":0.1,"top_k":3,"max_tokens":1000,"stream_tokens":false,"stop":["[INST]","\n\n"],"negative_prompt":"","sessionKey":"906e58d5-91b6-4ec9-990e-315b47ec5184","update_at":"2023-12-31T03:00:29.376Z"},"subjobs":[],"output":{"usage":{"prompt_tokens":15,"completion_tokens":2,"total_tokens":17},"result_type":"language-model-inference","choices":[{"text":""}]}}%

This also happens when using the javascript code that is generated.

The generated curl command is working correctly for all other models that I have tried.

Thanks for any guidance,
Jeremy