metadata
tags:
- deepseek
- commercial use
- function-calling
- function calling
extra_gated_prompt: Purchase access to this repo [HERE](https://buy.stripe.com/14k3cY4pL4PFbQY3dq)
Function Calling Fine-tuned DeepSeek Coder 33B
Purchase access to this model here.
Performance demo video here.
This model is fine-tuned for function calling.
- The function metadata format is the same as used for OpenAI.
- The model is suitable for commercial use.
- AWQ and GGUF are available on request after purchase.
Check out other fine-tuned function calling models here.
Quick Server Setup
Runpod one click templates: [You must add a HuggingFace Hub access token (HUGGING_FACE_HUB_TOKEN) to the environment variables as this is a gated model.]
Runpod Affiliate Link (helps support the Trelis channel).
Inference Scripts
See below for sample prompt format.
Complete inference scripts are available for purchase here:
- Easily format prompts using tokenizer.apply_chat_format (starting from openai formatted functions and a list of messages)
- Automate catching, handling and chaining of function calls.
Prompt Format
B_FUNC, E_FUNC = "You have access to the following functions. Use them if required:\n\n", "\n\n"
B_INST, E_INST = "\n### Instruction:\n", "\n### Response:\n" #DeepSeek Coder Style
prompt = f"{B_INST}{B_FUNC}{functionList.strip()}{E_FUNC}{user_prompt.strip()}{E_INST}\n\n"
Using tokenizer.apply_chat_template
For an easier application of the prompt, you can set up as follows:
Set up messages:
[
{
"role": "function_metadata",
"content": "FUNCTION_METADATA"
},
{
"role": "user",
"content": "What is the current weather in London?"
},
{
"role": "function_call",
"content": "{\n \"name\": \"get_current_weather\",\n \"arguments\": {\n \"city\": \"London\"\n }\n}"
},
{
"role": "function_response",
"content": "{\n \"temperature\": \"15 C\",\n \"condition\": \"Cloudy\"\n}"
},
{
"role": "assistant",
"content": "The current weather in London is Cloudy with a temperature of 15 Celsius"
}
]
with FUNCTION_METADATA as:
[
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "This function gets the current weather in a given city",
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "The city, e.g., San Francisco"
},
"format": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "The temperature unit to use."
}
},
"required": ["city"]
}
}
},
{
"type": "function",
"function": {
"name": "get_clothes",
"description": "This function provides a suggestion of clothes to wear based on the current weather",
"parameters": {
"type": "object",
"properties": {
"temperature": {
"type": "string",
"description": "The temperature, e.g., 15 C or 59 F"
},
"condition": {
"type": "string",
"description": "The weather condition, e.g., 'Cloudy', 'Sunny', 'Rainy'"
}
},
"required": ["temperature", "condition"]
}
}
}
]
and then apply the chat template to get a formatted prompt:
tokenizer = AutoTokenizer.from_pretrained('Trelis/deepseek-coder-33b-instruct-function-calling-v3', trust_remote_code=True)
prompt = tokenizer.apply_chat_template(prompt, tokenize=False, add_generation_prompt=True)
If you are using a gated model, you need to first run:
pip install huggingface_hub
huggingface-cli login
Manual Prompt:
Human: You have access to the following functions. Use them if required:
[
{
"type": "function",
"function": {
"name": "get_stock_price",
"description": "Get the stock price of an array of stocks",
"parameters": {
"type": "object",
"properties": {
"names": {
"type": "array",
"items": {
"type": "string"
},
"description": "An array of stocks"
}
},
"required": [
"names"
]
}
}
},
{
"type": "function",
"function": {
"name": "get_big_stocks",
"description": "Get the names of the largest N stocks by market cap",
"parameters": {
"type": "object",
"properties": {
"number": {
"type": "integer",
"description": "The number of largest stocks to get the names of, e.g. 25"
},
"region": {
"type": "string",
"description": "The region to consider, can be \"US\" or \"World\"."
}
},
"required": [
"number"
]
}
}
}
]
Get the names of the five largest stocks by market cap Assistant:
{
"name": "get_big_stocks",
"arguments": {
"number": 5
}
}<|EOT|>```
# Dataset
See [Trelis/function_calling_v3](https://huggingface.co/datasets/Trelis/function_calling_v3).
# License
This model may be used commercially for inference according to the terms of the DeepSeek license, or for further fine-tuning and inference. Users may not re-publish or re-sell this model in the same or derivative form (including fine-tunes).
**
The SFT chat fine-tuned model's repo card follows below.
**
<p align="center">
<img width="1000px" alt="DeepSeek Coder" src="https://github.com/deepseek-ai/DeepSeek-Coder/blob/main/pictures/logo.png?raw=true">
</p>
<p align="center"><a href="https://www.deepseek.com/">[🏠Homepage]</a> | <a href="https://coder.deepseek.com/">[🤖 Chat with DeepSeek Coder]</a> | <a href="https://discord.gg/Tc7c45Zzu5">[Discord]</a> | <a href="https://github.com/guoday/assert/blob/main/QR.png?raw=true">[Wechat(微信)]</a> </p>
<hr>
### 1. Introduction of Deepseek Coder
Deepseek Coder is composed of a series of code language models, each trained from scratch on 2T tokens, with a composition of 87% code and 13% natural language in both English and Chinese. We provide various sizes of the code model, ranging from 1B to 33B versions. Each model is pre-trained on project-level code corpus by employing a window size of 16K and a extra fill-in-the-blank task, to support project-level code completion and infilling. For coding capabilities, Deepseek Coder achieves state-of-the-art performance among open-source code models on multiple programming languages and various benchmarks.
- **Massive Training Data**: Trained from scratch on 2T tokens, including 87% code and 13% linguistic data in both English and Chinese languages.
- **Highly Flexible & Scalable**: Offered in model sizes of 1.3B, 5.7B, 6.7B, and 33B, enabling users to choose the setup most suitable for their requirements.
- **Superior Model Performance**: State-of-the-art performance among publicly available code models on HumanEval, MultiPL-E, MBPP, DS-1000, and APPS benchmarks.
- **Advanced Code Completion Capabilities**: A window size of 16K and a fill-in-the-blank task, supporting project-level code completion and infilling tasks.
### 2. Model Summary
deepseek-coder-33b-instruct is a 33B parameter model initialized from deepseek-coder-33b-base and fine-tuned on 2B tokens of instruction data.
- **Home Page:** [DeepSeek](https://deepseek.com/)
- **Repository:** [deepseek-ai/deepseek-coder](https://github.com/deepseek-ai/deepseek-coder)
- **Chat With DeepSeek Coder:** [DeepSeek-Coder](https://coder.deepseek.com/)
### 3. How to Use
Here give some examples of how to use our model.
#### Chat Model Inference
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("deepseek-ai/deepseek-coder-33b-instruct", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("deepseek-ai/deepseek-coder-33b-instruct", trust_remote_code=True).cuda()
messages=[
{ 'role': 'user', 'content': "write a quick sort algorithm in python."}
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
# 32021 is the id of <|EOT|> token
outputs = model.generate(inputs, max_new_tokens=512, do_sample=False, top_k=50, top_p=0.95, num_return_sequences=1, eos_token_id=32021)
print(tokenizer.decode(outputs[0][len(inputs[0]):], skip_special_tokens=True))
4. License
This code repository is licensed under the MIT License. The use of DeepSeek Coder models is subject to the Model License. DeepSeek Coder supports commercial use.
See the LICENSE-MODEL for more details.
5. Contact
If you have any questions, please raise an issue or contact us at [email protected].