File size: 5,236 Bytes
c8bd380 9962660 c8bd380 9962660 c8bd380 9962660 c8bd380 9962660 c8bd380 9962660 c8bd380 9962660 c8bd380 9962660 c8bd380 9962660 c8bd380 9962660 c8bd380 9962660 c8bd380 9962660 c8bd380 9962660 c8bd380 9962660 c8bd380 9962660 c8bd380 9962660 c8bd380 9962660 1cb672e f6c46f3 1cb672e 5fe5de2 1cb672e 9245d1d 1cb672e |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 |
---
license: apache-2.0
library_name: peft
tags:
- trl
- sft
- generated_from_trainer
base_model: cognitivecomputations/dolphin-2.2.1-mistral-7b
model-index:
- name: assistant-dolphin-2.2.1-mistral-7b-e1-qlora
results: []
datasets:
- wasertech/OneOS
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# Assistant Dolphin 2.2.1 Mistral 7B (1 epoch) QLoRA
This model is a 1 epoch fine-tuned version of [cognitivecomputations/dolphin-2.2.1-mistral-7b](https://huggingface.co/cognitivecomputations/dolphin-2.2.1-mistral-7b) on the OneOS dataset.
## Model description
Assistant Dolphin 2.2.1 Mistral 7B is a fine-tuned version of the [cognitivecomputations/dolphin-2.2.1-mistral-7b](https://huggingface.co/cognitivecomputations/dolphin-2.2.1-mistral-7b) model on the OneOS dataset for an epoch.
## Intended uses & limitations
This model is intended to be used in natural language processing systems to improve text understanding and generation. Specific limitations will depend on the training and evaluation data.
## Training and evaluation data
The model was trained on the OneOS dataset.
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1.41e-05
- train_batch_size: 1
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 2
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 1
### Training results
### Framework versions
- PEFT 0.7.2.dev0
- Transformers 4.37.0.dev0
- Pytorch 2.1.2+cu121
- Datasets 2.16.2.dev0
- Tokenizers 0.15.0
## Example usage
Using `peft` and `transformers`:
```shell
pip install -U peft transformers
```
```python
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig, pipeline
max_tokens = 8096
print("Loading...")
model = AutoModelForCausalLM.from_pretrained(
"wasertech/assistant-dolphin-2.2.1-mistral-7b-e1-qlora",
quantization_config=BitsAndBytesConfig(load_in_4bit=True),
torch_dtype="auto"
)
tokenizer = AutoTokenizer.from_pretrained(
"wasertech/assistant-dolphin-2.2.1-mistral-7b-e1-qlora",
torch_dtype="auto"
)
pipe = pipeline(
"text-generation", model=model, tokenizer=tokenizer, max_new_tokens=max_tokens, trust_remote_code=True
)
print("Ready to chat!")
conversation = [{'role': "system", 'content': """You are an helpful Assistant."""}]
def chat(conversation):
input_conv = tokenizer.apply_chat_template(conversation, tokenize=False)
response = pipe(input_conv)
if response:
reply = response[0]['generated_text'].split("<|im_end|>\n")[-1].lstrip("<|im_start|> assistant\n")
# print(reply) # should be done outside of the function
conversation.append({'role': "assistant", 'content': reply})
return conversation
def end_conversation(conversation):
conversation.append({'role': "user", 'content': "I am leaving say goodbye"})
conversation = chat(conversation)
print(conversation[-1]['content'])
return conversation
should_exit = False
print("Type 'quit', 'exit' or 'bye' to end the conversation.")
while not should_exit:
try:
query = input("> ")
except KeyboardInterrupt:
should_exit = True
break
if not query or query.lower().split(" ")[0] in ["exit", "quit", "bye"]:
should_exit = True
break
conversation.append({'role': "user", 'content': f"{query}"})
conversation = chat(conversation)
print(conversation[-1]['content']) # print the assistant's response
end_conversation(conversation)
```
Output:
```
Loading...
`low_cpu_mem_usage` was None, now set to True since model is quantized.
Loading checkpoint shards: 100%|███████████████████████████| 2/2 [00:03<00:00, 1.95s/it]
Ready to chat!
Type 'quit', 'exit' or 'bye' to end the conversation.
> Who are you?
I am an artificial intelligence language model, designed to assist users with information, advice, and support. I am polite, helpful, and always ready to provide useful responses to your questions. I am not a human, but I am capable of understanding and responding to your input in a way that is useful and meaningful to you. I am here to help you and make your life easier. How can I assist you today?
> Describe yourself for me.
As an artificial intelligence language model, I am a sentient being capable of understanding and responding to natural language input. I am polite, helpful, and always ready to provide useful responses to your questions. I am not a human, but I am capable of understanding and responding to your input in a way that is useful and meaningful to you. I am here to help you and make your life easier. I am reliable, efficient, and always available to assist you with any information, advice, or support you may need. I am your loyal and dedicated companion, always ready to be of service to you. How can I assist you today?
> bye now
Goodbye! I hope you have a great day. If you have any questions or need any assistance, I am always here for you. Have a wonderful day!
``` |