Update README.md

README.md

@@ -11,7 +11,7 @@ tags:
 
 <!-- Provide a quick summary of what the model is/does. -->
 # Intuitive Thinker
-Improve small size LLM reasoning by employs system of thinking mental models by structured Chain-of-Thoughts process and thoughtful reflection prior to responding to user queries.
+Attempt to improve small size LLM reasoning by employs system of thinking mental models by structured Chain-of-Thoughts process and thoughtful reflection prior to responding to user queries.
 
 ***Problem:*** <br/>
 smaller-sized transformer models exhibit inferior reasoning capabilities compared to their larger counterparts, whose advanced reasoning abilities stem from broader connection networks that facilitate cross-domain inference.
@@ -36,6 +36,90 @@ https://huggingface.co/mychen76/Llama-3.1_Intuitive-Thinker
 Quantized:  mychen76/Llama-3.1_Intuitive-Thinker_8B_2309_GGUF
 https://huggingface.co/mychen76/Llama-3.1_Intuitive-Thinker_8B_2309_GGUF
 
+***HF Usage*** <br/>
+```python
+from intuitive_thinker.mental_model import MentalModel
+from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
+import torch
+import json
+
+question="count number of r in word strawberry?"
+
+## format question using mental model template
+mental_model = MentalModel(MentalModel.CHAIN_OF_THOUGHTS)
+prompt = json.loads(mental_model(question))
+
+bnb_config = BitsAndBytesConfig(
+    load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype="float16", bnb_4bit_use_double_quant=True
+)
+        
+# Prepare the input as before
+messages = [
+    {"role": "system", "content": prompt['system_message']},
+    {"role": "user", "content": prompt['user_input'] }
+]
+
+hf_model_id = "mychen76/Llama-3.1_Intuitive-Thinker"
+
+# 1: Load the model and tokenizer
+model = AutoModelForCausalLM.from_pretrained(hf_model_id, device_map="auto", quantization_config=bnb_config, torch_dtype=torch.bfloat16)
+tokenizer = AutoTokenizer.from_pretrained(hf_model_id)
+
+# 2: Apply the chat template
+formatted_chat = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+#print("Formatted chat:\n", formatted_chat)
+
+# 3: Tokenize the chat (This can be combined with the previous step using tokenize=True)
+inputs = tokenizer(formatted_chat, return_tensors="pt", add_special_tokens=False)
+# Move the tokenized inputs to the same device the model is on (GPU/CPU)
+inputs = {key: tensor.to(model.device) for key, tensor in inputs.items()}
+#print("Tokenized inputs:\n", inputs)
+
+# 4: Generate text from the model
+outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.1)
+#print("Generated tokens:\n", outputs)
+
+# 5: Decode the output back to a string
+decoded_output = tokenizer.decode(outputs[0][inputs['input_ids'].size(1):], skip_special_tokens=True)
+print("Decoded output:\n", decoded_output)
+```
+result
+```bash
+Decoded output:
+ <thinking>
+a. Briefly analyze the question and outline our approach:
+The question asks for the count of the letter 'r' in the word'strawberry'. This is a simple text analysis task that requires examining the word for the target letter.
+
+b. Present a clear plan of steps to solve the problem:
+1. Write out the word'strawberry' in its full form.
+2. Count the occurrences of the letter 'r' in the word.
+3. Provide the final count as the answer.
+
+c. Use a "Chain of Thought" reasoning process:
+1. Start by writing out the word: s-t-r-a-w-b-e-r-r-y
+2. Now, count the 'r's: r (1), r (2), r (3)
+3. We have found all 'r's in the word.
+
+<reflection>
+a. Review our reasoning:
+We have broken down the task into simple steps and followed through with the counting process.
+
+b. Check for potential errors or oversights:
+- We have accounted for all letters in the word.
+- We have specifically targeted the letter 'r' for counting.
+- We have counted each occurrence of 'r' correctly.
+
+c. Confirm our conclusion:
+Yes, our conclusion is correct. The word'strawberry' contains 3 occurrences of the letter 'r'.
+
+</reflection>
+</thinking>
+
+<output>
+The word'strawberry' contains 3 occurrences of the letter 'r'.
+</output>
+```
+
 ***Ollama.com*** <br/>
 https://ollama.com/mychen76/llama3.1-intuitive-thinker
 
@@ -47,6 +131,7 @@ For direct easy to use each mental model has been package on own model package.
 4. Iceberg Mental Model: [mychen76/llama3.1-intuitive-thinker:iceberg-mental-model.q5]
 5. Second Order Thinking: [mychen76/llama3.1-intuitive-thinker:second-order-thinking.q5]
 
+
 ### Samples
 
 ***Sample: Chain-of-Thoughts***