prithivMLmods commited on
Commit
590e622
·
verified ·
1 Parent(s): 6b7b9a0

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +102 -1
README.md CHANGED
@@ -14,4 +14,105 @@ tags:
14
  - science
15
  - math
16
  - moe
17
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
14
  - science
15
  - math
16
  - moe
17
+ ---
18
+
19
+ ![12.png](https://cdn-uploads.huggingface.co/production/uploads/65bb837dbfb878f46c77de4c/FAnKrTFpUHZbu9J0fLKCL.png)
20
+
21
+ # **Eta-Aurigae-0.6B-Echelon1**
22
+
23
+ > **Eta-Aurigae-0.6B-Echelon1** is a compact, efficient model specialized in **science, factual accuracy**, and **structured reasoning**. Fine-tuned on **Qwen3-0.6B** using the **MoT (Mixture of Thoughts)** dataset—focused on scientific understanding and expert factual domains—it delivers high-precision outputs for STEM education, tutoring, and analytical thinking in resource-constrained environments.
24
+
25
+ > \[!note]
26
+ > GGUF: [https://huggingface.co/prithivMLmods/Eta-Aurigae-0.6B-Echelon1-GGUF](https://huggingface.co/prithivMLmods/Eta-Aurigae-0.6B-Echelon1-GGUF)
27
+
28
+ ---
29
+
30
+ ## **Key Features**
31
+
32
+ 1. **MoT Fine-Tuning for Science & Facts**
33
+ Trained on a **Mixture of Thoughts** dataset emphasizing scientific accuracy, explanatory depth, and structured reasoning across biology, physics, chemistry, and factual domains.
34
+
35
+ 2. **Scientific Precision in a Small Footprint**
36
+ Delivers clear, step-by-step reasoning in scientific problems—ideal for students, educators, and lightweight educational tools.
37
+
38
+ 3. **Factually Consistent Output Generation**
39
+ Optimized for **high factual alignment** and structured explanations—reliable for knowledge recall, concept breakdowns, and factual analysis.
40
+
41
+ 4. **Supports Markdown, LaTeX, and JSON**
42
+ Outputs clean, structured formats like **Markdown**, **LaTeX**, and **JSON**, useful for technical documentation and educational content.
43
+
44
+ 5. **Multilingual Science-Aware Responses**
45
+ Handles factual content in 20+ languages, especially in academic and technical contexts.
46
+
47
+ 6. **Lightweight and Inference-Ready**
48
+ Efficient on **CPUs**, **low-VRAM GPUs**, and **offline edge deployments** without sacrificing factual clarity.
49
+
50
+ ---
51
+
52
+ ## **Quickstart with Transformers**
53
+
54
+ ```python
55
+ from transformers import AutoModelForCausalLM, AutoTokenizer
56
+
57
+ model_name = "prithivMLmods/Eta-Aurigae-0.6B-Echelon1"
58
+
59
+ model = AutoModelForCausalLM.from_pretrained(
60
+ model_name,
61
+ torch_dtype="auto",
62
+ device_map="auto"
63
+ )
64
+ tokenizer = AutoTokenizer.from_pretrained(model_name)
65
+
66
+ prompt = "What causes the northern lights (Aurora Borealis)? Explain in simple terms."
67
+
68
+ messages = [
69
+ {"role": "system", "content": "You are a science tutor that explains complex concepts clearly."},
70
+ {"role": "user", "content": prompt}
71
+ ]
72
+
73
+ text = tokenizer.apply_chat_template(
74
+ messages,
75
+ tokenize=False,
76
+ add_generation_prompt=True
77
+ )
78
+
79
+ model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
80
+
81
+ generated_ids = model.generate(
82
+ **model_inputs,
83
+ max_new_tokens=512
84
+ )
85
+ generated_ids = [
86
+ output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
87
+ ]
88
+
89
+ response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
90
+ print(response)
91
+ ```
92
+
93
+ ---
94
+
95
+ ## **Intended Use**
96
+
97
+ * Science education and fact-based tutoring
98
+ * Concept explanations in physics, biology, and chemistry
99
+ * Structured technical content generation (e.g., LaTeX, Markdown)
100
+ * Deployment in low-resource, educational, or mobile scenarios
101
+ * Lightweight inference with high factual fidelity
102
+
103
+ ---
104
+
105
+ ## **Limitations**
106
+
107
+ * Not optimized for general conversation or creative writing
108
+ * Short context limits multi-document scientific reasoning
109
+ * Performance dips in abstract reasoning outside scientific scope
110
+ * Not tuned for code or free-form generation
111
+
112
+ ---
113
+
114
+ ## **References**
115
+
116
+ 1. [Qwen2.5 Technical Report (2024)](https://arxiv.org/pdf/2412.15115)
117
+ 2. [Mixture of Thoughts Dataset](https://huggingface.co/datasets/open-r1/Mixture-of-Thoughts)
118
+ 3. [YaRN: Efficient Context Extension for LLMs](https://arxiv.org/pdf/2309.00071)