Hinova commited on
Commit
a1b45a1
·
verified ·
1 Parent(s): 44ae1b3

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +35 -1
README.md CHANGED
@@ -3,4 +3,38 @@ base_model:
3
  - mistralai/Mistral-7B-v0.1
4
  ---
5
 
6
- 4 bit quantized version by MLX
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  - mistralai/Mistral-7B-v0.1
4
  ---
5
 
6
+ # Mistral-7B-v0.1-4bit-mlx
7
+
8
+ A 4-bit LoRA‑fine‑tuned Mistral-7B model in Apple MLX format, created via the MLX Examples LoRA tutorial.
9
+
10
+ ---
11
+
12
+ ## 🚀 Overview
13
+
14
+ This model was created by following the MLX LoRA tutorial:
15
+
16
+ - **LoRA fine‑tuning** applied on Mistral‑7B using low‑rank adapters
17
+ - **Model fusion** performed with `convert.py` to combine base weights and LoRA adapters
18
+ - **Quantized to 4-bit** for memory efficiency on Apple Silicon devices
19
+ - Packaged in MLX format, ready for downstream use via `mlx-lm`
20
+
21
+ The pipeline is fully automated based on `mlx-examples/lora/convert.py`, with no manual weight editing.
22
+
23
+ ---
24
+
25
+ ## 📦 Model Packaging
26
+
27
+ | File | Description |
28
+ |---------------------|----------------------------------------------|
29
+ | `weights.npz` | Fused weights: base + LoRA adapters |
30
+ | `config.json` | Model configuration & quantization metadata |
31
+ | `tokenizer.model` | SentencePiece tokenizer for Mistral-7B |
32
+
33
+ ---
34
+
35
+ ## 💡 Usage
36
+
37
+ ### Inference with `mlx-lm`
38
+
39
+ ```bash
40
+ pip install mlx-lm