Upload fine-tuned Qwen-1.8B LoRA adapters and tokenizer for stock quant education

README.md

@@ -1,3 +1,139 @@
----
-license: apache-2.0
----
+---
+license: apache-2.0
+language:
+  - zh
+  - en
+tags:
+  - qwen
+  - lora
+  - peft
+  - large-language-model
+  - quantitative-finance
+  - stock-market
+  - trading-strategy
+  - investment-education
+  - financial-explainer
+  - text-generation
+  - instruction-following
+base_model: Qwen/Qwen-1_8B-Chat
+pipeline_tag: text-generation
+widget:
+  # Example of how to use the model with PEFT
+  # (You'll need to adjust this based on how Qwen LoRA models are typically loaded)
+  - text: "请用大白话解释什么是移动平均线？"
+    example_title: "Explain Moving Average"
+---
+
+# Qwen-1.8B-Chat LoRA for Stock Market Quantitative Education (股票量化投教LoRA模型)
+
+This repository contains LoRA (Low-Rank Adaptation) adapters fine-tuned on the `Qwen/Qwen-1_8B-Chat` model.
+The goal of this fine-tuning is to create an AI assistant that can explain stock market and quantitative trading concepts in plain language ("大白话"), making these topics more accessible to beginners.
+
+## Model Description
+
+This model is a PEFT-LoRA adaptation of the `Qwen/Qwen-1_8B-Chat` large language model. It has been fine-tuned on a small, custom dataset of ~20 instruction-response pairs focused on financial education. Due to the small dataset size, this model should be considered **experimental and for demonstration purposes**.
+
+**Developed by:** 天算AI科技研发实验室 (Natural Algorithm AI R&D Lab) - jinv2
+
+## Intended Uses & Limitations
+
+**Intended Uses:**
+
+*   Educational tool for understanding basic stock market and quantitative trading terms.
+*   Generating simple explanations of financial concepts.
+*   Demonstrating the LoRA fine-tuning process on a chat model for a specific domain.
+
+**Limitations:**
+
+*   **Not for Financial Advice:** The information provided by this model is strictly for educational purposes and should NOT be considered financial advice. Always consult with a qualified financial advisor before making investment decisions.
+*   **Limited Knowledge:** Fine-tuned on a very small dataset. Its knowledge is restricted and may not be comprehensive or entirely accurate.
+*   **Potential for Hallucinations:** Like all LLMs, it may generate incorrect or nonsensical information.
+*   **Overfitting:** Due to the small dataset, the model may be overfit to the training examples.
+*   **Bias:** The training data may contain biases, which could be reflected in the model's responses.
+*   **Requires Base Model:** These are LoRA adapters and require the original `Qwen/Qwen-1_8B-Chat` base model to be loaded first.
+
+## How to Use with PEFT
+
+You would typically load the base model and then apply these LoRA adapters using the PEFT library.
+
+```python
+from peft import PeftModel
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+
+base_model_name = "Qwen/Qwen-1_8B-Chat"
+adapter_model_name = "jinv2/qwen-1.8b-chat-lora-stock-quant-edu" # Replace with your actual model name on Hub
+
+# Load tokenizer
+tokenizer = AutoTokenizer.from_pretrained(base_model_name, trust_remote_code=True)
+if tokenizer.pad_token_id is None:
+    tokenizer.pad_token_id = tokenizer.eos_token_id # Or <|endoftext|> ID: 151643
+
+# Load base model
+base_model = AutoModelForCausalLM.from_pretrained(
+    base_model_name,
+    torch_dtype=torch.float16, # or "auto"
+    device_map="auto",
+    trust_remote_code=True
+)
+
+# Load LoRA adapter
+model = PeftModel.from_pretrained(base_model, adapter_model_name)
+model = model.eval() # Set to evaluation mode
+
+# Example Inference (Qwen chat format)
+prompt = "请用大白话解释什么是MACD指标？"
+# For Qwen-Chat, using model.chat() is recommended
+response, history = model.chat(tokenizer, prompt, history=None, system="You are a helpful financial education assistant.")
+print(response)
+
+# Alternative generic generation
+# messages = [
+#     {"role": "system", "content": "You are a helpful financial education assistant."},
+#     {"role": "user", "content": prompt}
+# ]
+# text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
+# model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
+# generated_ids = model.generate(model_inputs.input_ids, max_new_tokens=512)
+# generated_ids = [output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)]
+# response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
+# print(response)
+
+Training Details
+
+Base Model: Qwen/Qwen-1_8B-Chat
+
+Fine-tuning Method: LoRA (Low-Rank Adaptation) via PEFT and TRL SFTTrainer.
+
+Dataset: ~20 custom instruction-response pairs for financial education.
+
+Training Configuration (Key Parameters):
+
+LoRA r: 8
+
+LoRA alpha: 16
+
+Target Modules: c_attn, c_proj, w1, w2
+
+Optimizer: AdamW (default from Trainer)
+
+Precision: FP32 (due to issues with FP16/BF16 GradScaler in the environment)
+
+Epochs: ~17 (based on 80 steps)
+
+Batch Size (effective): 4 (per_device_train_batch_size=1, gradient_accumulation_steps=4)
+
+Learning Rate: 2e-4
+
+Max Sequence Length: 512
+
+Disclaimer
+
+This model is provided "as-is" without any warranty. The developers are not responsible for any outcomes resulting from the use of this model. Always verify information and use at your own risk.
+
+Copyright Information:
+
+© 天算AI科技研发实验室 (Natural Algorithm AI R&D Lab) - jinv2
+All rights reserved unless otherwise specified by the license.
+
+

adapter_config.json

@@ -0,0 +1,30 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "Qwen/Qwen-1_8B-Chat",
+  "bias": "none",
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "w1",
+    "w2",
+    "c_proj",
+    "c_attn"
+  ],
+  "task_type": "CAUSAL_LM",
+  "use_dora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:94677c156cb9a430296fbb57613430e30cd77f2b0442ff2c29f2ed94d7248be0
+size 26867880

special_tokens_map.json

@@ -0,0 +1,4 @@
+{
+  "eos_token": "<|endoftext|>",
+  "pad_token": "<|endoftext|>"
+}

tokenizer_config.json

@@ -0,0 +1,14 @@
+{
+  "added_tokens_decoder": {},
+  "auto_map": {
+    "AutoTokenizer": [
+      "Qwen/Qwen-1_8B-Chat--tokenization_qwen.QWenTokenizer",
+      null
+    ]
+  },
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|endoftext|>",
+  "model_max_length": 8192,
+  "pad_token": "<|endoftext|>",
+  "tokenizer_class": "QWenTokenizer"
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a9de69189de29010330d67e0bb8539112856b4725f2bcc2cc26387be3e7b4a51
+size 5457