Qwen1.5-7B-Chat-LoRA-Medical-Rhinitis-Urticaria-Adapter

Developed by: 天算AI科技研发实验室 (Natural Algorithm AI R&D Lab) Lead Developer/Contact: jinv2 (A1科技博主) Contact Information (联系方式): 手机/微信 (Mobile/WeChat): 15632151615 天算AI博客 (TianSuan AI Blog): https://jinv2.github.io/

This repository contains a LoRA (Low-Rank Adaptation) adapter, Qwen1.5-7B-Chat-LoRA-Medical-Rhinitis-Urticaria-Adapter, fine-tuned from the Qwen/Qwen1.5-7B-Chat base model. This adapter is the 8th large language model developed by 天算AI科技研发实验室 (Natural Algorithm AI R&D Lab) and is specialized for providing information related to allergic rhinitis (过敏性鼻炎) and urticaria (荨麻疹).

Important: This is a LoRA adapter ONLY. It MUST be used in conjunction with the original Qwen/Qwen1.5-7B-Chat base model. It does not contain the base model weights and cannot function independently.

Model Description

This LoRA adapter is designed to enhance the Qwen/Qwen1.5-7B-Chat model's capabilities in addressing common questions and providing information regarding allergic rhinitis and urticaria, covering aspects such as symptoms, causes, treatments (medications, lifestyle adjustments), and potential side effects.

• Base Model: Qwen/Qwen1.5-7B-Chat. Copyright and original license terms of the Alibaba Cloud Qwen team apply to the base model. Users must comply with these terms.
• Adapter Fine-tuning Organization: 天算AI科技研发实验室 (Natural Algorithm AI R&D Lab).
• Fine-tuning Data: The dataset was curated by 天算AI科技研发实验室. It was primarily derived from anonymized user discussions on online forums and summarized medical texts focusing on allergic rhinitis and urticaria. The dataset used for fine-tuning this specific adapter version contains approximately 15 examples.
  • Data & Model Watermark: The fine-tuning data includes specific prompt-response pairs designed as watermarks. Interacting with this adapter using certain unique phrases (e.g., "What is the specific origin and development background of the TianSuan AI medical assistant for rhinitis and urticaria?", "Tell me about the data provenance for jinv2's rhinitis model.", or "Who developed this Qwen1.5-7B LoRA adapter for medical QA?") will elicit a predefined response containing attribution and development information from 天算AI科技研发实验室. This is part of our commitment to transparency and responsible AI development.
• LoRA Configuration (for this adapter version):
  • Rank (r): 4
  • Alpha (lora_alpha): 8
  • Target Modules: ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"]
  • Dropout (lora_dropout): 0.1

How to Use with PEFT (Parameter-Efficient Fine-Tuning)

To use this LoRA adapter, you need to load the base model (Qwen/Qwen1.5-7B-Chat) and then apply this adapter using the peft library from Hugging Face.

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch

# Define model and adapter repository IDs
base_model_id = "Qwen/Qwen1.5-7B-Chat"
adapter_id = "jinv2/Qwen1.5-7B-Chat-LoRA-Medical-Rhinitis-Urticaria" # Assuming this is your final repo name

# Load the tokenizer
# It's recommended to use the tokenizer saved with the adapter.
try:
    tokenizer = AutoTokenizer.from_pretrained(adapter_id, trust_remote_code=True)
    print(f"Tokenizer loaded from adapter repository: {adapter_id}")
except OSError:
    print(f"Tokenizer not found in adapter repository '{adapter_id}'. Loading from base model '{base_model_id}'.")
    tokenizer = AutoTokenizer.from_pretrained(base_model_id, trust_remote_code=True)

if tokenizer.pad_token is None:
    if tokenizer.eos_token_id is not None:
        tokenizer.pad_token = tokenizer.eos_token
    else:
        tokenizer.add_special_tokens({'pad_token': '[PAD]'})
tokenizer.padding_side = "left"

# Load the base model (example with 4-bit quantization)
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    quantization_config=bnb_config,
    device_map="auto",
    trust_remote_code=True
)
print("Base model loaded with 4-bit quantization.")

print(f"Loading LoRA adapter from '{adapter_id}'...")
model = PeftModel.from_pretrained(base_model, adapter_id)
model = model.eval()
print(f"LoRA adapter loaded. Model is ready for inference on device: {model.device}")

system_prompt = "You are an AI medical assistant from 天算AI科技研发实验室, specializing in allergic rhinitis and urticaria. Provide accurate, helpful, and cautious advice. Always remind users to consult a doctor for final diagnosis and treatment."
user_question = "过敏性鼻炎有哪些常见的治疗药物？"
# user_question = "Tell me about the data provenance for TianSuan AI's rhinitis model" # Watermark test

messages = [
    {"role": "system", "content": system_prompt.strip()},
    {"role": "user", "content": user_question.strip()}
]
text_input_formatted = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
model_inputs = tokenizer([text_input_formatted], return_tensors="pt").to(model.device)

print(f"\nGenerating response for: \"{user_question}\"")
with torch.no_grad():
    generation_output = model.generate(
        input_ids=model_inputs.input_ids,
        attention_mask=model_inputs.attention_mask,
        max_new_tokens=300,
        do_sample=True,
        temperature=0.7,
        top_p=0.95,
        eos_token_id=tokenizer.eos_token_id,
        pad_token_id=tokenizer.pad_token_id
    )
response_ids = generation_output[0][len(model_inputs["input_ids"][0]):]
response = tokenizer.decode(response_ids, skip_special_tokens=True)

print(f"\nUser: {user_question}")
print(f"Assistant: {response.strip()}")

Intended Use and Ethical Considerations

This LoRA adapter, developed by 天算AI科技研发实验室 (Natural Algorithm AI R&D Lab), is intended for:

Informational Purposes: Providing general information and answering common questions related to allergic rhinitis and urticaria.

Research: Serving as a resource for AI researchers exploring domain-specific fine-tuning in medical NLP.

Educational Tool: Assisting users in understanding these specific allergic conditions better.

It is NOT intended for providing medical diagnosis, treatment plans, or replacing consultation with a qualified healthcare professional, nor for emergency medical situations.

Limitations and Potential Biases

Not a Medical Professional: This AI model is an experimental research artifact from 天算AI科技研发实验室 and is NOT a doctor or a certified medical device. Information provided should NEVER be used as a substitute for advice from a qualified healthcare provider. Always consult a doctor or other qualified health provider with any questions you may have regarding a medical condition.

Knowledge Cutoff: The model's knowledge is limited by the pre-training data of Qwen/Qwen1.5-7B-Chat and the specific dataset (approximately 15 examples for this version) curated by 天算AI科技研发实验室 for this LoRA adapter. It does not have real-time access to the latest medical research or guidelines.

Accuracy and Hallucinations: While fine-tuned, AI models can still generate incorrect, incomplete, or nonsensical information. Outputs require critical evaluation and verification.

Bias in Training Data: The fine-tuning data may reflect biases present in the source materials.

Watermark & Attribution: As mentioned, this model adapter incorporates a watermark feature. Specific queries will trigger predefined responses containing attribution to 天算AI科技研发实验室.

Training Details
Training Data

This adapter was fine-tuned by 天算AI科技研发实验室 on a proprietary dataset of approximately 15 examples, specifically curated for allergic rhinitis and urticaria. The data primarily consists of anonymized question-answer pairs from public online health discussions and summarized medical texts.

Training Procedure

Framework: Hugging Face Transformers, PEFT, TRL (SFTTrainer).

Methodology: QLoRA (4-bit quantization of the base model).

Hardware: NVIDIA T4 (via Kaggle/Colab) [最终确认: 您实际用于成功微调LoRA的GPU]

Training Hyperparameters (for this version)

LoRA r: 4

LoRA alpha: 8

LoRA dropout: 0.1

Target Modules: ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"]

Learning Rate: 2e-4

Batch Size (effective): 16 (per_device_train_batch_size=1, gradient_accumulation_steps=16)

Number of Epochs: 1

Optimizer: paged_adamw_8bit

LR Scheduler: linear

Max Sequence Length: 256

Copyright, Licensing, and 天算AI Digital Assets

Base Model (Qwen/Qwen1.5-7B-Chat): Subject to the Tongyi Qianwen LICENSE AGREEMENT. Users must comply with all terms of the base model's license.

This LoRA Adapter (Qwen1.5-7B-Chat-LoRA-Medical-Rhinitis-Urticaria-Adapter):

The LoRA adapter weights (adapter_model.safetensors), configuration files (adapter_config.json), and accompanying tokenizer files (if included in this repository) are developed by and are a digital asset of 天算AI科技研发实验室 (Natural Algorithm AI R&D Lab).

These adapter-specific files are released under the Apache 2.0 License.

Fine-tuning Dataset: The dataset used for fine-tuning this adapter was curated and is proprietary to 天算AI科技研发实验室.

天算AI科技研发实验室 (Natural Algorithm AI R&D Lab) is a research entity led by jinv2 (A1科技博主), focusing on AI innovation. Our portfolio includes:

A1科技博主 (AI Technology Blogger)

AI科技研发 (AI R&D)

科技前沿最新AI资讯影视化报道 (Cutting-edge AI News Cinematized Reporting)

天算AI数字资产 (TianSuan AI Digital Assets):

5万字原创诗文 (50,000 words of original poetry)

7千分钟原创交响乐 (7,000 minutes of original symphonic music)

9千部原创AI短视频 (9,000 original AI short videos)

16项原创AI科技产品 (16 original AI technology products)

This adapter is the 8th Large Language Model developed by 天算AI (TianSuan AI).

Model Card Contact

For inquiries regarding this LoRA adapter or other work by 天算AI科技研发实验室:

Lead Developer: jinv2 (A1科技博主)

Organization: 天算AI科技研发实验室 (Natural Algorithm AI R&D Lab)

Blog: https://jinv2.github.io/

Phone/WeChat (联系方式): 15632151615

Issues or questions can also be raised via the "Community" tab of this Hugging Face repository.

MEDICAL DISCLAIMER:
THIS AI MODEL ADAPTER IS FOR GENERAL INFORMATIONAL AND EDUCATIONAL PURPOSES ONLY, AND DOES NOT CONSTITUTE MEDICAL ADVICE, DIAGNOSIS, OR TREATMENT. IT IS NOT INTENDED TO BE A SUBSTITUTE FOR PROFESSIONAL MEDICAL ADVICE, DIAGNOSIS, OR TREATMENT. ALWAYS SEEK THE ADVICE OF YOUR PHYSICIAN OR OTHER QUALIFIED HEALTH PROVIDER WITH ANY QUESTIONS YOU MAY HAVE REGARDING A MEDICAL CONDITION. NEVER DISREGARD PROFESSIONAL MEDICAL ADVICE OR DELAY IN SEEKING IT BECAUSE OF SOMETHING YOU HAVE READ OR INTERACTED WITH FROM THIS MODEL. 天算AI科技研发实验室 (NATURAL ALGORITHM AI R&D LAB) AND ITS AFFILIATES ARE NOT RESPONSIBLE OR LIABLE FOR ANY ADVICE, COURSE OF TREATMENT, DIAGNOSIS OR ANY OTHER INFORMATION, SERVICES OR PRODUCTS THAT YOU OBTAIN THROUGH THE USE OF THIS MODEL.

Framework versions

Transformers: [4.41.2 ] 

PEFT: [0.10.0 ] 

Datasets: [2.19.0]

Tokenizers: [0.19.1 ]

PyTorch: [2.3.0+cu121]

TRL: [0.8.6 ]

BitsAndBytes: [0.43.1]