Comparison Table of Test Results by Model (BFCL)

Test Item	Model: tool	Model: base	Score Difference (tool - base)
irrelevance	0.8417	0.8750	-0.0333
multi_turn_base	0.1050	0.0850	+0.0200
parallel_multiple	0.0000	0.8900	-0.8900
parallel	0.0000	0.8850	-0.8850
simple	0.9350	0.9325	+0.0025
multiple	0.9450	0.9200	+0.0250

The model learned pretty well. In fact, it is normal because there is no parallel call data in the training data and no irrelevance data.

See axolotl config

axolotl version: 0.10.0.dev0

base_model: Qwen/Qwen3-4B
hub_model_id: minpeter/LoRA-Qwen3-4b-v1-iteration-02-sf-apigen-02

plugins:
  - axolotl.integrations.cut_cross_entropy.CutCrossEntropyPlugin
strict: false

datasets:
  - path: minpeter/apigen-mt-5k-friendli
    data_files:
      - train.jsonl
      - test.jsonl
    type: chat_template
    roles_to_train: ["assistant"]
    field_messages: messages
    message_property_mappings:
      role: role
      content: content
chat_template: chatml

dataset_prepared_path: last_run_prepared

output_dir: ./output
val_set_size: 0.0

sequence_len: 20000
sample_packing: true
eval_sample_packing: true
pad_to_sequence_len: true

load_in_4bit: true
adapter: qlora
lora_r: 16
lora_alpha: 32
lora_target_modules:
  - q_proj
  - k_proj
  - v_proj
  - o_proj
  - down_proj
  - up_proj
lora_mlp_kernel: true
lora_qkv_kernel: true
lora_o_kernel: true

wandb_project: "axolotl"
wandb_entity: "kasfiekfs-e"
wandb_watch:
wandb_name:
wandb_log_model:

gradient_accumulation_steps: 2
micro_batch_size: 1
num_epochs: 1
optimizer: adamw_torch_4bit
lr_scheduler: cosine
learning_rate: 0.0002

bf16: auto
tf32: true

gradient_checkpointing: offload
gradient_checkpointing_kwargs:
  use_reentrant: false
resume_from_checkpoint:
logging_steps: 1
flash_attention: true

warmup_steps: 10
evals_per_epoch: 4
saves_per_epoch: 1
weight_decay: 0.0
special_tokens:

LoRA-Qwen3-4b-v1-iteration-02-sf-apigen-02

This model is a fine-tuned version of Qwen/Qwen3-4B on the minpeter/apigen-mt-5k-friendli dataset.

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0002
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 2
• optimizer: Use OptimizerNames.ADAMW_TORCH_4BIT with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 10
• num_epochs: 1.0

Training results

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.6.0+cu124
• Datasets 3.5.1
• Tokenizers 0.21.1