--- base_model: - TheDrummer/Fallen-Llama-3.3-R1-70B-v1 - Sao10K/L3-70B-Euryale-v2.1 - mlabonne/Hermes-3-Llama-3.1-70B-lorablated - TareksLab/Fetishist-V5-LLaMA-70B library_name: transformers tags: - mergekit - merge --- # MERGE3 This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit). ## Merge Details ### Merge Method This model was merged using the [DARE TIES](https://arxiv.org/abs/2311.03099) merge method using [mlabonne/Hermes-3-Llama-3.1-70B-lorablated](https://huggingface.co/mlabonne/Hermes-3-Llama-3.1-70B-lorablated) as a base. ### Models Merged The following models were included in the merge: * [TheDrummer/Fallen-Llama-3.3-R1-70B-v1](https://huggingface.co/TheDrummer/Fallen-Llama-3.3-R1-70B-v1) * [Sao10K/L3-70B-Euryale-v2.1](https://huggingface.co/Sao10K/L3-70B-Euryale-v2.1) * [TareksLab/Fetishist-V5-LLaMA-70B](https://huggingface.co/TareksLab/Fetishist-V5-LLaMA-70B) ### Configuration The following YAML configuration was used to produce this model: ```yaml models: - model: TheDrummer/Fallen-Llama-3.3-R1-70B-v1 parameters: weight: 0.25 density: 0.5 - model: TareksLab/Fetishist-V5-LLaMA-70B parameters: weight: 0.25 density: 0.5 - model: Sao10K/L3-70B-Euryale-v2.1 parameters: weight: 0.25 density: 0.5 - model: mlabonne/Hermes-3-Llama-3.1-70B-lorablated parameters: weight: 0.25 density: 0.5 merge_method: dare_ties base_model: mlabonne/Hermes-3-Llama-3.1-70B-lorablated parameters: normalize: false int8_mask: true out_dtype: bfloat16 chat_template: llama3 tokenizer: source: TareksLab/Fetishist-V5-LLaMA-70B ```