Bochkov/nemo_bvv_moe

Updated 3 days ago • 11

Note nemo_bvv_moe is a multi-lingual Mixture-of-Experts (MoE) model constructed by combining nemo_bvv_ru and nemo_bvv_zh—thanks to their fully shared and frozen token embeddings, making direct model fusion feasible without re-training embeddings. Proof: SOTA tokenizers can be used for compatible precomputed embeddings, industry can repeat with their tokenizers.

Bochkov/nemo_bvv_ru

Updated 3 days ago • 12

Note This is nemo_bvv_ru, a proof-of-concept Russian language causal language model trained with completely frozen, precomputed token embeddings based on SOTA Mistral/Nemo tokenizer (visual string composition, not standard semantic embeddings).

Bochkov/nemo_bvv_zh

Updated 3 days ago • 10

Note This is nemo_bvv_zh, a Chinese language GPT-style model trained with fully precomputed and frozen token embeddings using the Mistral/Nemo tokenizer (visual appearance-based). Specifically designed to demonstrate the compatibility of SOTA tokenizers in the fixed-embedding paradigm.