Tuan Tran

tuantm

AI & ML interests

None yet

Recent Activity

liked a model about 16 hours ago

GreenNode/GreenNode-Embedding-Large-VN-Mixed-V1

liked a model 3 months ago

reducto/RolmOCR

liked a model 3 months ago

lmstudio-community/openhands-lm-32b-v0.1-GGUF

View all activity

Organizations

liked a model about 16 hours ago

GreenNode/GreenNode-Embedding-Large-VN-Mixed-V1

liked 3 models 3 months ago

liked 2 models 5 months ago

mkurman/Qwen2.5-14B-DeepSeek-R1-1M

Text Generation • 15B • Updated Jan 27 • 2.57k • 52

FuseAI/FuseO1-DeepSeekR1-QwQ-SkyT1-32B-Preview

33B • Updated Jan 25 • 188 • 130

liked 3 models 6 months ago

google/timesfm-2.0-500m-pytorch

Time Series Forecasting • 0.5B • Updated Apr 16 • 5.38k • 179

clapAI/modernBERT-base-multilingual-sentiment

Text Classification • 0.1B • Updated Jan 8 • 5.05k • 22

nampham1106/snowflake-arctic-embed-m-v2.0

liked a model 7 months ago

sail/Sailor2-20B

Text Generation • 19B • Updated Apr 29 • 62 • 10

liked a dataset 7 months ago

microsoft/orca-agentinstruct-1M-v1

Viewer • Updated Nov 1, 2024 • 1.05M • 1.92k • 446

liked 5 models 8 months ago

tablegpt/TableGPT2-7B

8B • Updated Feb 13 • 19.8k • 213

facebook/MobileLLM-1B

Text Generation • Updated May 5 • 338 • 121

facebook/MobileLLM-125M

Text Generation • Updated May 5 • 624 • 120

HIT-TMG/KaLM-embedding-multilingual-mini-instruct-v1

Marqo/marqo-ecommerce-embeddings-L

0.7B • Updated Nov 12, 2024 • 10.9k • 37

reacted to tomaarsen's post with 🔥 8 months ago

Post

7189

📣 Sentence Transformers v3.2.0 is out, marking the biggest release for inference in 2 years! 2 new backends for embedding models: ONNX (+ optimization & quantization) and OpenVINO, allowing for speedups up to 2x-3x AND Static Embeddings for 500x speedups at 10-20% accuracy cost.

1️⃣ ONNX Backend: This backend uses the ONNX Runtime to accelerate model inference on both CPU and GPU, reaching up to 1.4x-3x speedup depending on the precision. We also introduce 2 helper methods for optimizing and quantizing models for (much) faster inference.
2️⃣ OpenVINO Backend: This backend uses Intel their OpenVINO instead, outperforming ONNX in some situations on CPU.

Usage is as simple as SentenceTransformer("all-MiniLM-L6-v2", backend="onnx"). Does your model not have an ONNX or OpenVINO file yet? No worries - it'll be autoexported for you. Thank me later 😉

🔒 Another major new feature is Static Embeddings: think word embeddings like GLoVe and word2vec, but modernized. Static Embeddings are bags of token embeddings that are summed together to create text embeddings, allowing for lightning-fast embeddings that don't require any neural networks. They're initialized in one of 2 ways:

1️⃣ via Model2Vec, a new technique for distilling any Sentence Transformer models into static embeddings. Either via a pre-distilled model with from_model2vec or with from_distillation where you do the distillation yourself. It'll only take 5 seconds on GPU & 2 minutes on CPU, no dataset needed.
2️⃣ Random initialization. This requires finetuning, but finetuning is extremely quick (e.g. I trained with 3 million pairs in 7 minutes). My final model was 6.6% worse than bge-base-en-v1.5, but 500x faster on CPU.

Full release notes: https://github.com/UKPLab/sentence-transformers/releases/tag/v3.2.0
Documentation on Speeding up Inference: https://sbert.net/docs/sentence_transformer/usage/efficiency.html