LLM resource-constrained Inference - a BHbean Collection

BHbean 's Collections

MoE LLM Systems

LLM resource-constrained Inference

New LLM Algorithms

LLM Internal Mechanism

Prompt Engineering

Speculative Decoding

KV Cache Compression

LLM reasoning systems

LLM resource-constrained Inference

updated 11 days ago

PRIMA.CPP: Speeding Up 70B-Scale LLM Inference on Low-Resource Everyday Home Clusters

Paper • 2504.08791 • Published 27 days ago • 127
70% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float

Paper • 2504.11651 • Published 18 days ago • 28