Pretraining Context Compressor for Large Language Models with Embedding-Based Memory
YuHong Dai
BroAlanTaps
AI & ML interests
CV(2D-Object Detection), NLP(Infer Accelerate)
Recent Activity
updated
a collection
21 days ago
PCC-Finetuned
updated
a collection
21 days ago
PCC-Pretrained
updated
a model
22 days ago
BroAlanTaps/Qwen3-Agentic-9B
Organizations
models
162
BroAlanTaps/Qwen3-Agentic-9B
Text Generation
•
9B
•
Updated
•
64
BroAlanTaps/Stage2-PCC-Large-16x
Updated
BroAlanTaps/Stage2-PCC-Large-128x
Updated
BroAlanTaps/Stage2-PCC-Large-4x
Updated
BroAlanTaps/PCC-Large-Encoder-Llama3-8B-Instruct
8B
•
Updated
•
38
BroAlanTaps/Stage1-PCC-Large-128x
Updated
BroAlanTaps/Stage2-PCC-Lite-256x
0.8B
•
Updated
BroAlanTaps/Stage2-PCC-Lite-128x
0.8B
•
Updated
•
10
BroAlanTaps/Stage2-PCC-Lite-64x
0.8B
•
Updated
BroAlanTaps/Stage2-PCC-Lite-16x
0.8B
•
Updated
•
6
datasets
9
BroAlanTaps/pretrain-corpus-test
Viewer
•
Updated
•
576
•
10
BroAlanTaps/GPT2-Large-Llama3-8B-fineweb-256-5Btokens
Viewer
•
Updated
•
19.2M
•
397
BroAlanTaps/efficiency_samples_8k
Viewer
•
Updated
•
1k
•
8
BroAlanTaps/Stage2-PCC-Large-SFT-Squad
Viewer
•
Updated
•
92.7k
•
14
BroAlanTaps/Stage2-PCC-SFT-HotpotQA
Viewer
•
Updated
•
90.4k
•
10
BroAlanTaps/Stage2-PCC-Lite-SFT-Squad
Viewer
•
Updated
•
92.7k
•
22
BroAlanTaps/Pretrain-Stage1-1024-new
Viewer
•
Updated
•
1.69M
•
118
BroAlanTaps/Pretrain-Stage1-512
Viewer
•
Updated
•
6.08M
•
194
BroAlanTaps/Pretrain-Stage1-1024
Viewer
•
Updated
•
1.69M
•
159