SuperMicro Demo
🐨
Ask questions about Supermicro documents
Vectara
company
AI & ML interests
retrieval augmented generation, grounded generation, large language models, LLMs, question answering, chatbot
Recent Activity
vectara's activity
ofermend
published
a
Space
about 21 hours ago
ofermend
updated
4
Spaces
1 day ago
stsui96
updated
a
dataset
1 day ago
joao-vectara
updated
a
Space
4 days ago
stsui96
updated
a
Space
7 days ago
clefourrier
posted
an
update
8 days ago
Post
1735
Gemma3 family is out! Reading the tech report, and this section was really interesting to me from a methods/scientific fairness pov.
Instead of doing over-hyped comparisons, they clearly state that **results are reported in a setup which is advantageous to their models**.
(Which everybody does, but people usually don't say)
For a tech report, it makes a lot of sense to report model performance when used optimally!
On leaderboards on the other hand, comparison will be apples to apples, but in a potentially unoptimal way for a given model family (like some user interact sub-optimally with models)
Also contains a cool section (6) on training data memorization rate too! Important to see if your model will output the training data it has seen as such: always an issue for privacy/copyright/... but also very much for evaluation!
Because if your model knows its evals by heart, you're not testing for generalization.
Instead of doing over-hyped comparisons, they clearly state that **results are reported in a setup which is advantageous to their models**.
(Which everybody does, but people usually don't say)
For a tech report, it makes a lot of sense to report model performance when used optimally!
On leaderboards on the other hand, comparison will be apples to apples, but in a potentially unoptimal way for a given model family (like some user interact sub-optimally with models)
Also contains a cool section (6) on training data memorization rate too! Important to see if your model will output the training data it has seen as such: always an issue for privacy/copyright/... but also very much for evaluation!
Because if your model knows its evals by heart, you're not testing for generalization.
joao-vectara
published
a
Space
14 days ago
ofermend
published
a
Space
14 days ago
nthakur
authored
a
paper
24 days ago
clefourrier
authored
a
paper
about 1 month ago
clefourrier
authored
a
paper
3 months ago
nthakur
authored
a
paper
4 months ago
nthakur
authored
a
paper
7 months ago
Post
1831
I'm excited to share our updated hallucination evaluation model (called HHEM-2.1-Open) as well as the updated leaderboard that ranks LLM by the propensity to hallucinate.
vectara/Hallucination-evaluation-leaderboard
vectara/Hallucination-evaluation-leaderboard
- 1 reply
clefourrier
authored
2
papers
9 months ago
Post
1756
If you are a debate fan or did this as an extracurricular activity as a kid, you might have fun with this demo - debate bot. Debate against AI/RAG:
vectara/debate-bot
vectara/debate-bot
-
- 4 replies