CohereForAI/c4ai-command-r7b-12-2024
Text Generation
•
Updated
•
29.1k
•
345
Shubham
shubhamg2208
AI & ML interests
NLP
Graphs
Recent Activity
liked
a model
about 1 month ago
CohereForAI/c4ai-command-r7b-12-2024
liked
a Space
7 months ago
openlifescienceai/open_medical_llm_leaderboard
liked
a model
7 months ago
jasperai/flash-sd3
Organizations
shubhamg2208's activity
liked
a
model
about 1 month ago
liked
a
Space
7 months ago
liked
a
model
7 months ago
liked
a
Space
10 months ago
reacted to
santiviquez's
post with ❤️
11 months ago
Post
Eigenvalues to the rescue? 🛟🤔
I found out about this paper thanks to @gsarti 's post from last week; I got curious, so I want to post my take on it. 🤗
The paper proposes a new metric called EigenScore to detect LLM hallucinations. 📄
Their idea is that given an input question, they generate K different answers, take their internal embedding states, calculate a covariance matrix with them, and use it to calculate an EigenScore.
We can think of the EigenScore as the mean of the eigenvalues of the covariance matrix of the embedding space of the K-generated answers.
❓But why eigenvalues?
Well, if the K generations have similar semantics, the sentence embeddings will be highly correlated, and most eigenvalues will be close to 0.
On the other hand, if the LLM hallucinates, the K generations will have diverse semantics, and the eigenvalues will be significantly different from 0.
The idea is pretty neat and shows better results when compared to other methods like sequence probabilities, length-normalized entropy, and other uncertainty quantification-based methods.
💭 What I'm personally missing from the paper is that they don't compare their results with other methods like LLM-Eval and SelfcheckGPT. They do mention that EigenScore is much cheaper to implement than SelfcheckGPT, but that's all on the topic.
Paper: INSIDE: LLMs' Internal States Retain the Power of Hallucination Detection (2402.03744)
I found out about this paper thanks to @gsarti 's post from last week; I got curious, so I want to post my take on it. 🤗
The paper proposes a new metric called EigenScore to detect LLM hallucinations. 📄
Their idea is that given an input question, they generate K different answers, take their internal embedding states, calculate a covariance matrix with them, and use it to calculate an EigenScore.
We can think of the EigenScore as the mean of the eigenvalues of the covariance matrix of the embedding space of the K-generated answers.
❓But why eigenvalues?
Well, if the K generations have similar semantics, the sentence embeddings will be highly correlated, and most eigenvalues will be close to 0.
On the other hand, if the LLM hallucinates, the K generations will have diverse semantics, and the eigenvalues will be significantly different from 0.
The idea is pretty neat and shows better results when compared to other methods like sequence probabilities, length-normalized entropy, and other uncertainty quantification-based methods.
💭 What I'm personally missing from the paper is that they don't compare their results with other methods like LLM-Eval and SelfcheckGPT. They do mention that EigenScore is much cheaper to implement than SelfcheckGPT, but that's all on the topic.
Paper: INSIDE: LLMs' Internal States Retain the Power of Hallucination Detection (2402.03744)
upvoted
a
paper
11 months ago
reacted to
DmitryRyumin's
post with ❤️
11 months ago
Post
🌟✨ Exciting Announcement: NVIDIA AI Foundation Models ✨🌟
🚀 Interact effortlessly with the latest SOTA AI model APIs, all optimized on the powerful NVIDIA accelerated computing stack-right from your browser! 💻⚡
🔗 Web Page: https://catalog.ngc.nvidia.com/ai-foundation-models
🌟🎯 Favorites:
🔹 Code Generation:
1️⃣ Code Llama 70B 📝🔥: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-foundation/models/codellama-70b
Model 🤖: codellama/CodeLlama-70b-hf
🔹 Text and Code Generation:
1️⃣ Gemma 7B 💬💻: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-foundation/models/gemma-7b
Model 🤖: google/gemma-7b
2️⃣ Yi-34B 📚💡: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-foundation/models/yi-34b
Model 🤖: 01-ai/Yi-34B
🔹 Text Generation:
1️⃣ Mamba-Chat 💬🐍: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-foundation/models/mamba-chat
Model 🤖: havenhq/mamba-chat
2️⃣ Llama 2 70B 📝🦙: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-foundation/models/llama2-70b
Model 🤖: meta-llama/Llama-2-70b
🔹 Text-To-Text Translation:
1️⃣ SeamlessM4T V2 🌐🔄: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-foundation/models/seamless-m4t2-t2tt
Model 🤖: facebook/seamless-m4t-v2-large
🔹 Image Generation:
1️⃣ Stable Diffusion XL 🎨🔍: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-foundation/models/sdxl
🔹 Image Conversation:
1️⃣ NeVA-22B 🗨️📸: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-foundation/models/neva-22b
🔹 Image Classification and Object Detection:
1️⃣ CLIP 🖼️🔍: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-foundation/models/clip
🔹 Voice Conversion:
1️⃣ Maxine Voice Font 🗣️🎶: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-foundation/models/voice-font
🔹 Multimodal LLM (MLLM):
1️⃣ Kosmos-2 🌐👁️: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-foundation/models/kosmos-2
🚀 Interact effortlessly with the latest SOTA AI model APIs, all optimized on the powerful NVIDIA accelerated computing stack-right from your browser! 💻⚡
🔗 Web Page: https://catalog.ngc.nvidia.com/ai-foundation-models
🌟🎯 Favorites:
🔹 Code Generation:
1️⃣ Code Llama 70B 📝🔥: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-foundation/models/codellama-70b
Model 🤖: codellama/CodeLlama-70b-hf
🔹 Text and Code Generation:
1️⃣ Gemma 7B 💬💻: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-foundation/models/gemma-7b
Model 🤖: google/gemma-7b
2️⃣ Yi-34B 📚💡: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-foundation/models/yi-34b
Model 🤖: 01-ai/Yi-34B
🔹 Text Generation:
1️⃣ Mamba-Chat 💬🐍: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-foundation/models/mamba-chat
Model 🤖: havenhq/mamba-chat
2️⃣ Llama 2 70B 📝🦙: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-foundation/models/llama2-70b
Model 🤖: meta-llama/Llama-2-70b
🔹 Text-To-Text Translation:
1️⃣ SeamlessM4T V2 🌐🔄: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-foundation/models/seamless-m4t2-t2tt
Model 🤖: facebook/seamless-m4t-v2-large
🔹 Image Generation:
1️⃣ Stable Diffusion XL 🎨🔍: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-foundation/models/sdxl
🔹 Image Conversation:
1️⃣ NeVA-22B 🗨️📸: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-foundation/models/neva-22b
🔹 Image Classification and Object Detection:
1️⃣ CLIP 🖼️🔍: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-foundation/models/clip
🔹 Voice Conversion:
1️⃣ Maxine Voice Font 🗣️🎶: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-foundation/models/voice-font
🔹 Multimodal LLM (MLLM):
1️⃣ Kosmos-2 🌐👁️: https://catalog.ngc.nvidia.com/orgs/nvidia/teams/ai-foundation/models/kosmos-2
- 2 replies
upvoted
a
collection
11 months ago
liked
2
models
over 1 year ago
