Spaces:

Salesforce
/

crm_llm_leaderboard

Running

App Files Files Community

yibum commited on Jul 2

Commit

6584952

•

1 Parent(s): c187ec2

update latency to better fit use scenario

Browse files

Files changed (2) hide show

.gitignore +1 -0
crm-results/hf_leaderboard_latency_cost.csv +37 -37

.gitignore CHANGED Viewed

@@ -11,3 +11,4 @@ eval-results/
 eval-queue-bk/
 eval-results-bk/
 logs/

 eval-queue-bk/
 eval-results-bk/
 logs/
+**/.DS_Store

crm-results/hf_leaderboard_latency_cost.csv CHANGED Viewed

@@ -1,37 +1,37 @@
-Model Name,Cost and Speed: Flavor,Version,Platform,Response Time (Sec),Mean Output Tokens,Mean Cost per 1K Requests,Cost Band
-AI21 Jamba-Instruct,Long,,AI21,4.0,232.9,1.6,Medium
-AI21 Jamba-Instruct,Short,,AI21,4.0,243.9,0.5,Low
-Claude 3 Haiku,Long,,Bedrock,2.8,236.9,1.0,Medium
-Claude 3 Haiku,Short,,Bedrock,2.2,245.4,0.4,Low
-Claude 3 Opus,Long,,Bedrock,12.2,242.7,61.1,High
-Claude 3 Opus,Short,,Bedrock,8.4,243.2,25.4,High
-Cohere Command R+,Long,,Bedrock,7.7,245.7,11.7,High
-Cohere Command R+,Short,,Bedrock,7.1,249.9,5.1,High
-Cohere Command Text,Long,,Bedrock,12.9,238.7,4.3,High
-Cohere Command Text,Short,,Bedrock,9.6,245.6,1.1,Medium
-Gemini Pro 1.5,Long,,Google,5.5,245.7,11.0,High
-Gemini Pro 1.5,Short,,Google,5.4,247.5,3.3,Medium
-Gemini Pro 1,Long,,Google,6.0,228.9,1.7,Medium
-Gemini Pro 1,Short,,Google,4.4,247.4,0.6,Low
-GPT 3.5 Turbo,Long,,OpenAI,4.5,249.9,1.6,Medium
-GPT 3.5 Turbo,Short,,OpenAI,4.2,238.3,0.6,Low
-GPT 4 Turbo,Long,,OpenAI,12.3,247.6,32.0,High
-GPT 4 Turbo,Short,,OpenAI,12.3,250.0,11.7,High
-GPT4-o,Long,,OpenAI,5.1,248.4,15.9,High
-GPT4-o,Short,,OpenAI,5.0,250.0,5.8,High
-Mistral 7B,Long,Mistral-7B-Instruct-v0.2,Self-host (g5.48xlarge),8.83,242.0,0.5,Low
-Mistral 7B,Short,Mistral-7B-Instruct-v0.2,Self-host (g5.48xlarge),8.31,247.0,0.1,Low
-LLaMA 3 8B,Long,Meta-Llama-3-8B-Instruct,Self-host (g5.48xlarge),3.76,251.5,1.1,Medium
-LLaMA 3 8B,Short,Meta-Llama-3-8B-Instruct,Self-host (g5.48xlarge),3.23,243.6,0.3,Low
-LLaMA 3 70B,Long,llama-3-70b-instruct,Self-host (p4d.24xlarge),20.1,243.9,8.8,High
-LLaMA 3 70B,Short,llama-3-70b-instruct,Self-host (p4d.24xlarge),29.4,251.2,2.2,Medium
-Mixtral 8x7B,Long,mixtral-8x7b-instruct,Self-host (p4d.24xlarge),2.44,248.5,1.5,Medium
-Mixtral 8x7B,Short,mixtral-8x7b-instruct,Self-host (p4d.24xlarge),2.41,250.0,0.4,Low
-SF-TextBase 7B,Long,CRM-TextBase-7b-22k-g5 (endpoint),Self-host (g5.48xlarge),8.99,248.5,0.5,Low
-SF-TextBase 7B,Short,CRM-TextBase-7b-22k-g5 (endpoint),Self-host (g5.48xlarge),8.29,248.7,0.1,Low
-SF-TextBase 70B,Long,TextBase-70B-8K,Self-host (p4de.24xlarge),6.52,253.7,8.8,High
-SF-TextBase 70B,Short,TextBase-70B-8K,Self-host (p4de.24xlarge),6.24,249.7,2.2,Medium
-SF-TextSum,Long,CRM-TSUM-7b-22k-g5 (endpoint),Self-host (g5.48xlarge),8.85,244.0,0.5,Low
-SF-TextSum,Short,CRM-TSUM-7b-22k-g5 (endpoint),Self-host (g5.48xlarge),8.34,250.4,0.1,Low
-XGen 2,Long,EinsteinXgen2E4DSStreaming (endpoint),Self-host (p4de.24xlarge),3.71,250.0,2.9,Medium
-XGen 2,Short,EinsteinXgen2E4DSStreaming (endpoint),Self-host (p4de.24xlarge),2.64,250.0,0.7,Medium

+Model Name,Cost and Speed: Flavor,Version,Platform,Mean Output Tokens,Mean Cost per 1K Requests,Cost Band,Response Time (Sec)
+AI21 Jamba-Instruct,Long,,AI21,232.9,1.6,Medium,4.0
+AI21 Jamba-Instruct,Short,,AI21,243.9,0.5,Low,4.0
+Claude 3 Haiku,Long,,Bedrock,236.9,1.0,Medium,2.8
+Claude 3 Haiku,Short,,Bedrock,245.4,0.4,Low,2.2
+Claude 3 Opus,Long,,Bedrock,242.7,61.1,High,12.2
+Claude 3 Opus,Short,,Bedrock,243.2,25.4,High,8.4
+Cohere Command R+,Long,,Bedrock,245.7,11.7,High,7.7
+Cohere Command R+,Short,,Bedrock,249.9,5.1,High,7.1
+Cohere Command Text,Long,,Bedrock,238.7,4.3,High,12.9
+Cohere Command Text,Short,,Bedrock,245.6,1.1,Medium,9.6
+Gemini Pro 1.5,Long,,Google,245.7,11.0,High,5.5
+Gemini Pro 1.5,Short,,Google,247.5,3.3,Medium,5.4
+Gemini Pro 1,Long,,Google,228.9,1.7,Medium,6.0
+Gemini Pro 1,Short,,Google,247.4,0.6,Low,4.4
+GPT 3.5 Turbo,Long,,OpenAI,249.9,1.6,Medium,4.5
+GPT 3.5 Turbo,Short,,OpenAI,238.3,0.6,Low,4.2
+GPT 4 Turbo,Long,,OpenAI,247.6,32.0,High,12.3
+GPT 4 Turbo,Short,,OpenAI,250.0,11.7,High,12.3
+GPT4-o,Long,,OpenAI,248.4,15.9,High,5.1
+GPT4-o,Short,,OpenAI,250.0,5.8,High,5.0
+Mistral 7B,Long,Mistral-7B-Instruct-v0.2,Self-host (g5.48xlarge),242.0,16.5,Low,3.74
+Mistral 7B,Short,Mistral-7B-Instruct-v0.2,Self-host (g5.48xlarge),247.0,15.5,Low,3.68
+LLaMA 3 8B,Long,Meta-Llama-3-8B-Instruct,Self-host (g5.48xlarge),251.5,7.0,Medium,3.42
+LLaMA 3 8B,Short,Meta-Llama-3-8B-Instruct,Self-host (g5.48xlarge),243.6,6.0,Low,3.33
+LLaMA 3 70B,Long,llama-3-70b-instruct,Self-host (p4d.24xlarge),243.9,67.7,High,6.78
+LLaMA 3 70B,Short,llama-3-70b-instruct,Self-host (p4d.24xlarge),251.2,99.0,Medium,6.25
+Mixtral 8x7B,Long,mixtral-8x7b-instruct,Self-host (p4d.24xlarge),248.5,8.22,Medium,4.90
+Mixtral 8x7B,Short,mixtral-8x7b-instruct,Self-host (p4d.24xlarge),250.0,8.11,Low,4.54
+SF-TextBase 7B,Long,CRM-TextBase-7b-22k-g5 (endpoint),Self-host (g5.48xlarge),248.5,16.80,Low,8.99
+SF-TextBase 7B,Short,CRM-TextBase-7b-22k-g5 (endpoint),Self-host (g5.48xlarge),248.7,15.50,Low,8.29
+SF-TextBase 70B,Long,TextBase-70B-8K,Self-host (p4de.24xlarge),253.7,28.17,High,6.52
+SF-TextBase 70B,Short,TextBase-70B-8K,Self-host (p4de.24xlarge),249.7,26.96,Medium,6.24
+SF-TextSum,Long,CRM-TSUM-7b-22k-g5 (endpoint),Self-host (g5.48xlarge),244.0,16.55,Low,3.43
+SF-TextSum,Short,CRM-TSUM-7b-22k-g5 (endpoint),Self-host (g5.48xlarge),250.4,15.60,Low,3.38
+XGen 2,Long,EinsteinXgen2E4DSStreaming (endpoint),Self-host (p4de.24xlarge),250.0,16.03,Medium,5.04
+XGen 2,Short,EinsteinXgen2E4DSStreaming (endpoint),Self-host (p4de.24xlarge),250.0,11.40,Medium,4.66