Hugging Face's logo

LorMolf
/
mnrl-toolbench-bge-base-en-v1.5

Sentence Similarity
sentence-transformers
Safetensors
bert
feature-extraction
Generated from Trainer
dataset_size:30000
loss:MultipleNegativesRankingLoss
Eval Results
text-embeddings-inference
Model card Files Community
LorMolf commited on
Commit
1141abe
·
verified ·
1 Parent(s): d0a7bec

Add final trained model

Browse files
Files changed (2) hide show
  1. README.md +140 -146
  2. model.safetensors +1 -1
README.md CHANGED
@@ -8,15 +8,60 @@ tags:
8
  - loss:MultipleNegativesRankingLoss
9
  base_model: BAAI/bge-base-en-v1.5
10
  widget:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
11
  - source_sentence: I'm planning a surprise birthday party for my friend next week
12
  and I want to gather some interesting facts and news articles about birthdays.
13
  Can you provide me with random birthday facts and the latest news articles related
14
  to birthdays from different sources? Additionally, please recommend some popular
15
  party venues and catering services in my area.
16
  sentences:
17
- - "def numbers_get_trivia_fact:\n\t\"\"\"\n\tDescription:\n\tGet a trivia fact about\
18
- \ a number\n\n\tArguments:\n\t---------\n\t- number : STRING (required)\n\t Description:\
19
- \ The integer of interest\n\t Default: 42\n\t\"\"\""
20
  - "def reuters_business_and_financial_news_get_article_by_category_id_and_article_date:\n\
21
  \t\"\"\"\n\tDescription:\n\tGet Article by category id and article date\n\tex\
22
  \ :/api/v1/category-id-8/article-date-11-04-2021\n\t\n\tcategory - category id\
@@ -24,76 +69,25 @@ widget:
24
  \t- category : string (required)\n\t Default: 8\n\t- date : string (required)\n\
25
  \t Default: 11-04-2021\n\t- category-id : STRING (required)\n\t Default: 8\n\
26
  \t- ArticleDate : STRING (required)\n\t Default: 11-04-2021\n\t\"\"\""
27
- - "def wealth_reader_api__error_codes:\n\t\"\"\"\n\tDescription:\n\tListado de códigos\
28
- \ de error. Presta especial atención a que no todos los códigos de error deben\
29
- \ recibir el mismo tratamiento por parte de tu aplicación. Ante un error de password\
30
- \ incorrecto no debes reintentar la llamada con los mismos parámetros, pero ante\
31
- \ un error que te indique que la entidad está en mantenimiento sí puedes reintentarlo.\
32
- \ Pide una sesión técnica con nuestro equipo para resolver cualquier duda sobre\
33
- \ la gestión de errores.\n\t\n\n\tArguments:\n\t---------\n\t\"\"\""
34
- - source_sentence: I'm conducting research on the NFT market. Could you fetch the
35
- top-selling NFTs today and the volume and trades of the top trending collections
36
- this month? This information will be valuable for my analysis.
37
- sentences:
38
- - "def top_nft_sales_top_nfts_today:\n\t\"\"\"\n\tDescription:\n\tTop selling NFTs\
39
- \ today\n\n\tArguments:\n\t---------\n\t\"\"\""
40
- - "def quotsy_get_quotes:\n\t\"\"\"\n\tDescription:\n\treturn qoutes\n\n\tArguments:\n\
41
  \t---------\n\t\"\"\""
42
- - "def manga_scrapper_chapters_list_paginated:\n\t\"\"\"\n\tDescription:\n\tMake\
43
- \ request to fetch chapter collection for a specific webtoon from a specific provider.\n\
44
- \n\tArguments:\n\t---------\n\t- provider : STRING (required)\n\t Description:\
45
- \ Specify the webtoon provider' slug. See /providers for the provider list.\n\t\
46
- \ Default: cosmic\n\t- webtoon : STRING (required)\n\t Description: Specify\
47
- \ the webtoon's slug. See /webtoons for the webtoon list.\n\t Default: eleceed\n\
48
- \t- limit : NUMBER (required)\n\t Description: Number of results per page, between\
49
- \ 1 - 20.\n\t Default: 10\n\t- page : NUMBER (required)\n\t Description: Specify\
50
- \ the page to fetch.\n\t Default: 1\n\t\"\"\""
51
- - source_sentence: My friend is visiting me in Los Angeles and I want to show her
52
- around. Can you recommend some popular tourist attractions in the city? Additionally,
53
- provide me with the latest news about Los Angeles.
54
- sentences:
55
- - "def 360_business_tool_getcompaniessince:\n\t\"\"\"\n\tDescription:\n\tGet companies\
56
- \ modified since\n\n\tArguments:\n\t---------\n\t- timestamp : NUMBER (required)\n\
57
- \t\"\"\""
58
- - "def coinranking_get_coins_index:\n\t\"\"\"\n\tDescription:\n\tList of all coins\
59
- \ currently available on coinranking, for indexing purposes.\n\tThis endpoint\
60
- \ requires the **ultra** plan or higher.\n\n\tArguments:\n\t---------\n\t\"\"\""
61
- - "def ski_resort_forecast_current_snow_conditions:\n\t\"\"\"\n\tDescription:\n\t\
62
- Returns the current snow conditions for a given resort name\n\n\tArguments:\n\t\
63
- ---------\n\t- resort : string (required)\n\t Default: Jackson Hole\n\t\"\"\""
64
- - source_sentence: Hello, I'm an NFT enthusiast and I'm eager to explore the latest
65
- NFT news. Can you fetch the most recent news articles for me? Additionally, I'd
66
- like to check out the welcome page of the NFT API News tool to learn more about
67
- its functionalities.
68
- sentences:
69
- - "def nft_api_news_nft_news:\n\t\"\"\"\n\tDescription:\n\tThis is where you get\
70
- \ all the up to date NFT news. This is a live feed and is updated daily.\n\n\t\
71
- Arguments:\n\t---------\n\t\"\"\""
72
- - "def google_trends_trendings:\n\t\"\"\"\n\tDescription:\n\tThe endpoint used to\
73
- \ display some of the trending search keywords on Google in a specific region\
74
- \ and on a specific date.\n\n\tArguments:\n\t---------\n\t- date : DATE (YYYY-MM-DD)\
75
- \ (required)\n\t Description: To display trend data for a specific date\n\t-\
76
- \ region_code : STRING (required)\n\t Description: The region_code parameter\
77
- \ is used to display data only for the specified region.\n\tExample: **GB**, **ID**,\
78
- \ **US**, etc.\n\tTo view the list of supported regions, please check the **/regions**\
79
- \ endpoint.\n\t Default: US\n\t\"\"\""
80
- - "def apimail10_getnewemail10:\n\t\"\"\"\n\tDescription:\n\tget New Email 10\n\n\
81
- \tArguments:\n\t---------\n\t\"\"\""
82
- - source_sentence: I need to know the team information for the Seattle Storm. Also,
83
- please give me the list of all WNBA teams and the schedule data for the game on
84
- June 15, 2022.
85
  sentences:
86
- - "def soundcloud__song_info:\n\t\"\"\"\n\tDescription:\n\tGet basic information\
87
- \ of a song.\n\n\tArguments:\n\t---------\n\t- track_url : STRING (required)\n\
88
- \t Default: https://soundcloud.com/user-977421934/the-phoenix\n\t\"\"\""
89
- - "def node_express_api_tutorial_get_specific_user:\n\t\"\"\"\n\tDescription:\n\t\
90
- This endpoint will return back the user with the specified ID.\n\n\tArguments:\n\
91
- \t---------\n\t- id : string (required)\n\t\"\"\""
92
- - "def wnba_api_wnba_schedule:\n\t\"\"\"\n\tDescription:\n\tThis endpoint provides\
93
- \ you with the WNBA schedule data for a specified date as long as it is available.\n\
94
- \n\tArguments:\n\t---------\n\t- month : STRING (required)\n\t Default: 05\n\t\
95
- - day : STRING (required)\n\t Default: 05\n\t- year : STRING (required)\n\t \
96
- \ Default: 2022\n\t\"\"\""
97
  pipeline_tag: sentence-similarity
98
  library_name: sentence-transformers
99
  metrics:
@@ -126,58 +120,58 @@ model-index:
126
  type: dev
127
  metrics:
128
  - type: cosine_accuracy@1
129
- value: 0.7298969072164948
130
  name: Cosine Accuracy@1
131
  - type: cosine_accuracy@3
132
  value: 0.8494845360824742
133
  name: Cosine Accuracy@3
134
  - type: cosine_accuracy@5
135
- value: 0.8948453608247423
136
  name: Cosine Accuracy@5
137
  - type: cosine_accuracy@10
138
- value: 0.9463917525773196
139
  name: Cosine Accuracy@10
140
  - type: cosine_precision@1
141
- value: 0.7298969072164948
142
  name: Cosine Precision@1
143
  - type: cosine_precision@3
144
- value: 0.4618556701030928
145
  name: Cosine Precision@3
146
  - type: cosine_precision@5
147
- value: 0.3138144329896908
148
  name: Cosine Precision@5
149
  - type: cosine_precision@10
150
- value: 0.1762886597938145
151
  name: Cosine Precision@10
152
  - type: cosine_recall@1
153
- value: 0.4180756013745704
154
  name: Cosine Recall@1
155
  - type: cosine_recall@3
156
- value: 0.718213058419244
157
  name: Cosine Recall@3
158
  - type: cosine_recall@5
159
- value: 0.7950171821305843
160
  name: Cosine Recall@5
161
  - type: cosine_recall@10
162
- value: 0.8808247422680413
163
  name: Cosine Recall@10
164
  - type: cosine_ndcg@1
165
- value: 0.7298969072164948
166
  name: Cosine Ndcg@1
167
  - type: cosine_ndcg@3
168
- value: 0.7120713819724622
169
  name: Cosine Ndcg@3
170
  - type: cosine_ndcg@5
171
- value: 0.7468995524637367
172
  name: Cosine Ndcg@5
173
  - type: cosine_ndcg@10
174
- value: 0.7817739105126679
175
  name: Cosine Ndcg@10
176
  - type: cosine_mrr@10
177
- value: 0.8003616429389625
178
  name: Cosine Mrr@10
179
  - type: cosine_map@100
180
- value: 0.7269297341883608
181
  name: Cosine Map@100
182
  ---
183
 
@@ -231,9 +225,9 @@ from sentence_transformers import SentenceTransformer
231
  model = SentenceTransformer("LorMolf/mnrl-toolbench-bge-base-en-v1.5")
232
  # Run inference
233
  sentences = [
234
- 'I need to know the team information for the Seattle Storm. Also, please give me the list of all WNBA teams and the schedule data for the game on June 15, 2022.',
235
- 'def wnba_api_wnba_schedule:\n\t"""\n\tDescription:\n\tThis endpoint provides you with the WNBA schedule data for a specified date as long as it is available.\n\n\tArguments:\n\t---------\n\t- month : STRING (required)\n\t Default: 05\n\t- day : STRING (required)\n\t Default: 05\n\t- year : STRING (required)\n\t Default: 2022\n\t"""',
236
- 'def node_express_api_tutorial_get_specific_user:\n\t"""\n\tDescription:\n\tThis endpoint will return back the user with the specified ID.\n\n\tArguments:\n\t---------\n\t- id : string (required)\n\t"""',
237
  ]
238
  embeddings = model.encode(sentences)
239
  print(embeddings.shape)
@@ -278,26 +272,26 @@ You can finetune this model on your own dataset.
278
  * Dataset: `dev`
279
  * Evaluated with <code>src.port.retrieval_evaluator.DeviceAwareInformationRetrievalEvaluator</code>
280
 
281
- | Metric | Value |
282
- |:--------------------|:-----------|
283
- | cosine_accuracy@1 | 0.7299 |
284
- | cosine_accuracy@3 | 0.8495 |
285
- | cosine_accuracy@5 | 0.8948 |
286
- | cosine_accuracy@10 | 0.9464 |
287
- | cosine_precision@1 | 0.7299 |
288
- | cosine_precision@3 | 0.4619 |
289
- | cosine_precision@5 | 0.3138 |
290
- | cosine_precision@10 | 0.1763 |
291
- | cosine_recall@1 | 0.4181 |
292
- | cosine_recall@3 | 0.7182 |
293
- | cosine_recall@5 | 0.795 |
294
- | cosine_recall@10 | 0.8808 |
295
- | cosine_ndcg@1 | 0.7299 |
296
- | cosine_ndcg@3 | 0.7121 |
297
- | cosine_ndcg@5 | 0.7469 |
298
- | **cosine_ndcg@10** | **0.7818** |
299
- | cosine_mrr@10 | 0.8004 |
300
- | cosine_map@100 | 0.7269 |
301
 
302
  <!--
303
  ## Bias, Risks and Limitations
@@ -323,13 +317,13 @@ You can finetune this model on your own dataset.
323
  | | sentence_0 | sentence_1 | sentence_2 |
324
  |:--------|:------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
325
  | type | string | string | string |
326
- | details | <ul><li>min: 22 tokens</li><li>mean: 59.32 tokens</li><li>max: 163 tokens</li></ul> | <ul><li>min: 27 tokens</li><li>mean: 73.59 tokens</li><li>max: 512 tokens</li></ul> | <ul><li>min: 27 tokens</li><li>mean: 73.19 tokens</li><li>max: 512 tokens</li></ul> |
327
  * Samples:
328
- | sentence_0 | sentence_1 | sentence_2 |
329
- |:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
330
- | <code>I am planning a trip to Paris from July 10th to July 15th. Can you provide me with the working hours for this period, considering the Federal holidays in France? Also, recommend some events happening in Paris during this time and send me the calendar invites for these events.</code> | <code>def working_days__1_3_add_working_hours:<br> """<br> Description:<br> Add an amount of working time to a given start date/time<br><br> Arguments:<br> ---------<br> - start_date : STRING (required)<br> Description: The start date (YYYY-MM-DD)<br> Default: 2013-12-31<br> - country_code : STRING (required)<br> Description: The ISO country code (2 letters). See <a href=https://api.workingdays.org/api-countries >available countries & configurations</a><br> Default: US<br> - start_time : STRING (required)<br> Description: The start time in a 24 hours format with leading zeros.<br> Default: 08:15<br> """</code> | <code>def realty_in_us_agents_get_reviews:<br> """<br> Description:<br> Get agent reviews<br><br> Arguments:<br> ---------<br> - advertiser_id : NUMBER (required)<br> Description: The value of advertiser_id field returned in .../agents/list endpoint<br> Default: 1633379<br> """</code> |
331
- | <code>I'm organizing a company event and I need to find a venue that can accommodate 100 people. Can you suggest some event spaces in the city with good reviews? Also, I would like to gather information about nearby transportation options and recommend some local catering services.</code> | <code>def socie_get_members:<br> """<br> Description:<br> Retrieve all or some members of your community.<br><br> Arguments:<br> ---------<br> """</code> | <code>def stocks_get_the_current_stock_by_code:<br> """<br> Description:<br> Get the current stock by code.<br><br> Arguments:<br> ---------<br> - stockCode : string (required)<br> Default: GOOG<br> """</code> |
332
- | <code>I want to surprise my friends with a Netflix binge session and I'm looking for some highly ranked series. Can you provide me with a list of the top 100 ranked Netflix original series? Also, check if the word 'chimpo' is vulgar using the SHIMONETA API.</code> | <code>def shimoneta_send_a_word_to_check:<br> """<br> Description:<br> The API returns what the word means if the word is vulgar.<br><br> Arguments:<br> ---------<br> - word : STRING (required)<br> Default: chimpo<br> """</code> | <code>def bing_web_search_supported_languages:<br> """<br> Description:<br> These languages are supported for to search.<br><br> Arguments:<br> ---------<br> """</code> |
333
  * Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
334
  ```json
335
  {
@@ -472,36 +466,36 @@ You can finetune this model on your own dataset.
472
  | Epoch | Step | dev_cosine_ndcg@10 |
473
  |:------:|:----:|:------------------:|
474
  | -1 | -1 | 0.6750 |
475
- | 0.1875 | 6 | 0.6879 |
476
- | 0.375 | 12 | 0.7226 |
477
- | 0.5625 | 18 | 0.7435 |
478
- | 0.75 | 24 | 0.7592 |
479
- | 0.9375 | 30 | 0.7697 |
480
- | 1.0 | 32 | 0.7720 |
481
- | 1.125 | 36 | 0.7734 |
482
- | 1.3125 | 42 | 0.7727 |
483
  | 1.5 | 48 | 0.7745 |
484
- | 1.6875 | 54 | 0.7817 |
485
- | 1.875 | 60 | 0.7825 |
486
- | 2.0 | 64 | 0.7821 |
487
- | 2.0625 | 66 | 0.7824 |
488
- | 2.25 | 72 | 0.7806 |
489
- | 2.4375 | 78 | 0.7816 |
490
- | 2.625 | 84 | 0.7809 |
491
- | 2.8125 | 90 | 0.7832 |
492
- | 3.0 | 96 | 0.7829 |
493
- | 3.1875 | 102 | 0.7805 |
494
- | 3.375 | 108 | 0.7811 |
495
- | 3.5625 | 114 | 0.7813 |
496
- | 3.75 | 120 | 0.7812 |
497
- | 3.9375 | 126 | 0.7801 |
498
- | 4.0 | 128 | 0.7802 |
499
- | 4.125 | 132 | 0.7812 |
500
- | 4.3125 | 138 | 0.7822 |
501
- | 4.5 | 144 | 0.7818 |
502
- | 4.6875 | 150 | 0.7818 |
503
- | 4.875 | 156 | 0.7818 |
504
- | 5.0 | 160 | 0.7818 |
505
 
506
 
507
  ### Framework Versions
 
8
  - loss:MultipleNegativesRankingLoss
9
  base_model: BAAI/bge-base-en-v1.5
10
  widget:
11
+ - source_sentence: I'm organizing a surprise party for my sister and I need synonyms
12
+ for the word 'celebrate'. Could you also provide the lexical field for the word
13
+ 'birthday' and the Scrabble score for the word 'festivity'? Moreover, search for
14
+ translations of the phrase 'Happy anniversary' from English to French using the
15
+ MyMemory Translation Memory API.
16
+ sentences:
17
+ - "def endlessmedicalapi_getoutcomes:\n\t\"\"\"\n\tDescription:\n\tGetOutcomes\n\
18
+ \n\tArguments:\n\t---------\n\t\"\"\""
19
+ - "def dicolink_get_lexical_field:\n\t\"\"\"\n\tDescription:\n\tGet Lexical Field\
20
+ \ for a word\n\n\tArguments:\n\t---------\n\t- mot : string (required)\n\t Default:\
21
+ \ cheval\n\t\"\"\""
22
+ - "def Indie_Songs_:_DistroKid_&_Unsigned.Get_Top_50_indie_songs:\n\t\"\"\"\n\t\
23
+ Description:\n\tGet TOP 50 indie songs based on their daily stream increase ratio\n\
24
+ \n\tArguments:\n\t---------\n\t\"\"\""
25
+ - source_sentence: I'm planning a family game night and I need some new games to play.
26
+ Can you provide me with the details of a random card from Hearthstone and recommend
27
+ some PlayStation games with good deals?
28
+ sentences:
29
+ - "def playstation_store_deals_api_playstationdeals:\n\t\"\"\"\n\tDescription:\n\
30
+ \tThere is only 1 parameter for this API endpoint.\n\t\n\t1. playstation_deals/?count=0\n\
31
+ \t\n\tcount = 0 (Min is 0, starting of the list. Max value depends on the total\
32
+ \ number of games available.)\n\tNote: Since its a List of Items, If the maximum\
33
+ \ number of games available on deals is 771 then you have to enter (771-1) = 770\
34
+ \ to get the last game on the deal.\n\t\n\tThis will provide you with the game\
35
+ \ data as given below which contains name, price, platform, discount percent,\
36
+ \ discounted price, total no. of games, etc..:\n\t\n\t{\n\t \"name\": \"God of\
37
+ \ War III Remastered\",\n\t \"titleId\": \"CUSA01623_00\",\n\t \"platform\"\
38
+ : [\n\t \"PS4\"\n\t ],\n\t \"basePrice\": \"$19.99\",\n\t \"discountPercent\"\
39
+ : \"-50%\",\n\t \"discountPrice\": \"$9.99\",\n\t \"url\": \"https://store.playstation.com/en-us/product/UP9000-CUSA01623_00-0000GODOFWAR3PS4\"\
40
+ ,\n\t \"Total No. of Games\": 771\n\t}\n\n\tArguments:\n\t---------\n\t- count\
41
+ \ : NUMBER (required)\n\t Default: 0\n\t\"\"\""
42
+ - "def captcha_verify_the_captcha:\n\t\"\"\"\n\tDescription:\n\tVerify the captcha\n\
43
+ \n\tArguments:\n\t---------\n\t- captcha : STRING (required)\n\t Default: Captcha\
44
+ \ Text\n\t- uuid : STRING (required)\n\t Default: UUID\n\t\"\"\""
45
+ - "def teste_getinventory:\n\t\"\"\"\n\tDescription:\n\tReturns a map of status\
46
+ \ codes to quantities\n\n\tArguments:\n\t---------\n\t\"\"\""
47
+ - source_sentence: I'm conducting research on the NFT market. Could you fetch the
48
+ top-selling NFTs today and the volume and trades of the top trending collections
49
+ this month? This information will be valuable for my analysis.
50
+ sentences:
51
+ - "def icai_chartered_accountant_verification_get_call:\n\t\"\"\"\n\tDescription:\n\
52
+ \tUsed to fetch api result using the request id received in responses.\n\n\tArguments:\n\
53
+ \t---------\n\t- request_id : STRING (required)\n\t Default: 68bbb910-da9b-4d8a-9a1d-4bd878b19846\n\
54
+ \t\"\"\""
55
+ - "def top_nft_sales_top_nfts_today:\n\t\"\"\"\n\tDescription:\n\tTop selling NFTs\
56
+ \ today\n\n\tArguments:\n\t---------\n\t\"\"\""
57
+ - "def lorem_ipsum_api_sentence:\n\t\"\"\"\n\tDescription:\n\tCreate lorem ipsum\
58
+ \ by defining the number of sentences\n\n\tArguments:\n\t---------\n\t\"\"\""
59
  - source_sentence: I'm planning a surprise birthday party for my friend next week
60
  and I want to gather some interesting facts and news articles about birthdays.
61
  Can you provide me with random birthday facts and the latest news articles related
62
  to birthdays from different sources? Additionally, please recommend some popular
63
  party venues and catering services in my area.
64
  sentences:
 
 
 
65
  - "def reuters_business_and_financial_news_get_article_by_category_id_and_article_date:\n\
66
  \t\"\"\"\n\tDescription:\n\tGet Article by category id and article date\n\tex\
67
  \ :/api/v1/category-id-8/article-date-11-04-2021\n\t\n\tcategory - category id\
 
69
  \t- category : string (required)\n\t Default: 8\n\t- date : string (required)\n\
70
  \t Default: 11-04-2021\n\t- category-id : STRING (required)\n\t Default: 8\n\
71
  \t- ArticleDate : STRING (required)\n\t Default: 11-04-2021\n\t\"\"\""
72
+ - "def NPS-Net_Promoter_Score.Read_a_survey_NLP:\n\t\"\"\"\n\tDescription:\n\tGet\
73
+ \ a detail of customer survey answer by its survey id (sid), and applies to the\
74
+ \ third answer (a3) the sentiment analysis feature.\n\n\tArguments:\n\t---------\n\
75
+ \t- sid : string (required)\n\t\"\"\""
76
+ - "def bbc_good_food_api_categories_collections_ids:\n\t\"\"\"\n\tDescription:\n\
77
+ \tGet all categories collection with there names and namd id\n\n\tArguments:\n\
 
 
 
 
 
 
 
 
78
  \t---------\n\t\"\"\""
79
+ - source_sentence: I'm a big fan of Peruvian football and I'm curious about the competitions
80
+ and teams of televised football matches in the country. Can you provide me with
81
+ this information? Additionally, fetch me the premium tips and historical results
82
+ from the Betigolo Tips API to enhance my football knowledge and betting strategy.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
83
  sentences:
84
+ - "def car_data_types:\n\t\"\"\"\n\tDescription:\n\tget a list of supported types\n\
85
+ \n\tArguments:\n\t---------\n\t\"\"\""
86
+ - "def climate_change_live_v27_get_all_climate_change_news:\n\t\"\"\"\n\tDescription:\n\
87
+ \tThis endpoint will return back all news about Climate Change from all over the\
88
+ \ world.\n\n\tArguments:\n\t---------\n\t\"\"\""
89
+ - "def betigolo_tips_premium_tips:\n\t\"\"\"\n\tDescription:\n\tList of active Premium\
90
+ \ Tips\n\n\tArguments:\n\t---------\n\t\"\"\""
 
 
 
 
91
  pipeline_tag: sentence-similarity
92
  library_name: sentence-transformers
93
  metrics:
 
120
  type: dev
121
  metrics:
122
  - type: cosine_accuracy@1
123
+ value: 0.7154639175257732
124
  name: Cosine Accuracy@1
125
  - type: cosine_accuracy@3
126
  value: 0.8494845360824742
127
  name: Cosine Accuracy@3
128
  - type: cosine_accuracy@5
129
+ value: 0.8969072164948454
130
  name: Cosine Accuracy@5
131
  - type: cosine_accuracy@10
132
+ value: 0.9381443298969072
133
  name: Cosine Accuracy@10
134
  - type: cosine_precision@1
135
+ value: 0.7154639175257732
136
  name: Cosine Precision@1
137
  - type: cosine_precision@3
138
+ value: 0.45979381443298967
139
  name: Cosine Precision@3
140
  - type: cosine_precision@5
141
+ value: 0.31298969072164956
142
  name: Cosine Precision@5
143
  - type: cosine_precision@10
144
+ value: 0.17608247422680418
145
  name: Cosine Precision@10
146
  - type: cosine_recall@1
147
+ value: 0.407594501718213
148
  name: Cosine Recall@1
149
  - type: cosine_recall@3
150
+ value: 0.7134020618556701
151
  name: Cosine Recall@3
152
  - type: cosine_recall@5
153
+ value: 0.795704467353952
154
  name: Cosine Recall@5
155
  - type: cosine_recall@10
156
+ value: 0.8765292096219931
157
  name: Cosine Recall@10
158
  - type: cosine_ndcg@1
159
+ value: 0.7154639175257732
160
  name: Cosine Ndcg@1
161
  - type: cosine_ndcg@3
162
+ value: 0.7055299224270164
163
  name: Cosine Ndcg@3
164
  - type: cosine_ndcg@5
165
+ value: 0.7418598245527984
166
  name: Cosine Ndcg@5
167
  - type: cosine_ndcg@10
168
+ value: 0.7759821535840169
169
  name: Cosine Ndcg@10
170
  - type: cosine_mrr@10
171
+ value: 0.7923711340206183
172
  name: Cosine Mrr@10
173
  - type: cosine_map@100
174
+ value: 0.7207130597174141
175
  name: Cosine Map@100
176
  ---
177
 
 
225
  model = SentenceTransformer("LorMolf/mnrl-toolbench-bge-base-en-v1.5")
226
  # Run inference
227
  sentences = [
228
+ "I'm a big fan of Peruvian football and I'm curious about the competitions and teams of televised football matches in the country. Can you provide me with this information? Additionally, fetch me the premium tips and historical results from the Betigolo Tips API to enhance my football knowledge and betting strategy.",
229
+ 'def betigolo_tips_premium_tips:\n\t"""\n\tDescription:\n\tList of active Premium Tips\n\n\tArguments:\n\t---------\n\t"""',
230
+ 'def climate_change_live_v27_get_all_climate_change_news:\n\t"""\n\tDescription:\n\tThis endpoint will return back all news about Climate Change from all over the world.\n\n\tArguments:\n\t---------\n\t"""',
231
  ]
232
  embeddings = model.encode(sentences)
233
  print(embeddings.shape)
 
272
  * Dataset: `dev`
273
  * Evaluated with <code>src.port.retrieval_evaluator.DeviceAwareInformationRetrievalEvaluator</code>
274
 
275
+ | Metric | Value |
276
+ |:--------------------|:----------|
277
+ | cosine_accuracy@1 | 0.7155 |
278
+ | cosine_accuracy@3 | 0.8495 |
279
+ | cosine_accuracy@5 | 0.8969 |
280
+ | cosine_accuracy@10 | 0.9381 |
281
+ | cosine_precision@1 | 0.7155 |
282
+ | cosine_precision@3 | 0.4598 |
283
+ | cosine_precision@5 | 0.313 |
284
+ | cosine_precision@10 | 0.1761 |
285
+ | cosine_recall@1 | 0.4076 |
286
+ | cosine_recall@3 | 0.7134 |
287
+ | cosine_recall@5 | 0.7957 |
288
+ | cosine_recall@10 | 0.8765 |
289
+ | cosine_ndcg@1 | 0.7155 |
290
+ | cosine_ndcg@3 | 0.7055 |
291
+ | cosine_ndcg@5 | 0.7419 |
292
+ | **cosine_ndcg@10** | **0.776** |
293
+ | cosine_mrr@10 | 0.7924 |
294
+ | cosine_map@100 | 0.7207 |
295
 
296
  <!--
297
  ## Bias, Risks and Limitations
 
317
  | | sentence_0 | sentence_1 | sentence_2 |
318
  |:--------|:------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
319
  | type | string | string | string |
320
+ | details | <ul><li>min: 22 tokens</li><li>mean: 59.32 tokens</li><li>max: 163 tokens</li></ul> | <ul><li>min: 27 tokens</li><li>mean: 73.59 tokens</li><li>max: 512 tokens</li></ul> | <ul><li>min: 28 tokens</li><li>mean: 71.86 tokens</li><li>max: 512 tokens</li></ul> |
321
  * Samples:
322
+ | sentence_0 | sentence_1 | sentence_2 |
323
+ |:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
324
+ | <code>I am planning a trip to Paris from July 10th to July 15th. Can you provide me with the working hours for this period, considering the Federal holidays in France? Also, recommend some events happening in Paris during this time and send me the calendar invites for these events.</code> | <code>def working_days__1_3_add_working_hours:<br> """<br> Description:<br> Add an amount of working time to a given start date/time<br><br> Arguments:<br> ---------<br> - start_date : STRING (required)<br> Description: The start date (YYYY-MM-DD)<br> Default: 2013-12-31<br> - country_code : STRING (required)<br> Description: The ISO country code (2 letters). See <a href=https://api.workingdays.org/api-countries >available countries & configurations</a><br> Default: US<br> - start_time : STRING (required)<br> Description: The start time in a 24 hours format with leading zeros.<br> Default: 08:15<br> """</code> | <code>def betigolo_predictions_sample_predictions:<br> """<br> Description:<br> Get a list of a sample of matches of the previous day, including predictions for many markets.<br><br> Arguments:<br> ---------<br> """</code> |
325
+ | <code>I'm organizing a company event and I need to find a venue that can accommodate 100 people. Can you suggest some event spaces in the city with good reviews? Also, I would like to gather information about nearby transportation options and recommend some local catering services.</code> | <code>def socie_get_members:<br> """<br> Description:<br> Retrieve all or some members of your community.<br><br> Arguments:<br> ---------<br> """</code> | <code>def pinterest_apis_search_user:<br> """<br> Description:<br> Search user by keyword<br><br> Arguments:<br> ---------<br> - keyword : STRING (required)<br> Default: Trang Bui<br> """</code> |
326
+ | <code>I want to surprise my friends with a Netflix binge session and I'm looking for some highly ranked series. Can you provide me with a list of the top 100 ranked Netflix original series? Also, check if the word 'chimpo' is vulgar using the SHIMONETA API.</code> | <code>def shimoneta_send_a_word_to_check:<br> """<br> Description:<br> The API returns what the word means if the word is vulgar.<br><br> Arguments:<br> ---------<br> - word : STRING (required)<br> Default: chimpo<br> """</code> | <code>def NPS-Net_Promoter_Score.Read_a_survey_NLP:<br> """<br> Description:<br> Get a detail of customer survey answer by its survey id (sid), and applies to the third answer (a3) the sentiment analysis feature.<br><br> Arguments:<br> ---------<br> - sid : string (required)<br> """</code> |
327
  * Loss: [<code>MultipleNegativesRankingLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters:
328
  ```json
329
  {
 
466
  | Epoch | Step | dev_cosine_ndcg@10 |
467
  |:------:|:----:|:------------------:|
468
  | -1 | -1 | 0.6750 |
469
+ | 0.1875 | 6 | 0.6878 |
470
+ | 0.375 | 12 | 0.7236 |
471
+ | 0.5625 | 18 | 0.7443 |
472
+ | 0.75 | 24 | 0.7579 |
473
+ | 0.9375 | 30 | 0.7684 |
474
+ | 1.0 | 32 | 0.7687 |
475
+ | 1.125 | 36 | 0.7710 |
476
+ | 1.3125 | 42 | 0.7752 |
477
  | 1.5 | 48 | 0.7745 |
478
+ | 1.6875 | 54 | 0.7795 |
479
+ | 1.875 | 60 | 0.7769 |
480
+ | 2.0 | 64 | 0.7782 |
481
+ | 2.0625 | 66 | 0.7793 |
482
+ | 2.25 | 72 | 0.7808 |
483
+ | 2.4375 | 78 | 0.7791 |
484
+ | 2.625 | 84 | 0.7794 |
485
+ | 2.8125 | 90 | 0.7778 |
486
+ | 3.0 | 96 | 0.7773 |
487
+ | 3.1875 | 102 | 0.7765 |
488
+ | 3.375 | 108 | 0.7767 |
489
+ | 3.5625 | 114 | 0.7760 |
490
+ | 3.75 | 120 | 0.7756 |
491
+ | 3.9375 | 126 | 0.7768 |
492
+ | 4.0 | 128 | 0.7766 |
493
+ | 4.125 | 132 | 0.7766 |
494
+ | 4.3125 | 138 | 0.7759 |
495
+ | 4.5 | 144 | 0.7760 |
496
+ | 4.6875 | 150 | 0.7760 |
497
+ | 4.875 | 156 | 0.7760 |
498
+ | 5.0 | 160 | 0.7760 |
499
 
500
 
501
  ### Framework Versions
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:35d3c34a68e9e1e3c1db380627b9451202179bca6377b70035d8cb0198648886
3
  size 437951328
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c771f72cf77fff1301d3f3937a9b2ef31ce32c37c2bc5dca13b3409f50aa4dbd
3
  size 437951328