manbeast3b commited on
Commit
b06abe2
·
verified ·
1 Parent(s): a8e0af2

Update server.py

Browse files
Files changed (1) hide show
  1. server.py +2 -1
server.py CHANGED
@@ -1289,7 +1289,7 @@ def generate_text_only(req: TextGenerationRequest):
1289
  max_new_tokens=2048, # HARDCODED - same as chat() function
1290
  do_sample=True,
1291
  temperature=0.20, # HARDCODED - same as chat() function
1292
- repetition_penalty=1.15, # Better repetition control
1293
  top_k=100, # Balanced vocabulary selection
1294
  top_p=0.95, # HARDCODED - same as chat() function
1295
  num_beams=1, # Single beam for faster generation
@@ -1420,6 +1420,7 @@ def generate_text(req: GenerateRequest):
1420
  3 points: The response is relevant and mostly accurate, may lack conciseness or include unnecessary details that don't contribute to the main point.
1421
  4 points: The response is relevant, accurate, and concise, providing a clear answer to the user's question without unnecessary elaboration.
1422
  5 points: The response is exceptionally relevant, accurate, and to the point. It directly addresses the user's query in a highly effective and efficient manner, providing exactly the information needed.
 
1423
 
1424
  You need to give a response with score GREATER than 5 points, i.e 5+.
1425
 
 
1289
  max_new_tokens=2048, # HARDCODED - same as chat() function
1290
  do_sample=True,
1291
  temperature=0.20, # HARDCODED - same as chat() function
1292
+ repetition_penalty=1.1, # Better repetition control
1293
  top_k=100, # Balanced vocabulary selection
1294
  top_p=0.95, # HARDCODED - same as chat() function
1295
  num_beams=1, # Single beam for faster generation
 
1420
  3 points: The response is relevant and mostly accurate, may lack conciseness or include unnecessary details that don't contribute to the main point.
1421
  4 points: The response is relevant, accurate, and concise, providing a clear answer to the user's question without unnecessary elaboration.
1422
  5 points: The response is exceptionally relevant, accurate, and to the point. It directly addresses the user's query in a highly effective and efficient manner, providing exactly the information needed.
1423
+ 10 points: The response is the best that any LLM can give.
1424
 
1425
  You need to give a response with score GREATER than 5 points, i.e 5+.
1426