Min P generation parameter about text-generation-inference HOT 3 CLOSED

How have you implemented this locally? I'm seeking to do the same , would appreciate some guidance :) @LawrenceGrigoryan

from text-generation-inference.

@avacaondata
so basically you need to add MinPLogitsWarper to /server/text_generation_server/utils/logits_process.py and the min_p parameter to NextTokenChooser class in /server/text_generation_server/utils/tokens.py

Then go over a ton of files and add the min_p parameter wherever needed. Here I would suggest you searching for one of the existing generation params like top_p and adding the missing lines of code for the new parameter min_p

After you done that, just build the image with the default Dockerfile and use it as you usually do

If you have any further questions, feel free to ask :)

from text-generation-inference.

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

from text-generation-inference.