torchserve output customization about serve HOT 3 CLOSED

Hi @advaitraut do I understand correctly that you want to convert the output of your model (a tensor) into a string which you want to send back to the client? This would go into the postprocess step. You can just create any string you would like and return this in the postprocess method. Just make sure that you return a list of strings where each entry corresponds to a batch entry.

Something like this should work:

#assuming you get a (8, 1) tensor and batch size is 8
def postprocess(model_output: torch.Tensor):
    return [f"Element {idx}: {model_output[idx].item()}" for idx in range(model_output.size(0))]

Let me know if you get stuck.

from serve.

Hi @mreso , Thank you for reponse.
Your code helped me to realize that postprocess() needs a list to be returned.
It solved my problem.

from serve.

Great to hear @advaitraut will close this issue as you solved your problem but feel free to reopen if you have further question.

from serve.