Questions on evaluation data in the project and paper about gorilla HOT 2 CLOSED

I'm also very interested about it. Wait for reply!!!

from gorilla.

Thanks for the question @Jeffwan and your interest @FanWan

1. Yup - FT is just finetuned LLaMA. RT is Retriever-aware finetuned.
2. Great point. We also ran into this, and we decided to give the benefit of doubt for the LLMs making the evaluation more fair for closed-source LLMs that were typically harder to coax. In all these cases, we only compare the api_call field. You will notice we introduced a delimiter <<<api_call>>> to get the API call independent of the format.

Hope this answers the questions. I'll keep this issue open briefly to see if you have any follow-ups 🙂

from gorilla.