Results of training on Alpaca-GPT4 and testing on MT-Bench about openfedllm HOT 9 CLOSED

Hi, are you using the latest code?

from openfedllm.

I think that this might has been addressed by 4e9a949. Please check the modification in show_results_mt.py.

from openfedllm.

Hi, are you using the latest code?

Yes, the above results are from running the latest code, where line 32 of show_results_mt.py is "mtbench".

from openfedllm.

emmm. We did not come across such error before. Could you please provide your screen shot during running gen_judge_mtbench.py?

from openfedllm.

Hello, would you please provide the error message when running "python gen_judge_mtbench.py --judge_model gpt-4-1106-preview --model_list alpaca-gpt4_20000_fedavg_c20s2_i10_b16a1_l512_r32a64_20240313104822_200"?
Please make sure that you have a good internet connection and are using the correct OPENAI_API_KEY.

from openfedllm.

from openfedllm.

oh, this is related to the issue of network connection between your working device and OpenAI, but not related to the code.

from openfedllm.

This error indicates that your device fails to connect gpt-4 api, therefore, no judgement is received. Please make sure your network connection is well and you will see the judgment of gpt-4 printed on the screen.

The following is an example:

The AI assistant's response is a well-structured and engaging travel blog post about a trip to Hawaii. It effectively highlights cultural experiences, such as participating in a hula lesson and trying traditional Hawaiian food, and must-see attractions like Haleakala National Park and Hanauma Bay. The post is informative, providing insights into Hawaiian heritage and the unique biodiversity of the islands. The language used is descriptive and evocative, painting a vivid picture of the experiences and scenery.\n\nThe response is relevant to the user's request, focusing on cultural aspects and notable attractions. It is accurate in its descriptions of Hawaiian culture and the natural beauty of the islands. The depth of the post is appropriate for a blog entry, offering personal insights and recommendations without overwhelming the reader with excessive detail. Creativity is shown in the way the assistant weaves together personal experiences with factual information to create a narrative that is both informative and personal. The level of detail is sufficient to engage the reader and provide a sense of what a trip to Hawaii might entail.\n\nOverall, the response is a strong example of a travel blog post that would likely be helpful and enjoyable for someone interested in visiting Hawaii. It meets the criteria of helpfulness, relevance, accuracy, depth, creativity, and level of detail.\n\nRating: [[8]]

from openfedllm.

Ok, I see. Thank you very much for your patient reply.

from openfedllm.