[FEAT]: Add data point level filtering based on quality of all selected dialogues in train and test sets (whole dataset) about sotopia-pi HOT 5 CLOSED

For the quality-based classification, it is data point character level filtering.

from sotopia-pi.

Note that the quality should be based on environment + agent (in the same environment, one agent may produce quality data while the other may not).

from sotopia-pi.

Currently, we add quality filtering by using the goal score in rewards for each episode logs. The idea is that, for a given dialogue, we use the average goal scores to determine if this dialogue is considered "good" in achieving the goal. For all dialogues within the same scenario, we use the average goal score to determine its relative stand. If the average goal score for a dialogue is below the scenario average, we should remove it, UNLESS the scenario average is too high (such as 10).

To avoid removing dialogues with high goal score but stand relatively low in the scenario, we use 7 as a lower bound threshold. If a dialogue's avg score is above 7, regardless of its stand in the scenario, we are going to include the dialogue.

Note that 7 comes from observing the distribution of easy scenario, and for now, we apply the same threshold for our testing data which contains hard scenario. It might worth lower the bar for hard scenario [DISCUSSION].

By using 7 threshold, we reduce 450 dialogues to 388 dialogues.

from sotopia-pi.

For the code we apply the filtering, it currently sits in https://github.com/XuhuiZhou/sotopia/tree/feature/log-filtering-for-ft. We are considering copy the main function/building blocks from sotopia and rather have these files on our sotopia-llm-ft repo, but this copy/paste has not been executed yet.

from sotopia-pi.

Finish in #58

from sotopia-pi.