I'm trying to use the splited data in a study about disagreement and I found out that

Problems with some labels about convabuse HOT 1 OPEN

rbnuria commented on August 10, 2024

Problems with some labels

from convabuse.

Comments (1)

wojtek11530 commented on August 10, 2024

Hi, I have also encountered it. I have tried to omit the problem by dropping the rows which are not correct:

df = pd.read_csv(dataset_path / '2_splits'/ f'ConvAbuseEMNLP{split}.csv')
    
    df = df.astype({'example_id': 'int64'})  
    annot_columns = [f'Annotator{i+1}_is_abuse.{value}' for value in abuse_values for i in range(8)]
    index_to_drop = []

    for i, row in df.iterrows():
        for c in annot_columns:
            if row[c] not in ['0', '1', np.nan]:
                index_to_drop.append(i)
                break

    df = df.drop(index_to_drop).reset_index(drop=False)
    df = df.astype({c: 'float64' for c in annot_columns})

Unfortunately, it drops a large qunity of data (~10k) so it is half of all annotations. Therefore the test split has no examples labeled as -3, -2, 0 is_abusive. I suggest fixing the entire dataset & reuploading.

from convabuse.

Recommend Projects