The fg_cme from tal-ai

fg_cme's People

Contributors

Stargazers

Watchers

fg_cme's Issues

Question regarding alignment generation

Hi there!

Great paper and great code! I just have a small question regarding the binary alignment matrix part of the code and paper.

In this part in the main function:

align_info = open(align_path, 'r').readlines()[1:-1] # get rid off the head and tail info
        align_info = [re.split('\ +', x) for x in align_info]
        align_info = [(x[-1].strip('\n').split('(')[0].lower(),int(x[1]),int(x[2])) for x in align_info]
        # For the silence probably we can make some use
        align_info = [x for x in align_info if x[0] not in ['<s>','<sil>','</s>']]
        align_input = []
        for _, begin_time, end_time in align_info:
            begin_idx = int(begin_time * 0.01 / 0.01)
            end_idx = int(end_time * 0.01 / 0.01) + 1
            align_slice = torch.zeros(audio_input.size(0))
            align_slice[begin_idx:end_idx] = 1.0
            align_input.append(align_slice[None,:])
        align_input = torch.cat(align_input, dim=0)

If align info is a generic alignment file with timestamps based on seconds, then I am not sure how begin_idx = int(begin_time * 0.01 / 0.01) would work. According to my intuition this should probably be:

begin_idx = int(begin_time * sampling_rate / frame_step)

Same question for end_idx.

Please let me know if there is something wrong in my understanding!

Thank You

Hello, I want to reproduce the code

pkl file issueissue

Hello, I'm reproducing the code and want to verify it on other datasets. How to get the pkl file specifically? Thanks.

How data prepared?

Hello, I want to reproduce this experiment using IEOMCAP. But I am confused how do I organize the dataset properly. How to get glove300d_w2i.pkl, glove300d_full.npy and Session[1-5]-wav.pkl?

Recommend Projects