hltchkust / moel Goto Github PK

MoEL: Mixture of Empathetic Listeners

License: MIT License

Python 97.11% Perl 2.89%

transformer chatbot empathy dialogue-systems mixture-of-experts dialogue-generation transformer-pytorch

moel's Introduction

Installation

git clone https://github.com/HLTCHKUST/hltchkust.github.io.git
git fetch
git checkout source

Make sure you have Ruby and Bundler installed on your system (hint: for ease of managing ruby gems, consider using rbenv), then run

bundle install

To add new blog

Create a new markdown file under _posts, follow the format of previous ones.

To add an announcement to News

Create a new markdown file under _news, follow the format of previous ones.

To add a new publication

Just add your bibliography to the _bibliography/papers.bib. If you want to show it in the Selected publications, add the selected={true} field.

Deployment

To deploy new changes, just

./bin/deploy --user

then

git push origin source

Wait a few seconds, and refresh!

For local development

To build the latest content that you change

bundle exec jekyll build

To run the server locally

bundle exec jekyll serve

Then, a localhost is established at port 4000.

moel's People

Contributors

Stargazers

Watchers

Forkers

ttgit arushij1711 pengshanshan99 xl2248 aysha1302 wanghaoran-ucas mattc95 luomuqinghan srishtigupta253 d4sh12 alejandro-hang leoxiang66 xiaoheng-zhang99 kaushal-create

moel's Issues

dataset process

Thanks for your contribution. I am interesting your work.

May I ask for the code of processing the Empatheticdialogues?

Thanks a lot.

Bug in beam search of Expert

Hi all,

Thank you for open sourcing this repository.
I'm having trouble when decoding an expert model.

In particular, all I did was clone the repo and run the following command:

python3 main.py --model experts  --label_smoothing --noam --emb_dim 300 --hidden_dim 300 --hop 1 --heads 2 --topk 5 --cuda --pretrain_emb --softmax --basic_learner --schedule 10000 --save_path save/moel/

It seems like the model finishes training, and in line 97 of main.py, when trying to run evaluation, I get the following traceback:

Traceback (most recent call last):
  File "main.py", line 97, in <module>
    loss_test, ppl_test, bce_test, acc_test, bleu_score_g, bleu_score_b= evaluate(model, data_loader_tst ,ty="test", max_dec_step=50)
  File "/home/repos/empathetic_systems/MoEL/model/common_layer.py", line 824, in evaluate
    sent_b = t.beam_search(batch, max_dec_step=max_dec_step)
  File "/home/repos/empathetic_systems/MoEL/utils/beam_omt_experts.py", line 273, in beam_search
    active_inst_idx_list = beam_decode_step(inst_dec_beams, len_dec_seq, src_seq, src_enc, inst_idx_to_position_map, n_bm, enc_batch_extend_vocab, extra_zeros, mask_src, encoder_db, mask_transformer_db, DB_ext_vocab_batch)
  File "/home/repos/empathetic_systems/MoEL/utils/beam_omt_experts.py", line 205, in beam_decode_step
    dec_seq = prepare_beam_dec_seq(inst_dec_beams, len_dec_seq)
  File "/home/repos/empathetic_systems/MoEL/utils/beam_omt_experts.py", line 157, in prepare_beam_dec_seq
    dec_partial_seq = [b.get_current_state() for b in inst_dec_beams if not b.done]
  File "/home/repos/empathetic_systems/MoEL/utils/beam_omt_experts.py", line 157, in <listcomp>
    dec_partial_seq = [b.get_current_state() for b in inst_dec_beams if not b.done]
  File "/home/repos/empathetic_systems/MoEL/utils/beam_omt_experts.py", line 33, in get_current_state
    return self.get_tentative_hypothesis()
  File "/home/repos/empathetic_systems/MoEL/utils/beam_omt_experts.py", line 90, in get_tentative_hypothesis
    hyps = [self.get_hypothesis(k) for k in keys]
  File "/home/repos/empathetic_systems/MoEL/utils/beam_omt_experts.py", line 90, in <listcomp>
    hyps = [self.get_hypothesis(k) for k in keys]
  File "/home/repos/empathetic_systems/MoEL/utils/beam_omt_experts.py", line 100, in get_hypothesis
    hyp.append(self.next_ys[j+1][k])
IndexError: tensors used as indices must be long, byte or bool tensors

In particular, I'm seeing that the k value, as well as self.prev_ks, gets changed somewhere along the way and no longer hold indices but instead stores float values.

(Pdb) pp k                                                                              
tensor(0.0014, device='cuda:0')    

(Pdb) self.prev_ks                                                                                                                                                               
[tensor([0.0026, 0.0054, 0.0003, 0.0069, 0.0075], device='cuda:0'), tensor([1.4338e-03, 1.0014e+00, 2.0014e+00, 7.5908e-04, 1.0008e+00],                                         
       device='cuda:0')]

Do you have any pointers on how to resolve this?

Thank you in advance.

empathetic_dataset_preprocesing code

Can you please provide empathetic_dialogues dataset preprocessing code.It will be very helpful

about metric

Hi,

This is a very nice work! While there are some detials that I'm confused:

Does the Bleu means average Bleu score like the original paper "Towards Empathetic Open-domain Conversation Models: a New Benchmark and Dataset"?
Why don't you compare the basline result in "Towards Empathetic Open-domain Conversation Models: a New Benchmark and Dataset"?

Inference

Dear Authors,

Thank you for providing code.

I want to use the trained model to generate responses for a new text. Is there is a way to do the inference part for a new text.

Do you still have the original data set?

Do you still have the original data set? Can you send me a copy? Thank you！

how to run the get_ed_data.py ?

How to run the get_ed_data.py ?

Training time rquired to train the MoEL model

Sir,
I am running the program in Colab.It get stucks while training .Could you pls tell how much time needed to train the MoEL model?
Kindly help

ret_sentences.append(' '.join([self.model.vocab.index2word[idx] for idx in d[0]]).replace('EOS','')) KeyError: 9.5367431640625e-07

When I run the code for my dataset, I met this error. Do you know what happened?

Questions about code reproduction

When I ran the code locally, I successfully ran the program according to the README.md, but due to the patience of the program, my program ended early and entered the testing stage. I ran TRS and MultiTRS. The two programs were about 16000. When entering the testing stage when stepping, is this normal?
When I ran the MoEL, I finally got the result:
EVAL Loss PPL Accuracy Bleu_g Bleu_b
test 3 .7540 42.6931 0.37 2.85 2.87
How can I set up to reproduce the results of the paper?

Thanks