nicolasag / convai Goto Github PK

Joelle suggested us to see if we can modify the beam sampling in HRED to account for the user feedback, possibly trickling down the user rating for the conversation to sample the next best sentence.

[Peter] I think we should probably have anaphora resolution as a preprocessing step? Idk how hard this would be but I wanted to see what people thought about this?

[Koustuv] Coreference resolution could be useful for the ranker model preprocessing. However, i am not sure how the implementation would be: resolve pronouns in candidate response to the article?

In order to maximize immediate response, we could create an evaluation dataset, where given a context dialogs and article, we provide all the candidate responses, and ask user to rate / rank them.

get_response call

detect when user asks something related to the article.
example of training set: squad with (article, question, flag) triples where question is either related (flag=1) or not (flag=0) to the article

We already have Dual Encoder retrieval model, but the responses given by Rosemary in Ethics paper seems pretty good. can we quickly add the existing vhred model?

turns out, according to Rosemary we need the entire repo bcs "Vhred model don't allow interactive interface". But I'm not sure thats a valid reason...

When chatting with the bot and asking "What is your name?" the answer returned by ALICEBOT is "My name is Alexa Prize Social Bot". We need to avoid that so that users never select this candidate response in data collection phases!