chateval's Projects
A platform for the warehousing and evaluation of neural open domain chatbot models.
Public evaluation tool for non task driven neural open domain chatbots
Label, clean and enrich text datasets with LLMs.
BARTScore: Evaluating Generated Text as Text Generation
Code to publish HITs on Mechanical Turk to collect human baselines
A benchmark dataset for evaluating dialog system and natural language generation metrics.
BotSIM - a data-efficient end-to-end Bot SIMulation toolkit for evaluation, diagnosis, and improvement of commercial chatbots
Scripts for ChatEval and Dialog Annotation
Chatbot comparison webapp built using React.
ConTurE is a human-chatbot dataset that contains turn level annotations to assess the quality of chatbot responses.
D-score Framework For Open-domain Automatic Dialogue Evaluation
Code for ACL 2021 main conference paper "Conversations Are Not Flat: Modeling the Dynamic Information Flow across Dialogue Utterances".
Evaluate your dialog model with 17 metrics! (see paper)
NLG and NLU for dialogue processing
Efficient Annotation of Scalar Labels
Microservice to handle automatic evaluation of neural chatbot models. Multiple automated evaluation methods (including embedding-based metrics).
Source Code of Paper "GPTScore: Evaluate as You Desire"
kani (カニ) is a highly hackable microframework for chat-based language models with tool usage/function calling.
A suite of tools for managing crowdsourcing tasks from the inception through to data packaging for research use
Code and Data for the paper Investigating Evaluation of Open-Domain Dialogue Systems With Human Generated Multiple References SIGdial 2019
Code for the paper "Learning an Unreferenced Metric for Online Dialogue Evaluation", ACL 2020
The dataset and code released with the submission of NAACL 2018 paper "RankME: Reliable Human Ratings for Natural Language Generation"
All experiments and evaluation code for decoding diversity project!