Brainstorming validators to add about guardrails HOT 5 CLOSED

We're developing a library that makes it possible to validate generated text according to different criteria:

• summary quality
• translation quality
• factual consistency
• fluency
• toxicity

I sent one PR here, and if that looks useful I could add other ones too!
#48

from guardrails.

@neubig that's a great suggestion, having a deeper integration between Guardrails and Critique would be awesome! Happy to offer any support as you're adding demos. I'll also start hacking on adding a few pieces in.

from guardrails.

Great! That'd be awesome. It might be nice if it was possible to have a somewhat more generalized way to call the various Critique metrics (without re-implementing the boilerplate in every notebook). I can think about that more, but any pointers are welcome too.

from guardrails.

Just added a PR #61 that shows an example of a validator that uses embeddings to do a semantic check. The Critique validators sound super cool @neubig, maybe one way is to subclass Validator (CritiqueValidator(Validator)), do the boilerplate setup in the superclass and then implement all the downstream ones?

from guardrails.

we've added a number of validators that are more probabilistic in nature like provenance, etc. resolving this now.

from guardrails.