Mitigate dependent sources with "dependence mode" about confidis HOT 3 OPEN

Another way to do this:

Consider two graphs, DependentConfidis (DC) and IndependentConfidis (IC)

DC can construct an IC that uses beliefs as sources. DC can determine beliefs by examining the correlation of sources.

Another note:

The number of beliefs is (ideally) a function of the number of questions but there can't be more beliefs than sources, because each source is a superset of beliefs. Each belief maps to a set of "correct" questions. Each source has a probability of predicting a belief. The number of beliefs can be adapted to achieve the highest accuracy.

from confidis.

Another method:

At the time of question confidence computation, compute the correlation between sources via agreement/(total both answered) (not sure how to handle low total both answered). If the correlation is above threshold, don't use.

Extension: Use the correlation to weight the contribution to the confidence. This gets a bit hairy with Bayes rule, but is possible to do in a theoretically sound way. That said, correlations will be based off limited data.

from confidis.

Ok so in a non-independent mode one can compute the confidence of an answer using:

Where A and B are two sources. It could be expanded to more sources, but it's not clear how you could handle low-data situations.

This is better than the fully independent solution (below) because it handles correlations using historical data.

1 - (1 - P(correct|A))(1 - P(correct|B))

from confidis.