--raw_predictions doesn't work with --ect and --cb, but works with --oaa about vowpal_wabbit HOT 5 CLOSED

These aren't defined, because it's very unclear how they should be
defined. What do you intuitively expect?

-John

On 05/07/2014 04:33 PM, sergeyf wrote:

Looks like --raw_predictions works with binary classification and
multi-class with --oaa, but not with --ect (and contextual bandit
raw_predictions is also broken).

Here are the details:

train.dat:

2 | a c
1 | b d
3 | a b c
1 | b c
3 | a d

test.dat:

2 | a c d
1 | c d

Commands that produce a non-empty raw.txt:

vw -d train.dat --oaa 3 -f cb.model
vw -t -d test.dat -i cb.model --raw_predictions raw.txt

Commands that produce an empty raw.txt:

vw -d train.dat --ect 3 -f cb.model
vw -t -d test.dat -i cb.model --raw_predictions raw.txt

—
Reply to this email directly or view it on GitHub
#297.

from vowpal_wabbit.

Ah, good to hear that it's not a bug!

Intuitively, I would expect some kind of discriminant for each arm (for the --cb mode) or class (for the --ect mode). I assume that not all of these are even computed or exist? Is --cb using ect-like machinery under the hood?

from vowpal_wabbit.

It doesn't exist at all for ECT. For --cb, it sometimes exist,
depending on exactly how the cost sensitive classification is implemented.

-John

On 05/07/2014 05:04 PM, sergeyf wrote:

Ah, good to hear that it's not a bug!

Intuitively, I would expect some kind of discriminant for each arm
(for the --cb mode) or class (for the --ect mode). I assume that not
all of these are even computed or exist? Is --cb using ect-like
machinery under the hood?

—
Reply to this email directly or view it on GitHub
#297 (comment).

from vowpal_wabbit.

I see. That makes sense for ECT. It would be good to have for --cb when available. Mostly for the purpose of developing exploration/exploitation algorithms that are discriminant dependent.

from vowpal_wabbit.

Go ahead and make a patch if you want this.

from vowpal_wabbit.