Hi!
Great code and article!
Excellent idea :)!.
Just trying the code in a jupyter notebook with python 3 and i am not sure i am doing it right.
The last error i have is.

hi i having issue running the above script. below is the output.

andrewcz@andrewcz-PORTEGE-Z30t-B ~/adversarial-validation/numerai $ python distinguish_train_test.py
/home/andrewcz/miniconda3/lib/python3.5/site-packages/sklearn/cross_validation.py:44: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.
"This module will be removed in 0.20.", DeprecationWarning)
loading...
cross-validating logistic regression...
Traceback (most recent call last):
File "distinguish_train_test.py", line 44, in
scoring = 'roc_auc', cv = 5, verbose = 1 )
File "/home/andrewcz/miniconda3/lib/python3.5/site-packages/sklearn/cross_validation.py", line 1571, in cross_val_score
for train, test in cv)
File "/home/andrewcz/miniconda3/lib/python3.5/site-packages/sklearn/externals/joblib/parallel.py", line 758, in call
while self.dispatch_one_batch(iterator):
File "/home/andrewcz/miniconda3/lib/python3.5/site-packages/sklearn/externals/joblib/parallel.py", line 608, in dispatch_one_batch
self._dispatch(tasks)
File "/home/andrewcz/miniconda3/lib/python3.5/site-packages/sklearn/externals/joblib/parallel.py", line 571, in _dispatch
job = self._backend.apply_async(batch, callback=cb)
File "/home/andrewcz/miniconda3/lib/python3.5/site-packages/sklearn/externals/joblib/_parallel_backends.py", line 109, in apply_async
result = ImmediateResult(func)
File "/home/andrewcz/miniconda3/lib/python3.5/site-packages/sklearn/externals/joblib/_parallel_backends.py", line 326, in init
self.results = batch()
File "/home/andrewcz/miniconda3/lib/python3.5/site-packages/sklearn/externals/joblib/parallel.py", line 131, in call
return [func(*args, **kwargs) for func, args, kwargs in self.items]
File "/home/andrewcz/miniconda3/lib/python3.5/site-packages/sklearn/externals/joblib/parallel.py", line 131, in
return [func(*args, **kwargs) for func, args, kwargs in self.items]
File "/home/andrewcz/miniconda3/lib/python3.5/site-packages/sklearn/cross_validation.py", line 1665, in _fit_and_score
estimator.fit(X_train, y_train, **fit_params)
File "/home/andrewcz/miniconda3/lib/python3.5/site-packages/sklearn/linear_model/logistic.py", line 1173, in fit
order="C")
File "/home/andrewcz/miniconda3/lib/python3.5/site-packages/sklearn/utils/validation.py", line 521, in check_X_y
ensure_min_features, warn_on_dtype, estimator)
File "/home/andrewcz/miniconda3/lib/python3.5/site-packages/sklearn/utils/validation.py", line 407, in check_array
_assert_all_finite(array)
File "/home/andrewcz/miniconda3/lib/python3.5/site-packages/sklearn/utils/validation.py", line 58, in _assert_all_finite
" or a value too large for %r." % X.dtype)
ValueError: Input contains NaN, infinity or a value too large for dtype('float64').
andrewcz@andrewcz-PORTEGE-Z30t-B ~/adversarial-validation/numerai $

I was wondering if you have some time to assist many thanks.
Andrew

The numerai data I get from the web seems to be little wrong, if there is a link to numerai dataset will be better