Clarification on cross val splits? about ase_ani HOT 3 CLOSED

Splitting on conformers. In the ANI-1 paper we put all 22M conformers in a bucket. Then, from the bucket we pick 80% for training, 10% for valid, and 10% for test. This means there is a probability that some of the 57k configurations (from which the conformers were generated) have no conformers from the training set.

from ase_ani.

Just so I understand - you're allowing a single chemotype (Eg. Methane, CH4), whose 1000 conformers, to be split such that 800 CH4 conformers is in train, 100 CH4 conformers is in valid, 100 CH4 conformers is in test? This seems a little strange since I imagine a better test of generalizibility is to split on the chemotype (i.e. given a collection of methane, ethane, propane, butane, etc.), it's split such that all of methane/ethane conformers are in train, all of propane is in valid, and all of butane is in test.

from ase_ani.

Just to be clear again, we're doing validation such that we guarantee training/valid/test disjointly partition the 57k configurations with absolutely no overlap.

from ase_ani.