opennewslabs / datasmells Goto Github PK

Description of Error: Data can have legit outliers, but sometimes it's more nefarious. Check yourself before you wreck yourself.

Things to look for: Sort or order all applicable fields. Double-check the veracity of any super-high or super-low numbers. Call to confirm with your data source if needed. Related: are there any negative values? This can indicate an import or calculation error. Calculate average and standard deviation of fields and inspect any values that aren't within two or three standard deviations.

Examples:

Description of Error :
Latitude and longitude are reversed, resulting in the points being in completely the wrong place.
Things to look for :
pick something not near the equator to test. Ideally, you should have an idea of what its lat/long should be
Examples:

Description of Error :
Things to look for :
Examples :

Description of Error :
Things to look for :
Examples :

A general page to cover issues with implicit type coercion

Description of Error:

Things to look for:

Examples:

Description of Error :
Things to look for :
Examples :

This is the classic case where there might not be something wrong within the data but the number of rows suggests very strongly that the data was truncated by Excel and is thus incomplete.

Description of Error :
Things to look for :
Examples :

Description of Error: Is your data shooting blanks? This can be an import issue.

Things to look for: Search for null or blank values in numeric fields. Pay close attention to fields that are entirely null or blank; this can indicate an import error.

Examples:

Description of Error :

Things to look for :

Examples :