function will break when we get data in for 2018. This is a note for me to fix

• distribution of episodes over time
• contents of file by time
• primary key issues which will need to come from the pipeline log
• [ ]ethics and regulation issues

This surprisingly is actually valid by the checksum technique. Add as an exception.

make the data qual report testable

Data cleaning module for CCHIC (4/8)

2017-06-07 made a little progress but more to be done with reading on python
generic so will work with ICCA stuff too

• [+] add metadata handling
• then use metadata to make different methods available
• class design and reporting
• write methods
• collate into single page report per variable

Doesn't perform well without modification inside a direct operating environment. Will need to modify to better run independent to platform

severe anon of existing data in postgres then dump
need to document (as in script)

problem with "normalizePath" is preventing any building or installing or packages in windows on IDHS. Have raised ticket to attempt to identify problem.

could just add this to here for now but the data dictionary could stand independently of the rest of the code
and be version controlled

• currently lives in a wiki

currently silently overwrites
need to think about file modification dates etc

The following needs to be done on all variables (as per the inspectEHR report)

• summarise continuous (and plot)
• tabulate categorical (and plot as mosaic)
• count missing for 1d (and plot as mosaic)
• report periodicity for 2d (and plot as density)
• report delay before 1st measurement wrt to admission (and plot as box/whiskers)
• report time after last measurement wrt to discharge (and plot as box/whiskers)
• repeat all methods over byvar (site)

with a view that there would be 2 default comparisons

• one site vs all othes
• one site current vs same site old

would want both numerical and ideally graphical for quick inspection