melbournebioinformatics / analytics Goto Github PK

Review project titles and descriptions to identify main project categories
[guess at example categories, Molecular dynamics, Fluid dynamics, data pipeline, class/training, system administration, ...]

Currently, rows slurm data are "dependent", the might be multiple job steps for a single job. Need to collapse dependent rows into a single independent job description.
e.g.
1234.bash
1234.0
1234.1
...

convert to
1234

Generate a distribution graphs for each data field. Comment on each one, report interesting findings.

Get all job (sacct), project and disk data

There are columns (data fields) that are "uninformative" in the sense that they might be filled with "nan" or has the same value throughout (0 information content). Should write code that does such checking. If a column is uninformative for all dataset, we should workout why and remove it from downstream analysis.

Fields such as wait time, total memory request/used, job frequency, etc could be derived from the data.
Want candidate list for discussion

Need README, licence etc