lightning-universe / lightning-covid19 Goto Github PK

Rewrite and possibly improve COVIDx dataset like data preparation.
Described in https://arxiv.org/pdf/2003.09871.pdf
and in https://github.com/lindawangg/COVID-Net.
Note: We should be careful with the AGPL-3.0 license and copy-pasting the code.

Consider implementing it in torchxrayvision

See https://github.com/toandaominh1997/EfficientDet.Pytorch/blob/master/train.py

Priorities:
1 fast-dev-run https://pytorch-lightning.readthedocs.io/en/latest/debugging.html#fast-dev-run
2. single node training command on one node
..
..
N -1 distributed training

Setup CI skeleton for running end-to-end functional test e.g. for fast_dev_run see #3

Take inspiration from https://github.com/PyTorchLightning/pytorch-lightning/blob/master/.circleci/config.yml

Plot Area Under The Curve

Depends on #15

This proposal suggests a principal solution to build a model that will predict Bounding Box (BB) around lungs, artifacts and other well-defined objects on the images.

The COVID19 and other x-ray chest datasets contain chest X-ray images capturing not only Lungs but sometimes also arms, neck, etc.
The images also contain text labels and other artifacts.

The advantages:

• If the model is trained to differentiate between artifacts and the chest ray region - it can help as a form of regularization for COVID19 detection precision
• Bounding box chest array prediction will allow us explicitly compute recall - it helps with model understanding and evaluation
  • If the image does not contain lungs or the model does not focus on lungs (e.g. it focuses on an arm, or on an artifact - textual label in the image), we know that the model classification is poor even if it predicts the COVID19 label correctly
• It would be nice to detect key regions for detecting COVID19 for doctors to review (See also disadvantages: lack of BBs)

The disadvantages:

• BB predictions is a much harder task than classification
  • Having so few data, it can prove too hard to train the model, letting the classification to fail completely
    • With more data it should not be a problem -> what is the amount of the data to start using BB?
• The annotations for Bounding Boxes are generally missing
  • Some Bounding Boxes types can be automatically created - we can easily generate artifacts and add them to chest ray images
  • We can create heuristics for selecting lungs regions

Example images with textual artifacts, "pipes in lungs artifact", the edge of image artifact, ..

• describe project in README
• contributing guideline

Link below datasets which can be used for data pretraining and data augmentation

Comment about quality and how they can complement the main dataset we use
https://github.com/ieee8023/covid-chestxray-dataset

Possible data leakage?
On the original dataset, there are several images from the same patient see for example patient number 2
Should we take this into account when splitting the data?

Originally posted by @shpotes in #10

Replace the Cocodataset in the EfficientDet.Pytorch by Covid dataset

See the training script for models which we want to use
https://github.com/toandaominh1997/EfficientDet.Pytorch/blob/97f2851b6063da59a7ae73dadfc4b5e5022473e3/train.py#L182

See the Covid dataset dataloader https://github.com/mlmed/torchxrayvision/blob/master/torchxrayvision/datasets.py#L814

Dataloader
https://github.com/mlmed/torchxrayvision/blob/master/torchxrayvision/datasets.py#L814
more infor about the dataset
https://github.com/ieee8023/covid-chestxray-dataset

Add reasonable image augmentation

• horizontal/vertical flip
• rotation
• zoom
• etc

1. The _parse_batch() function throws a Key Error while trying to fetch x["PA"] . This is not consistent with torchxrayvision
2. The Kornia based data augmentation adds an additional dimension while doing unsqueeze.

Hi @Borda,

can we allow 119 characters on a line instead of 79?

Please, please :)

We should compute stats for

• precision
• recall
• AUC and Average mean precision AMP.

and replace ACC by compute Prec, recall, F1, AMP after each epoch.

Plotting AUC is described in issue #16