Source: NIH : A chest x-ray identifies a lung mass.

Please refer to this article to install horovod and tensorflow.

https://github.com/dellemc-hpc-ai/ai-radiologist-GPU.git

cd ai-radiologist-GPU

Run the following commands from the ai-radiologist-GPU directory.

./download_dataset.sh

you must be able to see tars folder.

./extract_images.sh

You must be able to see all the extracted images inside images_all/images folder.

At this point, you can start to train the cheXNet model using raw images, refer to Run the Job if you'd like to continue training with raw images. However, if you want to train the model with TF Records then convert the raw images to TF Records.

This will convert your raw images to TF Records which improves the training speed by ~8-15% percent.

./write_tfrecords.sh

Please Ensure that:

• You edit the conda env name to your env name in your submission script.
• Change the paths from the submission script to your MPI, CUDA, etc build locations.

If you're using slurm as scheduler, submit the corresponding script based on the data you want to run. You can change numbers for N and n inside the scripts.

sbatch job_submissions/slurm/{raw_1gpu.sh/tfrec_1gpu.sh}

• Enabling AI Workloads in HPC Environments
• Training an AI Radiologist using Distributed Deep Learning with Nvidia GPUs
• Scaling Performance and Training CheXNet using Bare Metal vs Kubernetes
• Optimization Techniques for Training CheXNet on Dell C4140 with Nvidia V100 GPUs
• Tips and Tricks to Optimize workflow with TF and Horovod on GPUs

• NIH Dataset
• CheXNet Paper
• Horovod