Giter VIP home page Giter VIP logo

modular-encoder-decoders-segmentation's Introduction

Pytorch Lightning for Segmentation Models

Run

An example of running a training job can be found in train_segmentation.sh. Make sure to include your w&b API key and adjust any filepaths and hyperparameters here.

Docker

Modify the docker/run.sh file to mount your data to the container. Build the runtime container using the helper script ./docker/build.sh. Run the container with docker/run.sh.

Custom Datasets

To add a custom dataset you need to do a few things. First define the dataset class and add it to the build_dataset() function in datasets/segmentation.py by adding it as a return in the switch statement. Next add the revelant information to the db_info dictionary in datasets/segmentation.py. An example of a valid entry would be:

"cityscapes": {
    "n_classes": 19,
    "size": [768, 768], # [height, width]
    "ignore_index": -100,
    "class_labels": CITYSCAPES_SEG_CLASSES,
    "normalisation": [[0.288, 0.327, 0.286], [0.190, 0.190, 0.187]],
}

You can leave the "class_labels" field for the dataset equal to None, this will log each classes IoU as a number from [0, n_classes-1]. If you want to have IoU logged with the pixel classifications natural language name you can define a dictionary in containing the details of each class. An example of PascalVOC can be found in /datasets/constants.py. Once the dataset if defined, added to build_dataset() and has an entry in db_info you can use it in train_segmentation.sh as the --dataset-name.

modular-encoder-decoders-segmentation's People

Contributors

adrianorenstein avatar xdynames avatar

Stargazers

 avatar

Watchers

 avatar

Forkers

scottwedge

modular-encoder-decoders-segmentation's Issues

Prettify readme

The readme needs some images, gifs? and keywords to get noticed

example keywords:
pytorch
pytorch-lightning
multi-class segmentation / semantic segmentation
multi-label segmentation

Review dataset.py

Target encoding likely needs to be re-examined for all the dataset classes in dataset.py. Primarily because lightning's IoU doesn't let us provide a custom ignore label which will need to be handled in both IoU calculation and cross-entropy loss.

Additionally cityscapes is loading targets with a much larger set of class ID values than expected. All classes should be checked though to ensure correct loading.

Lightning Metric return NaN

Current dropped in the pytorch-lightning metric.classification.IoU module which returns NaN for both training and validation after ~10 epochs.

Haven't looked into how ignore label / ignore background works yet but this would be my first point of interrogation
Other than that it might have something to do with how they handle the numerical softening to prevent division by zero

Multi-GPU Training

Since upgrading the version of lightning specifying the GPUs to use gives the following error:

You requested GPUs: [2]
But your machine only has: [0, 1]

Need to investigate if their API has changed with respect to this

Examples - Colab/JupyerNotebook

I think these examples should be made in a notebook (preferably a colab?)

  • making your own resnet block, integrating that with our pipeline
  • freezing backbone weights, training everything else

Add anymore you think are needed

Augmentations

A stand in class with a simple resize, normalize and to tensor transform was added to support initial training tests in augment.py

Should add support for a more broad set of image augmentations, while respecting paired-geometry of images and their semantic masks and avoiding channel based transforms being applied to targets

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.