oulu-imeds / solt Goto Github PK

https://arxiv.org/pdf/2001.04086.pdf

Need support for 32 and and 16 bit images.

Often happens when p(transform)=0.5. The pipeline crashes.

Need to enable different policies for augmentations

https://github.com/MIPT-Oulu/solt/blob/485bfb0d471134f75e09d06aa5e9ce4b57c0e13c/solt/transforms/_transforms.py#L1018 tells something about noise. Should talk about the intensities instead.

Stack optimization is useful only when the transform time is very long. Therefore, it will be off by default in the newest release.

Current serialization needs improvement. I does not work correctly all the time. We also need to make sure that we can load the transforms back after serialization

Warning, treated as error:
/home/travis/miniconda/envs/solt_test_env/lib/python3.7/site-packages/solt/transforms/_transforms.py:docstring of solt.transforms.IntensityRemap:15:Footnote [1] is not referenced.

@soupault Please fix.

For images and numeric types

https://github.com/MIPT-Oulu/solt/blob/0f462eae0b49489caa69cb652d484fe6cc4b7737/solt/transforms/_transforms.py#L1129

README.md has the old documentation link instead of the new one: https://oulu-imeds.github.io/solt/.

Support of different interpolation types is needed. Now only bilinear and bicubic are available

The use case for this is that for some images it might be better to rely on their mean intensity and increase their intensity by n% of the mean value.

Code to reproduce:

import solt
import solt.transforms as slt
import numpy as np


if __name__ == "__main__":
    trf = solt.Stream([slt.Resize(resize_to=(50, 50)),
                       slt.Flip()])
    img1 = np.ones((100, 100, 3), dtype=np.int32)
    img2 = np.ones((110, 110, 3), dtype=np.int32)
    trf_img = trf({'images': (img1, img2)})
    print('Done')

Adding a resize transformation at the beginning doesn't help. A clear error message is needed for this case.

Repository has been moved to new location, and we thus need to adjust the deployment of docs on the website

https://arxiv.org/pdf/2003.06158.pdf

Let's cover the mainstream cases when we need to unwrap the DataContainer, such as (img, label) and (img, mask).

Rotations by 90 degrees should not distort the image.

Works fine with 1.1, but not 1.5.
Traceback:

    as_dict=as_dict, scale_keypoints=scale_keypoints, normalize=normalize, mean=mean, std=std,
  File ".../site-packages/solt/core/_data.py", line 273, in to_torch
    img.sub_(mean)
RuntimeError: output with backend CPU and dtype Byte doesn't match the desired backend CPU and dtype Float
Process finished with exit code 1

Support of elastic transforms for keypoints, masks and images is needed

Make random crop support the contours so that each crop's will belong to that contour

Existing API allows to do this already. Bounding box is simply a label + 4 keypoints.

Bounding Boxes need to support different formats, such as COCO.

the augbench package needs further development and its results need to be reported in the README.

the benchmark needs to cover random transformations rather than the static ones. It is important to make comparison for three cases: image, image mask, and image+10 masks (instance segmentation task).

Shift-Scale-Rotate type of transform is needed as many libraries have it

Example with satellite images

Would be useful to increase the probability of augmentation for rare classes. This should also work in a multi-label setting

Currently, selective stream takes a parameter that tells how many transforms to select from the list of transformations. I was recently suggested to make it a range and allow to sample random number of augmentations out of this range.

For segmentation it is often good to use nearest neighbors interpolation for masks, while for images bilinear or bicubic still needed to be kept.

Module 3D transforms would be nice to have. First suggested transforms are crops, flips and rotations 90 degrees

https://github.com/MIPT-Oulu/solt/blob/0f462eae0b49489caa69cb652d484fe6cc4b7737/solt/transforms/_transforms.py#L751

https://github.com/MIPT-Oulu/solt/blob/8738faeb088bcc276943963a14a94bd49f0b8a5f/tests/test_transforms.py#L1223

The attribute crop_size needs to be renamed into crop_to to match the constructor. Otherwise, the serialization does not work well.

https://github.com/MIPT-Oulu/solt/blob/770e397884bcafe80a11723c229e275c1c1f8b5a/solt/transforms/_transforms.py#L715

The user might want to specify that 1, 2, 3 or a randomly chose number of cutout holes needs to be made.

While testing one of the notebook algorithm, this error occured:
kpts = None for annotation_fname in glob.glob(os.path.join('Data', 'helen_annotations', '*.txt')): with open(annotation_fname) as f: if f.readline()[:-1] == fname.split('.')[0]: ktps = [] for l in f: tmp = l.split() ktps.append([float(tmp[0]), float(tmp[2])]) break kpts = np.array(ktps)

Specific Error:

NameError Traceback (most recent call last)
in
8 ktps.append([float(tmp[0]), float(tmp[2])])
9 break
---> 10 kpts = np.array(ktps)

NameError: name 'ktps' is not defined

I found useful to have multi-modal setups in some projects, thus, it might be of need to have also a numeric type, and remove the label type at all.

The transformation resize will distort ratio aspect if the input image is not square and resize_to is int.
https://github.com/MIPT-Oulu/solt/blob/50b064b398306ff0b21f8c39968e9201b697ddc1/solt/transforms/_transforms.py#L637-L638

I'd request to avoid that from happening. The feature is available in Resize of TorchVision (https://pytorch.org/docs/stable/torchvision/transforms.html#torchvision.transforms.Resize).

It would good to have list as accepted type here to support reading from yaml-file.

According to the benchmarks from albumentations, SOLT is too slow. It should work similarly or better than imgaug / augmentor: https://github.com/albu/albumentations#benchmarking-results