oulu-imeds / solt Goto Github PK

I found useful to have multi-modal setups in some projects, thus, it might be of need to have also a numeric type, and remove the label type at all.

Make random crop support the contours so that each crop's will belong to that contour

According to the benchmarks from albumentations, SOLT is too slow. It should work similarly or better than imgaug / augmentor: https://github.com/albu/albumentations#benchmarking-results

README.md has the old documentation link instead of the new one: https://oulu-imeds.github.io/solt/.

https://github.com/MIPT-Oulu/solt/blob/0f462eae0b49489caa69cb652d484fe6cc4b7737/solt/transforms/_transforms.py#L751

Rotations by 90 degrees should not distort the image.

the augbench package needs further development and its results need to be reported in the README.

the benchmark needs to cover random transformations rather than the static ones. It is important to make comparison for three cases: image, image mask, and image+10 masks (instance segmentation task).

Shift-Scale-Rotate type of transform is needed as many libraries have it

https://github.com/MIPT-Oulu/solt/blob/485bfb0d471134f75e09d06aa5e9ce4b57c0e13c/solt/transforms/_transforms.py#L1018 tells something about noise. Should talk about the intensities instead.

Often happens when p(transform)=0.5. The pipeline crashes.

https://arxiv.org/pdf/2001.04086.pdf

https://github.com/MIPT-Oulu/solt/blob/0f462eae0b49489caa69cb652d484fe6cc4b7737/solt/transforms/_transforms.py#L1129

The user might want to specify that 1, 2, 3 or a randomly chose number of cutout holes needs to be made.

Example with satellite images

The transformation resize will distort ratio aspect if the input image is not square and resize_to is int.
https://github.com/MIPT-Oulu/solt/blob/50b064b398306ff0b21f8c39968e9201b697ddc1/solt/transforms/_transforms.py#L637-L638

I'd request to avoid that from happening. The feature is available in Resize of TorchVision (https://pytorch.org/docs/stable/torchvision/transforms.html#torchvision.transforms.Resize).

Warning, treated as error:
/home/travis/miniconda/envs/solt_test_env/lib/python3.7/site-packages/solt/transforms/_transforms.py:docstring of solt.transforms.IntensityRemap:15:Footnote [1] is not referenced.

@soupault Please fix.

Stack optimization is useful only when the transform time is very long. Therefore, it will be off by default in the newest release.

The use case for this is that for some images it might be better to rely on their mean intensity and increase their intensity by n% of the mean value.

Module 3D transforms would be nice to have. First suggested transforms are crops, flips and rotations 90 degrees

Support of elastic transforms for keypoints, masks and images is needed

Let's cover the mainstream cases when we need to unwrap the DataContainer, such as (img, label) and (img, mask).

https://github.com/MIPT-Oulu/solt/blob/8738faeb088bcc276943963a14a94bd49f0b8a5f/tests/test_transforms.py#L1223

For segmentation it is often good to use nearest neighbors interpolation for masks, while for images bilinear or bicubic still needed to be kept.

Would be useful to increase the probability of augmentation for rare classes. This should also work in a multi-label setting

It would good to have list as accepted type here to support reading from yaml-file.

The attribute crop_size needs to be renamed into crop_to to match the constructor. Otherwise, the serialization does not work well.

https://github.com/MIPT-Oulu/solt/blob/770e397884bcafe80a11723c229e275c1c1f8b5a/solt/transforms/_transforms.py#L715

Need support for 32 and and 16 bit images.

Need to enable different policies for augmentations

Currently, selective stream takes a parameter that tells how many transforms to select from the list of transformations. I was recently suggested to make it a range and allow to sample random number of augmentations out of this range.

Repository has been moved to new location, and we thus need to adjust the deployment of docs on the website

While testing one of the notebook algorithm, this error occured:
kpts = None for annotation_fname in glob.glob(os.path.join('Data', 'helen_annotations', '*.txt')): with open(annotation_fname) as f: if f.readline()[:-1] == fname.split('.')[0]: ktps = [] for l in f: tmp = l.split() ktps.append([float(tmp[0]), float(tmp[2])]) break kpts = np.array(ktps)

Specific Error:

NameError Traceback (most recent call last)
in
8 ktps.append([float(tmp[0]), float(tmp[2])])
9 break
---> 10 kpts = np.array(ktps)

NameError: name 'ktps' is not defined

For images and numeric types

Support of different interpolation types is needed. Now only bilinear and bicubic are available

Current serialization needs improvement. I does not work correctly all the time. We also need to make sure that we can load the transforms back after serialization

Code to reproduce:

import solt
import solt.transforms as slt
import numpy as np


if __name__ == "__main__":
    trf = solt.Stream([slt.Resize(resize_to=(50, 50)),
                       slt.Flip()])
    img1 = np.ones((100, 100, 3), dtype=np.int32)
    img2 = np.ones((110, 110, 3), dtype=np.int32)
    trf_img = trf({'images': (img1, img2)})
    print('Done')

Adding a resize transformation at the beginning doesn't help. A clear error message is needed for this case.

Existing API allows to do this already. Bounding box is simply a label + 4 keypoints.

Bounding Boxes need to support different formats, such as COCO.

https://arxiv.org/pdf/2003.06158.pdf

Works fine with 1.1, but not 1.5.
Traceback:

    as_dict=as_dict, scale_keypoints=scale_keypoints, normalize=normalize, mean=mean, std=std,
  File ".../site-packages/solt/core/_data.py", line 273, in to_torch
    img.sub_(mean)
RuntimeError: output with backend CPU and dtype Byte doesn't match the desired backend CPU and dtype Float
Process finished with exit code 1