oulu-imeds / solt Goto Github PK
View Code? Open in Web Editor NEWStreaming over lightweight data transformations
Home Page: https://oulu-imeds.github.io/solt/
License: MIT License
Streaming over lightweight data transformations
Home Page: https://oulu-imeds.github.io/solt/
License: MIT License
README.md
has the old documentation link instead of the new one: https://oulu-imeds.github.io/solt/.
https://github.com/MIPT-Oulu/solt/blob/485bfb0d471134f75e09d06aa5e9ce4b57c0e13c/solt/transforms/_transforms.py#L1018 tells something about noise. Should talk about the intensities instead.
Support of different interpolation types is needed. Now only bilinear and bicubic are available
Repository has been moved to new location, and we thus need to adjust the deployment of docs on the website
Stack optimization is useful only when the transform time is very long. Therefore, it will be off by default in the newest release.
According to the benchmarks from albumentations, SOLT is too slow. It should work similarly or better than imgaug / augmentor: https://github.com/albu/albumentations#benchmarking-results
the augbench
package needs further development and its results need to be reported in the README.
the benchmark needs to cover random transformations rather than the static ones. It is important to make comparison for three cases: image, image mask, and image+10 masks (instance segmentation task).
Current serialization needs improvement. I does not work correctly all the time. We also need to make sure that we can load the transforms back after serialization
For segmentation it is often good to use nearest neighbors interpolation for masks, while for images bilinear or bicubic still needed to be kept.
Let's cover the mainstream cases when we need to unwrap the DataContainer, such as (img, label) and (img, mask).
Often happens when p(transform)=0.5. The pipeline crashes.
The use case for this is that for some images it might be better to rely on their mean intensity and increase their intensity by n%
of the mean value.
For images and numeric types
Need to enable different policies for augmentations
Works fine with 1.1, but not 1.5.
Traceback:
as_dict=as_dict, scale_keypoints=scale_keypoints, normalize=normalize, mean=mean, std=std,
File ".../site-packages/solt/core/_data.py", line 273, in to_torch
img.sub_(mean)
RuntimeError: output with backend CPU and dtype Byte doesn't match the desired backend CPU and dtype Float
Process finished with exit code 1
Need support for 32 and and 16 bit images.
Would be useful to increase the probability of augmentation for rare classes. This should also work in a multi-label setting
The user might want to specify that 1, 2, 3 or a randomly chose number of cutout holes needs to be made.
Module 3D transforms would be nice to have. First suggested transforms are crops, flips and rotations 90 degrees
Code to reproduce:
import solt
import solt.transforms as slt
import numpy as np
if __name__ == "__main__":
trf = solt.Stream([slt.Resize(resize_to=(50, 50)),
slt.Flip()])
img1 = np.ones((100, 100, 3), dtype=np.int32)
img2 = np.ones((110, 110, 3), dtype=np.int32)
trf_img = trf({'images': (img1, img2)})
print('Done')
Adding a resize transformation at the beginning doesn't help. A clear error message is needed for this case.
The transformation resize
will distort ratio aspect if the input image is not square and resize_to
is int
.
https://github.com/MIPT-Oulu/solt/blob/50b064b398306ff0b21f8c39968e9201b697ddc1/solt/transforms/_transforms.py#L637-L638
I'd request to avoid that from happening. The feature is available in Resize
of TorchVision (https://pytorch.org/docs/stable/torchvision/transforms.html#torchvision.transforms.Resize).
Shift-Scale-Rotate type of transform is needed as many libraries have it
Rotations by 90 degrees should not distort the image.
It would good to have list
as accepted type here to support reading from yaml-file.
Make random crop support the contours so that each crop's will belong to that contour
The attribute crop_size
needs to be renamed into crop_to
to match the constructor. Otherwise, the serialization does not work well.
While testing one of the notebook algorithm, this error occured:
kpts = None for annotation_fname in glob.glob(os.path.join('Data', 'helen_annotations', '*.txt')): with open(annotation_fname) as f: if f.readline()[:-1] == fname.split('.')[0]: ktps = [] for l in f: tmp = l.split() ktps.append([float(tmp[0]), float(tmp[2])]) break kpts = np.array(ktps)
NameError Traceback (most recent call last)
in
8 ktps.append([float(tmp[0]), float(tmp[2])])
9 break
---> 10 kpts = np.array(ktps)
NameError: name 'ktps' is not defined
Support of elastic transforms for keypoints, masks and images is needed
Example with satellite images
Warning, treated as error:
/home/travis/miniconda/envs/solt_test_env/lib/python3.7/site-packages/solt/transforms/_transforms.py:docstring of solt.transforms.IntensityRemap:15:Footnote [1] is not referenced.
@soupault Please fix.
Currently, selective stream takes a parameter that tells how many transforms to select from the list of transformations. I was recently suggested to make it a range and allow to sample random number of augmentations out of this range.
I found useful to have multi-modal setups in some projects, thus, it might be of need to have also a numeric type, and remove the label type at all.
Existing API allows to do this already. Bounding box is simply a label + 4 keypoints.
Bounding Boxes need to support different formats, such as COCO.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.