ceades / pytorchh5dataset Goto Github PK
View Code? Open in Web Editor NEWThis project is an accelerated dataset for PyTorch. The dataset stores compressed batched data that is cropped while loading. By this, I/O operations and network traffic are reduced. Since the cropping is performed at loading time, the overall loaded data is reduced. Slow lossy jpeg encoding is dropped in favor of a lossless blosc compression. It comes with a function to create a dataset from a directory.
License: BSD 3-Clause "New" or "Revised" License