Comments (4)
From [email protected] on November 18, 2008 21:38:39
These properties exist in the current svn version; they'll be part of the upcoming
1.0 release. One issue with an "extendable" flag may be that not every axis is
guranteed to be extendable; if you declare the maxshape as (10,None,10), then only
the second axis can be extended. I'll see if there's a way to deal with this.
from h5py.
From [email protected] on November 19, 2008 13:48:47
Thanks! Regarding 'extendable', a developer will need to know the axises and
chunksizes to extend, so the 'None in t.maxsize' is about as good as we can get as a
single boolean flag. I could forsee uses like:
def extend(t,newchunks):
rank = len(t.shape)
if isinstance(newchunks,int):
newchunks = [newchunks]_rank
if len(newchunks) != rank:
raise ValueError('Invalid rank')
newsize = []
for nchunks,n,msize,csize in izip(newchunks,t.shape,t.maxsize,t.chunksize):
if msize is None:
raise ValueError('Cannot resize axis')
newsize.append( n+nchunks_csize )
t.resize( tuple(newsize) )
I'd rename the current 'extend' to 'resize', since it takes an absolute size and
rework 'extend' to do as above.
from h5py.
From [email protected] on November 19, 2008 15:39:22
It turns out you don't need to make the dataset size an exact multiple of the chunk
size; although HDF5 will allocate storage space in chunks you can specify any shape
you want. Dataset chunking is supposed to be transparent; I'm a little suspicious of
any method that wants you to do chunk arithmetic.
Extend() has been deprecated because it no longer hooks in to the HDF5 H5Dextend()
function, which actually did something slightly different (guaranteed a minimum
size). H5Dextend has coincidentally been deprecated in HDF5 1.8.
If you want to grow an existing array by a fixed amount you can do something like:
dset.resize([x+y for x, y in zip(dset.shape, extension_sequence)])
or for a single axis
dset.resize(dset.shape[ax]+amount, axis=ax)
from h5py.
From [email protected] on November 19, 2008 16:09:37
That simplifies everything. Never mind-- a plain resize works fine for me now that I
know that the chunking is a back end detail (to my great relief). I'd rather not
have to worry about growing by fixed amounts and simply resize arbitrarily as needed,
provided I don't change fixed dimensions.
from h5py.
Related Issues (20)
- Numpy 2.x incompatibilities HOT 3
- leak if we use the "try dict[key] / except KeyError" idiom with h5py attrs HOT 7
- Not finding libhdf5.so on ubuntu when building h5py from source. HOT 5
- Numpy 2.0 compat plan HOT 11
- CVE-2023-45853 - zlib.dll is installed as a dependency for h5py HOT 11
- undefined symbol: H5Rcreate_object HOT 4
- UserWarning of `numpy.longdouble` trigger when importing h5py with numpy>=1.25 HOT 3
- Add api to `h5py.File` for reading and writing userblock. HOT 13
- [Question] ways to speed up reading data slice HOT 3
- Remember to unpin pytest HOT 3
- Disable Fletcher32 check before reading HOT 3
- Clarify description of parallel hdf5 HOT 2
- Supporting new HDF5 data types
- Memory leak with dataset `fields` function HOT 2
- h5py unable to create file in gcs bucket. HOT 6
- VirusTotal has found out a trojan "Trojan.Malware.300983.susgen" HOT 4
- VirusTotal has found out a trojan "Trojan.Shelma.neo" HOT 1
- VirusTotal has found out trojans "Trojan.Inject.cgqx" and "Trojan.Malware.300983.susgen" HOT 1
- h5py build errors when building with gcc 14 HOT 3
- BUG: Nightly wheels not 2.0 compatible HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from h5py.