Comments (10)
@mdsumner if it is not clear (as with nc files of consecutive times), the set of arrays can be bound as a collection of attributes, or ordered as subarrays along a new dimension in a single new target array. I believe there's no way ahead of time to say what makes more sense. You can now specify this with the along
arg to read_stars
. along=NA
will create a set of attributes, along="ensemble"
will create a new dim called ensemble
. The latter is easier if you want to reduce them using st_apply
(which you typically want). Also, split
and merge
(or c
) methods let you move stuff back and forth between attributes and dimension.
from stars.
Thanks!
from stars.
Thanks from me too, to both of you. A few interesting aspects here
from stars.
I think this is a feature. Why would you want to have layers with identical time stamps in a single object?
from stars.
I guess that's a matter of opinion, but a warning should at least be issued.
One example where this can happen to me using netCDF files, CDO and NCO: take 100 different simulations covering the same period which I want to compare with something. I take the time average of each of the simulations using NCO or CDO : this creates files with a single timestep (the average time). All the files will have the same time.
This might be a bad design decision on part of CDO and NCO, but there are reasons why this might be useful (e.g. being able to set standard time_bnds
attributes and values).
This means that before being able to import them into stars
as a single object I have to reopen them, remove the time, and add a stacking or ensemble dimension with a different value for each file.
Compare this with raster
, where I can simply stack
any number of files I want:
obs <- raster(observations)
s <- stack(filenames)
bias <- s - obs
#... do data analysis on the bias
#... or transform now to stars: st_as_stars(bias) will have a "band" dimension
This is a limitation in stars
by which dimensions must always be tied to some (usually physical) quantity present in the files themselves beforehand. While I do agree that this is the most correct way of doing things, it is much more tedious, which is annoying especially for taking a quick and dirty look at the data (which is often most of a scientist's work, isn't it).
I understand that stars
is this way by design; I'm not saying it shouldn't be so by default. However I think that either you should add a big warning when something like this happens, and/or that you should provide an option to layer multiple input files on an additional "virtual" variable.
from stars.
There's not so much design yet behind it, we just have to make it work. The point is, all layers should have equal time, but should vary along a different dimension (like ensemble member, or simulation).
Right now I think there's only checking for regular dimensions (where we have offset and delta), not for rectilinear ones, where the dimension values are in the "values" field of the dimension. I don't think there is checking going on that these values should be unique, so the array model of having a single record for each unique combination of dimensions is not held: dimension index yes, but dimension values are not uniqued, right now.
from stars.
The more I think about this the more I see how common of a use-case this is: opening the outputs from several different simulations and compare them on the fly. This is very easy to do with raster
, which stacks the file regardless of the layer dimension.
Maybe the most convenient approach in stars
would be an option to forcefully add a faux dimension in which to record from which file the data was taken. (it's not very elegant, is it)
from stars.
Isn't it a new attribute? (what am I missing?)
from stars.
I guess you could see it as a dimension but also as an attribute, yes. However I do not think you can perform actions (e.g. subsetting or using dplyr
verbs) over attributes. My "solution" feels a bit hackish to me, i'm sure something more appropriate is possible.
from stars.
@edzer awesome! Tested, works. Feel free to close if you think this is definitive.
from stars.
Related Issues (20)
- Pixels in the border get weird values with `st_rasterize()` HOT 5
- Enhancement Request: Direct Access to Band Values Using Band Names in stars Object HOT 2
- Non `udunits` confirm units break the read in of netcdf files HOT 5
- st_rasterize Error in rep_len HOT 4
- Unexpected behavior: st_transform a raster and transforming back, it is not identical unless applying st_warp HOT 7
- Three different stars object descriptions from the same file HOT 4
- Proposal: Record time dimension in stars objects using CFtime package HOT 6
- Issues with particular file: North American Land Cover (NALC) Dataset HOT 5
- Additional nest in dimension while reading HDF5 file and terra workaround solution HOT 7
- GDAL Access window out of range on read-in of stars_proxy HOT 2
- aggregate.stars mean by time (by = "2 days") HOT 2
- netcdf dimensions get messed up if they are out of order and if axis attributes are available HOT 12
- starsdata not available, stars fails revdeps. HOT 7
- issue using `st_cells()` on a cropped image HOT 3
- workflow recommendations for simple spatial mean computations HOT 5
- An error occurred while using the st_get_dimension_values function to obtain the time HOT 1
- `mutate.stars` fails on attributes with spaces in its name HOT 1
- A better (efficient) way to convert a list to stars object with time dimension having only one value HOT 2
- Error with `mdim` test: `Error: cannot open driver` HOT 8
- Some tests cannot find files from `stars/nc` directory HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from stars.