Comments (6)
Some field names changed around April, suspect this was one of them. Looking at ala_fields("occurrence_indexed")
I see a field called outlier_layer
(whose description is "Outlier for layer"). Is this it?
The corresponding column name in the occurrence download data is outlierLayer
.
We do prettify the column names in ALA4R, which was originally to try and make them consistent across functions. With the new field name changes we'll need to look at that at make sure we're still doing sensible things. Backwards compatibility might become an issue here, but we'll see how we go I guess.
from ala4r.
Thanks @raymondben - I saw the outlier_layer
column, but the data in it is stuff like el845
or whatever, so I think it means that the data for those environmental layers are considered outliers(?). Previously the field was a TRUE/FALSE indicating a suspected spatial outlier.
I suspect it's simply gone, will dig around a bit more. Thanks again for your quick response, and once again for your efforts with this fantastic package.
from ala4r.
The contents of the el845
etc environmental fields are populated from gridded environmental data, so if the position of the observation is a spatial outlier then those values will be outliers with respect to the norm for that species. So I'm guessing that previously the "outlier" status was calculated on the basis of environmental layers and now it is just giving more info about which layers indicate its outlieriness ... but I'm only guessing. @nickdos @adam-collins - can you enlighten us?
from ala4r.
When we introduced our "offline" downloads, we changed the way the download file was generated but tried to keep most fields the same. The old download format is still available but is limited to 100,000 records and the new newer offline has no limit (for now). E.g. http://biocache.ala.org.au/ws/occurrences/download?q=genus:Macropus
(e.g. without the /offline
part).
The difference is the old download is produced directly from the SOLR index, whereas the new download is produced from the Cassandra database directly (SOLR index is a subset* of data in Cassandra). The outlier_layer
is only in the SOLR index* I think, so we either need to calculate that value on the fly for the offline download, etc.
For now, the SOLR download is still available, so I think ALA4R could provide an option to use the older SOLR download (100,000 max) or the newer offline download. The SOLR download will be quicker and may suite some users better but it makes the API more complicated, trying to explain the existence of 2 similar but slightly different APIs.
Another work around () would be to use the web interface to build 2 queries, one with records where detectedOutlier
is true and another where it is false. Then trigger 2 downloads and then merge them after manually setting the values for detectedOutlier
... if that makes sense.
from ala4r.
ALA4R does both the indexed (SOLR) and offline methods. The outlierforLayers column seems to be returned in both.
I think our main question here is whether that field is equivalent to the old Suspected.outlier
or detectedOutlier
field.
from ala4r.
Thanks once again @nickdos! And @raymondben I may have the answer to that question. Just looking at a recent download, the outlierForLayer
field has data like el882
, el865
etc, which refer to bioclimatic variables such as temperature, precipitation, etc. This is also the same as Outlier for layer
and Outlier layer count
filters on the web interface.
A download from some months ago includes both the Outlier.for.layer
field (with data as above) and the Suspected.outlier
field, which to my understanding is a TRUE/FALSE indicator of spatial outliers. So they seem to be different fields, yes. It would be great to know if a) the spatial outlier field still exists, and b) if it can be gotten at.
EDIT: I've tried Nick's suggestion for the old download format, it doesn't have suspectedOutlier
but does have the Outlier.for.layer
though it's a 0/1 field.
As usual, many thanks to everyone, and any assistance is greatly appreciated :)
P.S. I should add, I'm not 100% sure that the old suspectedOutlier
field was actually what I think it was!
from ala4r.
Related Issues (20)
- Make package skeleton HOT 1
- Change Macropus genus in example HOT 1
- Google analytics
- Document API dependencies HOT 2
- Case study - Latitudonal gradient analysis HOT 1
- Case study - area report HOT 1
- Case study - text mining Darwin Core terms HOT 1
- Develop ALA4R training materials
- sites_by_species can fail on valid WKT
- ala_list() test failing HOT 1
- ALA4R Travis build failing
- problem accessing records using occurrences() HOT 6
- Sites by species failing while waiting for task to complete
- Make use of new image web service
- Generate DOIs for downloads HOT 1
- Publish to CRAN - v 1.8.0 HOT 1
- Retrieve data resource information
- Implement facet search
- Include data quality filters in ala_occurrences
- Progress bar for downloads, or an initial count on how many records to expect HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ala4r.