poisotlab / gbif.jl Goto Github PK

When calling qualitycontrol! followed by next!, the actual number of records is not counted correctly, which results in the wrong occurrences being returned. As the filters are not part of the GBIFRecords object (but this may be an improvement for a future version), the solution is most likely to call showall! when next! is called.

Currently, taxon performs only text search, but sometimes the ID of the taxon is known (for example, when we want the genus of a given species). This would require a second method.

The taxon function is defined here: https://github.com/EcoJulia/GBIF.jl/blob/master/src/taxon.jl

The GBIF API endpoint for taxa by ID is here: http://api.gbif.org/v1/species/216

Because the paging system uses Int for the number of pages, offset, etc., the Dict containing the query must be typed appropriately. One solution would be to use Pairs (as in mangal filtering), which can also be passed using ... (and this would remove two methods for `occurrences at the cost of breaking the current code (but I can probably still use the dict and give a deprecation warning).

It's hard to change a query after occurrences. It would make sense to be able to update e.g. "limit" easily.

• Taxon types
• Remove references to Query
• Taxon search
• General updates

Paging through occurrences is done with push operations, and this gets progressively slower over time - need to benchmark how much.

The need to push can be removed through the pre-allocation of the arrays as we know the number of occurrences we will retrieve.

See if it is possible to use Query to filter occurrences.

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

Apparently GBIF does not validate the dates, so sometimes the date can have a day as 31 in a month with 30 days. This is an easy fix, the function to parse the date needs to be wrapped in a try/catch block, and return missing is the date is F'ed up.

My best guess as to why is: sometimes the most recent observations have no issues, and so the tests (which is testing the fact that the length before/after changes) are failing. This can be solved by putting a function which will remove some records e.g. at random, like f(g::GBIFRecord) = rand() < 0.5.

Hi, is this package not 1.0 compliant? If not, what would it take to make it? I'd really like to use this.

The documentation is not building (and the code coverage result are not sent to codecov)

• Occurrence
• Occurrences

See https://docs.julialang.org/en/stable/manual/types/#Custom-pretty-printing-1

This causes the occurrences function to fail.

Requests is about to be deprecated.

On the feature-view branch, update! is not accounting for filters. Ideally, the workflow would be

1. get first batch
2. apply filters
3. update sets while filters are applied in real time

size and view currently don't use base semantics. Single-arg size(x) usually returns a Tuple, and view(x) normally returns a zero dimensional SubArray.

It might make sense if values added with occurances! are appended to an existing array that would start with zero length, so that none of these things are even needed. Then GBIFRecords <: AbstractVector would makes sense too, and all of these methods would have the normal semantics. length would just be the length of the parent Array.

@mkborregaard that also means usingcollect to get all the records might not make sense? as it should return an Array but we actually still want a GBIFRecords object, and if it collected all the records, it also ends up longer than what length returns for the GBIFRecords - which is also kind of confusing for how collect usually works.

I feel like this is more like reading a stream to an array, so it's possible we could use e.g. read!(oc) to mean read the whole stream?

Currently, a lot of fields in Occurrence are Union{Whatever, Void} -- it will make more sense to have the rely on Missing.

This would make a lot of workflows easier, and we can still warn the user.