cioos-siooc / pyobistools Goto Github PK
View Code? Open in Web Editor NEWPython port of OBIS's obistools QC R package
License: BSD 3-Clause "New" or "Revised" License
Python port of OBIS's obistools QC R package
License: BSD 3-Clause "New" or "Revised" License
Looks like the OBIS / WoRMS checks are replicated across these two files.
taxa.py works against obis's API, but doesn't actually implement anything to do with ITIS, just has an empty placeholder function there.
I think taxa.py tries to name things and put things where a R obistools user might expect them, but that the check_scientificname_and_ids.py implementation is far more advanced and actually functions.
Suggestions on how to move forward? We could migrate the functionality from the check_scientificname_and_ids.py into the naming scheme from taxa.py , rename the check_scientificname_and_ids.py functions in-place, or decide not to be congruous with the R implementation, as we please.
Integrate the publishing of the documentation to GH Pages via GH Actions
Our fields check is far more robust than the current R offering, we should allow a user to specify whether to get confirmations of well-formed columns and entries in the returned report, and also support which formats to check compliance with: Event Core, Occurrence Core, Event w/ Occurrence Extension.
Acknowledgements template:
https://github.com/marbl/MUMmer3/blob/master/ACKNOWLEDGEMENTS
Crediting OGSL and DFO with the initial effort should come with some visual aid, perhaps we should add their logos to the README especially if it's not too big a file for either logo
Tracking the testing infrastructure setup using pytest
.
When querying Worms with a scientific name, the answer might might be longer than 1. Currently, the function returns the first answer. In the updated code, the functions should check the status of the answer it returns to make sure it is accepted.
A check can be made with scientific name 'Monstrilla grandis' to see how the function currently works.
R based doc:
https://github.com/iobis/obistools/blob/master/README.md
Build up a similar resource for the pyobistools version of the same tests.
Add this on the README? https://cioos-siooc.github.io/pyobistools/
Function check_scientificname_and_ids must handle empty values and ensure values are proper types before attemping operations on them
Running record of the packages that are flagged to me as i run through the check notebooks. So far it's been nbformat
and httpx
for me.
This will be a bit of an ongoing saga, but there are tests that do not behave as expected with latest Pandas. We need to find and pin a stable but fresh-er version of Pandas that doesn't break the current codebase.
If the Pandas expected behaviour changes, we will change with it at that point.
in addition to the presence check of field basisOfRecord, add a check of the column content to match DWC recommended vocabulary: https://dwc.tdwg.org/terms/#dwc:basisOfRecord
When a user uses 'names_taxons_ids' in check_scientificname_and_ids and obtain a result from Worms database for a given row that has an Itis LSID, it will return as a no match because the validation will be done comparing a Worms LSID to an Itis LSID even though the Itis LSID value could be valid.
recommended to use https://pypi.org/project/python-dateutil/ for date verification/parsing
Some of the functions have HTML and logic together in a single function, where we find those we should identify and split them into logic and display functions. ie:
The 4 archive types and their required fields are inline in the code right now, and it could be cleaner to separate them into definitional YML and then reimport them in the check functions.
yaml.load() will bring in a dictionary and can be crafted to produce the same structures as are expected by the function presently.
The aim is to have functions that perform roughly the same tasks as the R package functions in the pyobistools module, with implementations / decorators using those functions and still performing the same tasks they were already doing seamlessly.
First order functions to write/migrate:
pyobis and xylookup aren't registered with any package index, it would benefit us when using them as dependencies to get them registered into places they can be installed automatically.
We have text strings in bilingual French/English format right now, we can localize those strings with python i18n perhaps and open the door to translation into more languages.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.