The gnresolver from globalnamesarchitecture

gnresolver's Issues

terms which start with the same letters as modifiers break parsing

see "BUGS" context in QueryParserSpec

Complete frontend for new bunch of API search modifiers

use Typescript

Upload and import database to server

Limit: 5 hours

Successful name_strings search results always have type Fuzzy(0)

We probably should not have Fuzzy search kind at all for name_strings search matching.

Currently we have searches like

http://gnresolver.globalnames.org/api/name_strings?search_term=Pardosa%20moesta

having "Fuzzy(0)" for all the results

{
nameString: {
name: {
id: "b18f184f-1d46-572d-90ee-bb4f3687ea57",
value: "Pardosa moesta Kaston, 1948"
},
canonicalName: {
id: "aefc0ec7-f875-5b83-9cb7-ff71cc790994",
value: "Pardosa moesta"
}
},
dataSourceId: 169,
kind: "Fuzzy(0)"
}

use parsing quality result as a component to calculate a matching score

@dimus commented on Tue May 10 2016

For example if canonical form had been generated from a name string with parsing problems -- penalize resulting score for matches with such canonical form. For example see comment from

GlobalNamesArchitecture/gni#48 (comment)

Implement fuzzy matching by canonical name

Build canonical names string list

Use parsing quality result as a component to calculate a matching score

@dimus commented on Tue May 10 2016

For example if canonical form had been generated from a name string with parsing problems -- penalize resulting score for matches with such canonical form. For example see comment from

GlobalNamesArchitecture/gni#48 (comment)

Extract importing subproject to a separate project

Create DB schema for effective match by UUIDs

UUIDs of exact match and of canonical form

Show data source and links for a scientific name

Implement faceted search API calls from UI

Add full text search that starts with lowered case letter

Implement simple case: capitalize the first letter

Research how to scale Postgres DB

Ensure that everything is addressable in search engines

Make sure that Angular does not break things

Figure out what to do with names where parser UUID differs from UUID at MySQL

We don't have to store additional column (MySQL UUID) in DB anymore. I just put those 37 records here:

id  id_mysql    name    canonical_uuid  canonical
198608fe-1017-5f8a-8afd-163a4f98a03c    c1385f67-de4a-5c0c-91ef-ea5f366aff66    Lasionycta perplexella Crabo et Lafontaine, 2009    9001d45e-073b-5963-ad6b-919695b35a68    Lasionycta perplexella
9e6a5af0-4d72-5528-8cee-6ab14dde3ef3    4351cf1b-53c8-5a3b-adaa-02f011a506a2    Argemone mexicana Wikispecies   64e0e820-7a6a-55a8-876f-ee9fea1c4c7d    Argemone mexicana
e50d70b3-7cdf-584b-bf64-fa179f278a66    7bf29604-9597-5e08-8d58-540ebfabb2bf    Pseudodrephalys hypargus Mabille, 1891 See text.    28df501d-f98c-5b43-b23d-534dde48119b    Pseudodrephalys hypargus
f4fa81e8-f137-5c67-95e3-272f0d4e5b28    b9026034-82da-55d2-803e-3f0feba662ea    Pseudodrephalys sohni Burns, 1998 See text. 1e40ee3a-0b1b-52d3-9e7e-895df0f0ab67    Pseudodrephalys sohni
d0b57842-cecc-5a6b-b726-8e46a7983c24    1a0e79af-ad45-5a8c-801b-a71ce09f88c5    Pseudodrephalys atinas Mabille, 1888 See text.  1d34d3d1-cb0f-5154-acfe-2d5b8c2d4a7e    Pseudodrephalys atinas
52cf794d-8cfa-5416-aafa-a23574a1aebe    48372bef-f0e0-53ab-857b-48aae84ad496    Pediocactus winkleri K.D.Heil Winklers Cacuts photo taken near Clawson, Utah.   1ada0d5d-dc31-54fb-8845-ab904da27dfe    Pediocactus winkleri photo taken
55738f44-e108-5ddd-8510-542d26742dce    87cb4568-cc40-5960-af01-de689e4b2c0d    Rumex patientia L. Sources: ING, UniProt, ITIS, IPNI    4bf7b6be-cc49-5fad-a711-9a2021168906    Rumex patientia
6b475e86-79b8-509f-a164-f42bc6b88c72    444a4744-1029-5787-8100-8309e329504c    P. rangei P. vanzyli    63702596-1f2f-5fc4-bc33-621321edb432    P. rangei vanzyli
d8eefa3a-c94b-5f9f-abb1-7a082f35bd9b    12b6c03e-514f-562e-a8bc-ed91f92d00b1    Ambrosia trifida L. (1753) Source: IPNI 1c3c9c9d-2f94-5db9-9def-9b85f0230cb9    Ambrosia trifida
36bcbc03-2834-59b0-b0b6-ed3da7f7f9ed    2f0255a3-0021-538d-be83-1633c2d21c5e    Anaphalis margaritacea (L.) Benth. & Hook.f. (1873) Sources: NatureServe, IPNI, GRIN    f681a2d2-4748-59d9-9952-94d6ae1fd95b    Anaphalis margaritacea
cc083908-3af9-522f-9f40-85fef6ceefa8    12fe0747-ee87-58f8-a28f-9dea60cca6ed    C. longiceps. (1937). C. kitchingi. C. rubidgei. C. broomianus. C. tenuirostris.    c4d35606-d519-54b3-a2da-8d1a3613eaec    C.
5cefde05-25cd-506b-97e2-004d12dfa3c1    7bacd08b-b058-5301-ad94-5ccb5474ce15    Alophia Herb. Bot. Mag. 66: t. 3779 (1840). e198a17d-b0ea-5fb6-b14f-d205b1ada9fd    Alophia
b2ecd7ae-4d26-50f6-9f89-ebeab8762f79    760e07c6-0f67-530a-a618-886bc1d2e131    Lobocheilos bo bo (non Popta, 1904) 3c6f58f5-ddeb-546b-88ad-d740cda00f3b    Lobocheilos bo bo
81601e6b-c2ae-56bb-aaca-5a4613fd4798    144756cb-5f3b-5232-9099-e6c5b21949d5    Rhabdolithus siccus Stradner in Bachmann, Papp & Stradner (1963) ., 1963    48dc1337-c434-5a5e-ac0b-666df08f7bc5    Rhabdolithus siccus
945d713f-1881-5975-86f3-d892bb64ce53    41779834-092a-5ea9-87a3-787934f9a561    Sirvid, P.J. et al . 2010: [Chapter] SIX Phylum ARTHROPODA CHELICERATA horseshoe crabs, arachnids, sea spiders. Pp. 50-89 in Gordon, D.P. (ed.): New Zealand inventory of biodiversity. Volume 2. Kingdom Animalia. Chaetognatha, Ecdysozoa, ichnofossils . Can 1a9e9055-4579-59bd-9a83-795574d7dd27    Sirvid
b0aff339-147e-5f48-8317-0aa0f455da20    c7c34951-0022-5718-a0e8-b010b66de708    Banksieaephyllum Cookson & Duigan , in Cookson, Isabel C. and Duigan, Suzanne L. (1950). "Fossil Banksieae from Yallourn, Victoria, with notes on the morphology and anatomy of living species". Australian Journal of Scientific Research, Series B (Biologica d9ea5a2b-45d2-5490-9d90-5810b31d61e9    Banksieaephyllum
edc837d7-ef52-5801-affc-f54c9a0bc08b    be9a60fe-74e6-5878-bfb9-7d25a8a15909    Calodactylodes illingworthorum <ref>Named after Margaret and Percy Illingworth. As the name honours two people named Illingworth, the specific epithet is incorrect and must be emended to illingworthorum according to BAUER & DAS (2000).</ref> Deraniyagala, 97529c29-7b7f-54b3-a6d5-4cc8594729c3    Calodactylodes illingworthorum
6942cf3d-5448-5316-873f-ceb421d6f406    9381027a-4143-5a99-9e80-5cf24ed754ad    Diorhabda carinata (Faldermann, 1837)<ref>This species was incorrectly synonymyzed under Galeruca elongata by Reiche and Saulcy (1858) and removed from synonymy with D. elongata by Berti and Rapilly (1973). Tracy and Robbins (2009) provide keys to separat 82c19df3-f224-5897-961f-72758ba2c950    Diorhabda carinata
2fa15b34-f889-5e0b-8843-2e7b012fefd2    7afbd3f3-3abe-5c2d-bf72-10e3b467ef7e    Diorhabda sublineata (Lucas, 1849)<ref>This species was incorrectly synonymyzed under Galeruca elongata by Reiche and Saulcy (1858) and removed from synonymy with D. elongata by Tracy and Robbins (2009). Tracy and Robbins (2009) provide keys to separate D 571f5eac-b2cb-5cc8-8d39-7bbdad6557d1    Diorhabda sublineata
fea29807-52ea-593e-89d8-d5bfe6c83bae    c04b41d4-212e-5e92-bd32-03732a511233    Diorhabda meridionalis Berti & Rapilly, 1973<ref>Tracy and Robbins (2009) elevated D. meridionalis from a subspecies of D. carinulata to species status. Tracy and Robbins (2009) provide keys to separate D. meridionalis from four sibling species of the Dio ed02bc0f-4a86-5f13-bbab-0c5bfe96fa45    Diorhabda meridionalis
0a284cee-da19-53b5-a082-ce4ddf620f00    1cf21419-f04b-5b79-b4f5-8902ee37881b    Diorhabda carinulata (Desbrochers, 1870)<ref>This species was incorrectly synonymyzed under D. elongata var. carinata by Weise (1893) and removed from synonymy with D. elongata by Berti and Rapilly (1973). Tracy and Robbins (2009) provide keys to separate 8f8fe062-c605-5764-8056-453112633892    Diorhabda carinulata
f6547f6f-192a-59bb-a8c4-cfb61de69011    d3b64357-5ecd-51e1-8c34-a7a760b39bfd    Lepilemur hubbardorum Louis Jr., Engberg, Lei, Geng, Sommer, Randriamampionona, Randriamanana, Zaonarivelo, Andriantompohavana, Randria, Prosper, Ramaromilanto, Rakotoarisoa, Rooney & Brenneman, 2006 <!--emmend. Hoffman, Grubb, Groves, Hutterer, Van der S c21e0ed1-65ea-5586-9986-94222960207c    Lepilemur hubbardorum
c0d6ded7-2524-5f6f-a843-e537dfd7be2e    f0e5a9b4-f66d-571b-8fdb-cb68c28e34a4    Birlimarr gaffneyi Megirian, D. & Murray, P. 1999.<ref name=MN99>Megirian, D. & Murray, P. 1999. Chelid turtles (Pleurodira, Chelidae) from the Miocene Camfield Beds, Northern Territory of Australia, with a description of a new genus and species. The Beag 50ef1751-1aba-5935-bada-05eaf868275a    Birlimarr gaffneyi
8eb3aaf7-2f39-5fa7-8831-dc681cd5cd45    1e9ec54b-92a6-561d-88c1-6acaac568e73    Lindmanioideae Givnish, 2007 in: Thomas J. Givnish, Kendra C. Millam, T. M. Evans, J. C. Pires, Paul E. Berry, Kenneth J. Sytsma: Phylogeny, biogeography, and ecological evolution in Bromeliaceae-insights from ndhF sequences. , in J. T. Columbus, E. A. Fr 6c2d5d63-f1cb-5576-b806-40c22d5167d4    Lindmanioideae
f934f252-0f78-58b4-a923-b47df29ab320    5b9833de-b0de-544c-8941-04150051672a    Dorylaimia Inglis 1983  5651f08d-e575-52c6-b7aa-57da693cef80    Dorylaimia
56baa39e-22c0-5f44-9511-03c4321feee5    ce1dfd8a-78e6-5d4b-b896-974f38bf2803    Zelmiarcha Grimaldi, Amorim & Blagodervo 2003: 372 Archizelmiridae  527f1207-e711-50b2-aff1-3881a2a564ec    Zelmiarcha
fbdc83d8-250c-5366-9748-94a3f5bfa13d    670bddd4-cbbb-55ba-9d1c-37ac9e6854bc    Veriplecia Blagoderov 1999: 12 Paraxymyiidae    df65845b-0bce-5a32-8311-426b341c9099    Veriplecia
d0c1e1d9-d789-5276-99bc-769980854970    abf1d232-0660-5de2-9a99-d0c4a6bdc016    Virginiptera Blagoderov & Grimaldi 2007: 15 Paraxymyiidae   c7409901-0baf-5def-a57e-d1639cac3bd8    Virginiptera
8ca6ade0-3f9a-541b-bc85-c98d49baebed    38fce6ec-c2ca-5e7e-8eb7-3ae1fc6a7aa1    Virginiptera certa Blagoderov & Grimaldi, 2007: 16 Paraxymyiidae    910953ae-0818-5a4a-80d0-40bca1b3affb    Virginiptera certa
96a24a16-699d-54f7-8b3c-d9f03c88fa1b    d5374a74-4f6f-57a5-9af5-2066e843a089    Delarbrea balansae Lowry PP GM Plunkett, V Raquet, TS Sprenkle, J Jérémie 2004 Inclusion of the endemic New Caledonian genus Pseudosciadium in Delarbrea (Apiales, Myodocarpaceae) Adansonia 26 2 251–256 http://www.mnhn.fr/museum/front/medias/publication/32 f72092b6-145b-5316-b3ab-d07f9a7cce91    Delarbrea balansae
fc01d6bf-152c-5243-80f5-2944e3c11098    ed1ff214-34f8-540c-91a8-f7b3664150f7    Hydrocotyle bonariensis http://plants.usda.gov/java/profile?symbol=HYBO PLANTS Profile, Hydrocotyle bonariensis 2008-04-25 Natural Resources Conservation Service (NRCS) Natural Resources Conservation Service HTML The PLANTS Database [[United States Depart 16a183bf-72a6-563c-91b9-2e1111c05a77    Hydrocotyle bonariensis
a2a7c247-af60-53dc-9c5d-919fab587440    1cb3c5f9-70df-52f1-82e7-9e503d18d625    Lotononis platycarpa http://www.ars-grin.gov/cgi-bin/npgs/html/taxon.pl?410277 Taxon: Lotononis platycarpa (Viv.) Pic. Serm. 2004-08-20 Germplasm Resources Information Network (GRIN) Germplasm Resources Information Network 2007-10-05 HTML Taxonomy for Pla f9142d62-e9c2-56d2-91e9-bfbf244734f4    Lotononis platycarpa
a67ac8c3-a686-5408-b7c7-d059222f5f7e    3c29405a-18a8-5c98-882f-753304cf44af    Juncus acutus http://www.ars-grin.gov/cgi-bin/npgs/html/taxon.pl?447103 Taxon: Juncus acutus L. 2008-04-26 Germplasm Resources Information Network (GRIN) Germplasm Resources Information Network 2004-09-08 HTML Taxonomy for Plants [[United States Departmen a3662a99-91d8-55dd-9a02-e52ea782ba9d    Juncus acutus
b48df8e4-60f7-5469-9d71-59bfa65ecfda    8903484f-3077-51fa-a6a9-bc2cef60aa48    Senecio crassiflorus http://www.ars-grin.gov/cgi-bin/npgs/html/taxon.pl?33658 Taxon: Senecio crassiflorus (Poir.) DC. 2008-04-23 Germplasm Resources Information Network (GRIN) Germplasm Resources Information Network 1985-11-07 HTML Taxonomy for Plants [[U 3860d3b7-97e5-5402-8fb0-523a0b9867bf    Senecio crassiflorus
335fd9c3-eaf5-5365-ba2d-42bc8733d82e    c91b4fbb-9322-539d-9ade-5ba1deed87f0    Plantago amplexicaulis http://www.ars-grin.gov/cgi-bin/npgs/html/taxon.pl?28779 Taxon: Plantago amplexicaulis 2008-04-22 Germplasm Resources Information Network (GRIN) Germplasm Resources Information Network 2002-12-16 HTML Taxonomy for Plants [[United St efe25b8c-e91f-5c78-b868-f15c0d35934d    Plantago amplexicaulis
0793f1b7-538c-5f71-b237-27746a48c357    24c38287-0290-553d-b88d-9faf7376e2fe    Senecio leucanthemifolius http://www.ars-grin.gov/cgi-bin/npgs/html/taxon.pl?450218 Taxon: Senecio leucanthemifolius Poir. 2008-04-23 Germplasm Resources Information Network (GRIN) Germplasm Resources Information Network 2007-01-03 HTML Taxonomy for Plant 5cb77fb6-8887-5bdb-a346-cb7771815fb2    Senecio leucanthemifolius
4ff2e40b-4bcc-5be6-8368-c926e097f680    4b2e1ecb-bf8a-5f0f-84b2-5a21e8ce8892    Atriplex hortensis http://www.ars-grin.gov/cgi-bin/npgs/html/taxon.pl?6017 Taxon: Atriplex hortensis L. 2008-03-23 Germplasm Resources Information Network (GRIN) United States Department of Agriculture, Agricultural Research Service, Beltsville Area http: dbd6dbfe-ea87-51aa-aabf-75fe21af2765    Atriplex hortensis

Create simple web UI

static resources management
current API methods calls

Server error instead of empty results

http://gnresolver.globalnames.org/api/name_strings?search_term=ddfds

returns "There was an internal server error."

It should return empty result.

API: Merge request by `uuid` of verbatim and canonical

Tune SBT for DB seeding for different environments (develop, production, test)

apply to languages table

Create gnmatcher_name_strings_canonical.dump in System.tmp folder

As a user of gnresolver I need resolution information existed in json output for gni and resolver

We need following information in output by importing data from database.

data_source_id: 4,
data_source_title: "NCBI",
gni_uuid: "2cf19440-46c2-52c5-9fce-d66194286102",
name_string: "Pomatomus saltator",
canonical_form: "Pomatomus saltator",
classification_path: "|Eukaryota|Opisthokonta|Metazoa|Eumetazoa|Bilateria|Deuterostomia|Chordata|Craniata|Vertebrata|Gnathostomata|Teleostomi|Euteleostomi|Actinopterygii|Actinopteri|Neopterygii|Teleostei|Osteoglossocephalai|Clupeocephala|Euteleosteomorpha|Neoteleostei|Eurypterygia|Ctenosquamata|Acanthomorphata|Euacanthomorphacea|Percomorphaceae|Pelagiaria|Scombriformes|Pomatomidae|Pomatomus|Pomatomus saltator",
classification_path_ranks: "|superkingdom||kingdom||||phylum|subphylum|||||superclass|class|subclass|infraclass|||||||||||order|family|genus|species",
classification_path_ids: "131567|2759|33154|33208|6072|33213|33511|7711|89593|7742|7776|117570|117571|7898|186623|41665|32443|1489341|186625|1489388|123365|123366|123367|123368|123369|1489872|1489885|1489894|30864|75033|94948",
taxon_id: "94948",
edit_distance: 0,
match_type: 1,

We do need this information, but it will come from new scoring system, so it is a separate ticket.

prescore: "3|0|0",
score: 0.988

Remove names from `name_strings` that has no corresponding `name_string_index`

Migrate data from GNI DB to newly created DB

Figure out how Aerospike works internally

Prepare API for second level referencing between data

Input:

list of local_ids OR list of name strings in input data source
name of input data source (e.g.: irmng, open-tree-of-life)
name of target data source (e.g.: ncbi, open-tree-of-life)
name of transitive data source (optional, e.g.: open-tree)

Output:

list of local ids in output data source

name_strings with search_term "notaname" should return empty result

when we make a search like

http://gnresolver.globalnames.org/api/name_strings?search_term=%22sfsf%22

it returns virus names instead of an empty result

Migrate data from GNI DB to newly created DB

name_strings table
Relation to name_strings_indicies and data_sources

Implement rest of modifiers in API

As a user of API I want to use "per_page" and "page" parameters instead of "take and "drop"

Analogous with gni:

per_page (default is 30)

number of items per one page (numbers larger than 1000 will be decreased to 1000)

page (default is 1)

page number you want to see.

Search will return ‘next_page’ field if more data exist for the search

Figure out how to launch Stratio deep-spark

Compute canonized forms of existing names, compute UUIDs and put to DB

Implement API methods for first set of modifiers of faceted search

Support modifers:

wildcard
exact
can
ns

suggest to include fishbase_taxon_cache as name source

as discussed with @dimus @alexander-myltsev last wk, I've created a project https://github.com/jhpoelen/fishbase_taxon_cache that generates a fishbase taxon cache that contains all species accessible through sealifebase.org and fishbase.org . Please let me know if there's anything I can do to help include the full taxonomic paths of this resource into global names. I am hoping to start using this data source through your name resolver.

Develop load performance scripts

use JMeter

Design feature list to cover functionality of GN resolver and parser

estimate complexity

Grab second level UUIDs from Tree of Life data from http://files.opentreeoflife.org/ott/
prepare data seeds with flywaydb for environments (develop, test, production)

Implement name resolution by full name and canonical name

Note: space normalization to single space
Resolution by UUID
"resolver" project

globalnamesarchitecture / gnresolver Goto Github PK

gnresolver's People

Contributors

Watchers

gnresolver's Issues

Recommend Projects

Recommend Topics

Recommend Org