biothings / discovery-app Goto Github PK
View Code? Open in Web Editor NEWThe Data Discovery Engine project by the CD2H Data working-group
Home Page: http://discovery.biothings.io
License: Apache License 2.0
The Data Discovery Engine project by the CD2H Data working-group
Home Page: http://discovery.biothings.io
License: Apache License 2.0
For people who have already registered their dataset in a repository (like figshare, dataverse, NCBI GEO, etc.), would be nice to pre-populate our form with matching fields. Note that in some cases (e.g., figshare, dataverse) they will already expose structured metadata via schema.org standards. In other cases (e.g., NCBI GEO), we'd need to provide the mapping leveraging the work we did with metadataplus.
Suggested NIAID/DDWG 2020-10-20
When I try to edit a dataset's metadata, for instance to fix author.affiliation
to make it an object, I get a validation error that creator
is a required field. According to the schema, creator
isn't required: author
is. Maybe pulling the incorrect schema for validation?
error message:
'creator' is a required property
Hint: name,description,creator,publisher,identifier
Something like "create a new dataset record like this". Not sure whether to display that on the existing dataset details page, or have a dataset selector on the dataset registration page, or something else... This can be accomplished by downloading and importing the JSON, so this would be an easier way of accomplishing that...
https://discovery.biothings.io/view/niaid/NiaidDataset
Initial state of the toggles isn't synced. Clicking "show/hide all properties on page" doesn't work unless you uncheck the β "show all inherited properties"
It appears that of the 39 datasets currently at https://discovery.biothings.io/dataset, Google Dataset Search is only indexing the Wellderly dataset: https://datasetsearch.research.google.com/search?query=site%3Adiscovery.biothings.io
Not sure if it's the only reason, but rich results test on https://discovery.biothings.io/dataset/da4905854c18028d gives a "Page partially loaded" error...
e.g. for https://discovery.biothings.io/dataset/dcaab754073ad77b, display that it came from https://discovery.biothings.io/guide/niaid/
Based off of feedback from the 2020-10-20 NIAID DDWG working group demo, it seems like it'd be helpful to outline all the ways the /guide can be used. A few use cases:
identifier
and make minor adjustments to other fields: see #34Using the DDE playground to create a new schema seems to work fine when 'extending' it from a schema.org class. The schema generated in the preview can be saved to github, loaded into the schema playground and correctly visualized. Example schema.
This is not the case when you try to extend from a (non-schema.org) schema registered in the DDE playground such as the outbreak schema or even the BioMedicalDataset schema. Example 1: bte:BioMedicalDataset schema-derived class. Example 2: outbreak:Dataset schema-derived class.
The new schemas created from the registered class will give the following error when you try to load them. "Failed because:
field about in $validation is not correctly documented"
on a dataset details page (e.g. https://discovery.biothings.io/dataset/da4905854c18028d) display who registered dataset. Even better to have a link to see all the other datasets that person registered...
for class page like this one: http://discovery.biothings.io/schema/Dataset
use case would be: to quickly check a known property using browser "find", which requires all properties displayed on the page.
Here is an example:
This is the original schema file:
https://discovery.biothings.io/api/registry/n3c/n3c:Dataset
This is the registered version accessed from the API:
https://discovery.biothings.io/api/registry/n3c/n3c:Dataset
extra prefixes like outbreak
and niaid
were added, which are not related to the n3c schema
When you search for a term against an ontology, it'd be nice to transfer the input search term to the "Didn't find what you are looking for? Enter value here:" input box.
Steps to reproduce:
After I enter all metadata using the NIAID guide, I get to this screen:
Clicking the "Register" link produces this modal:
Clicking "Register Now" seems to just make the modal disappear -- was expecting it to report back "Success" or "Failure" on a new page.
It appears that the dataset was not successfully registered since I do not see it at https://discovery.biothings.io/dataset
Suggestion from 2020-10-20 DDWG working group meeting.
When one enters a term that has a lot of hits, it's really hard to find what you're looking for in the long list of matching terms. For instance: if you enter "influenza a" for "infectiousAgent" in the NIAID dataset schema, the terms are cut off (since they're long):
Suggest:
Right now, it's difficult to drag and drop from the validation options to properties further down the list.
Either enable scrolling of the list itself (so user can scroll to property) and then do the drag and drop, or allow the drag/drop menu to float so that when user scrolls down the screen, the drag/drop menu doesn't move out of view.
It's possible that users might try to register a dataset that already exists in the DDE, or they might want to work from a template of a similar dataset (see #34 ).
Similar to how Stack Overflow suggests you look at previously asked/answered questions rather than duplicating a question, it'd be cool as you enter data for it to suggest that the dataset might already be registered.
Suggested ways of detecting similarity:
Error message: "field analysisCode in $validation is not correctly documented"
Steps to reproduce:
Try to visualize https://raw.githubusercontent.com/SuLab/niaid-data-portal/master/schema/Demo-Extended-Dataset.jsonld on https://discovery.biothings.io/schema-playground
Steps to create .json schema
analysisCode
of type schema:SoftwareSourceCode
There are currently 39 datasets in the dataset registry (https://discovery.biothings.io/dataset). When I search "snohomish", I expect to get only one data set back (https://discovery.biothings.io/dataset/220edb57fb41eeea). The discovery app correctly highlights that dataset, but the API is currently returning all 39 (https://discovery.biothings.io/api/dataset?q=snohomish)
Steps to reproduce:
... --> no preview
The use case is some field in the schema is not intended for user-input.
If an array can be entered into the guide with a comma-separated list of values, provide helper text to user about this capability
but can be returned with meta=1/true
query param.
The "branding" on https://discovery.biothings.io/portal/cvisb is a little muddled because we mix up CViSB and NIAID. Currently low priority, but at some point we should make that clearer...
Add asterisks or similar to make explicit which properties are required for objects within a schema (e.g. funder.name
, identifier
are required; rest are optional for NIAID:funding
)
Specify the parent object in the field name somehow. For instance, NIAID has two descriptions: description
and funding.description
. When filling in the funding description, some users were confused about whether that was referring to the dataset description or the funding description
Based off of feedback from 2020-10-20 NIAID DDWG call.
e.g. for /guide/niaid, display https://discovery.biothings.io/dataset/21d8178fdf22998b, https://discovery.biothings.io/dataset/2aedff2a85b11474, ....
Bug: When registering a dataset, children seem to be inheriting their parent's marginality
Example: Why is url
a required property on creator
in NIAID? It should be optional according to the schema:
https://github.com/SuLab/niaid-data-portal/blob/master/schema/NIAIDDataset.json
Steps to reproduce:
See also: funding:description, funding:url, funding:parentOrganization, ...
@flaneuse to generate the list of repos that schema.org compliant.
@marcodarko to figure out how to display on https://discovery.biothings.io/guide/niaid/
π when extending a schema that has re-used a schema.org property, if you make one of those properties required, itβll duplicate it in the required object. This will throw an error when you try to visualize the schema. Need to de-duplicate the required
list.
Steps to replicate:
When creating a new property, the schema editor provides an easy-to-use interface, UNLESS, you happen to make a mistake at the input type step. Since there's no way to remove an erroneous input type, you have no choice but to submit the erroneous property, delete it, and start all over.
The temptation is to click in the box listing the input types and to hit backspace (in order to delete the erroneous input type), which doesn't delete it, but can send you back one page (in the browser) which can potentially cause the user to need to restart.
This issue seems having two behaviors:
If accessing https://discovery.biothings.io/view/ctsa_blah/ for the first time, the page shows empty schema list, which is correct, but need a more explicit msg saying the namespace does not exist.
If accessing https://discovery.biothings.io/view/ctsa_blah/ after I viewed another schema (e.g. by clicking one of example links), somehow, the first link shows the content from the previous schema. Looks like it picked up the cached data from the local storage.
When there's an API event (someone added a schema, deleted a dataset, ...),
discovery-app/discovery/web/api/schema.py
Line 179 in b598b7e
discovery-app/discovery/web/notify.py
Line 259 in b598b7e
discovery-app/discovery/web/notify.py
Line 23 in b598b7e
What we are interested in next, is to make this one way operation two way, and enabling things like:
Note we are not planning on supporting anything but the first use case now, but ideally the structure should support further extension.
In the NiaidDataset, funding.funder
, funding.identifier
, and funding.funder.name
should be required:
"monetaryGrant": {
"type": "object",
"@type": "MonetaryGrant",
"description": "Funding that supports (sponsors) the collection of this dataset through some kind of financial contribution",
"properties": {
"funder": {
"description": "An organization associated with a creator or funder of a dataset",
"oneOf": [
{
"$ref": "#/definitions/funder"
},
{
"type": "array",
"items": {
"$ref": "#/definitions/funder"
}
}
]
},
"description": {
"type": "string",
"description": "description about the funding award / grant"
},
"url": {
"type": "string",
"description": "award / grant URL"
},
"identifier": {
"type": "string",
"description": "Unique identifier(s) for the grant(s) used to fund the Dataset"
}
},
"required": [
"funder",
"identifier"
]
}
When registering a dataset (I think in the DataDistribution section), I see this dialog box:
To promote good data practices, suggest switching to YYYY-MM-DD
to follow ISO 8601 standard.
To reproduce:
https://discovery.biothings.io/guide/n3c/dataset
author
field:
author
Use case: a user has a bunch of datasets to register in https://discovery.biothings.io/guide/niaid/. Rather than submitting them one by one, it'd be cool to allow users to be able to register multiple simultaneously.
Ideally the inputs would be a .json array or a .csv/.tsv file.
Right now, I believe the form defaults to Person. I think it would be better if users have a choice between Person and Organization.
Would be nice in https://discovery.biothings.io/view/niaid/NiaidDataset (or other schemas) to have a link to download the schema as .jsonld file.
https://discovery.biothings.io/editor shows both variableMeasured (right) and variablesMeasured (wrong) when you try to extend schema:Dataset.
CreativeWork shows 89 inherited properties, but I count 103 on the current Dataset page (counting by hand so might be off by +/- 1).
Suggest pulling schema.org properties from their latest definition at https://schema.org/version/latest/schemaorg-current-http.jsonld (schema.org for developers)
It seems to give the array of strings instead.
In NIAID Dataset, the infectiousAgent
property is shown as a CreativeWork
with type string and Non-Strict term lookup. It should just be a (non-strict) enumerated string, not a CreativeWork object.
Currently, the "Discovery Guide" link in the header goes to https://discovery.biothings.io/best-practices. A few possible suggestions:
Is /best-practices
the best location for this page? "Best practices" and "Discovery Guide" seem like two different things...
At the top of that page, perhaps have a short text description of what a "Discovery Guide" is (not obvious from that phrase alone). Basically explain that it is a wizard that allows dataset registration according to a given schema.
Now that we have three guides, have links on this page to all three of them
A login link like this:
https://discovery.biothings.io/login?next=/guide/n3c/dataset
If go with the github login, the redirection follows the url passed to next
parameter, however, the login via UNA does not.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.