Comments (6)
LucaCappelletti94
commented on June 12, 2024
1
Hello @callahantiff, sorry for the delay! I believe something like that may work, but if it takes more than an hour (and in my experience configuring stuff that is not completely straightforward in Google Cloud tend to combust at some point spontaneously) possibly, we could hard-code some reference URL(s) for the latest version of Pheknowlator into a metadata JSON within Ensmallen that can be updated as necessary.
Would this be a (low-time requirement) sensible solution? Indeed it does not allow automatic retrieval of newly deployed graphs unless they replace the old one at the previous URLs.
from pheknowlator.
LucaCappelletti94
commented on June 12, 2024
1
One such example is how I am currently handling the kg-hub graphs since they still have some oddities.
from pheknowlator.
callahantiff
commented on June 12, 2024
1
Hey @LucaCappelletti94 -
Thanks for circling back with me. We would love to be included and we do provide data in a format other than OWL. Information on all of the output files produced, including a snippet of the output, can be found here: https://github.com/callahantiff/PheKnowLator/wiki/KG-Construction#table-knowledge-graph-build-output. Per our conversation on Slack, I think the files for each build that will work the best and be the easiest to use with existing infrastructure will be the XXXX_Triples_Identifiers.txt.
There are two other updates that I wanted to give you below.
Update One: PheKnowLator GCS Bucket Directory Listing Solution
There is a JSON file called pheknowlator_builds.json that can serve as a proxy for a directory listing. It gets updated each month and can be accessed from the following URL: https://storage.googleapis.com/pheknowlator/pheknowlator_builds.json.
Currently, there is an entry for metadata, and then one for each monthly build/release (ordered temporally), where each kg build is referenced by key within each monthly build (hope that makes sense -- additional information on how the 12 builds differ is shown below). Note that if a particular file was not available for a monthly build/release, it will be noted with the value null. A snippet of the output is shown below. Let me know if you see any problems with this structure, it's very easy to make changes!
{
"metadata": "For more information on the PheKnowLator Builds, please visit the project GitHub: https://github.com/callahantiff/PheKnowLator",
"v2.0.0-2020-5-10": {
"instance-inverseRelations-owl": "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_10MAY2020/knowledge_graphs/instance_builds/inverse_relations/owlnets/PheKnowLator_v2.0.0_full_Instance_inverseRelations_noOWL_Triples_Identifiers.txt",
"instance-inverseRelations-owlnets": "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_10MAY2020/knowledge_graphs/instance_builds/inverse_relations/owlnets/PheKnowLator_v2.0.0_full_Instance_inverseRelations_noOWL_Triples_Identifiers.txt",
"instance-inverseRelations-owlnets-purified": null,
"instance-relationsOnly-owl": "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_10MAY2020/knowledge_graphs/instance_builds/relations_only/owlnets/PheKnowLator_v2.0.0_full_Instance_relationsOnly_noOWL_Triples_Identifiers.txt",
"instance-relationsOnly-owlnets": "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_10MAY2020/knowledge_graphs/instance_builds/relations_only/owlnets/PheKnowLator_v2.0.0_full_Instance_relationsOnly_noOWL_Triples_Identifiers.txt",
"instance-relationsOnly-owlnets-purified": null,
"subclass-inverseRelations-owl": "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_10MAY2020/knowledge_graphs/subclass_builds/inverse_relations/owlnets/PheKnowLator_v2.0.0_full_subclass_inverseRelations_noOWL_Triples_Identifiers.txt",
"subclass-inverseRelations-owlnets": "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_10MAY2020/knowledge_graphs/subclass_builds/inverse_relations/owlnets/PheKnowLator_v2.0.0_full_subclass_inverseRelations_noOWL_Triples_Identifiers.txt",
"subclass-inverseRelations-owlnets-purified": null,
"subclass-relationsOnly-owl": "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_10MAY2020/knowledge_graphs/subclass_builds/relations_only/owlnets/PheKnowLator_v2.0.0_full_subclass_relationsOnly_noOWL_Triples_Identifiers.txt",
"subclass-relationsOnly-owlnets": "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_10MAY2020/knowledge_graphs/subclass_builds/relations_only/owlnets/PheKnowLator_v2.0.0_full_subclass_relationsOnly_noOWL_Triples_Identifiers.txt",
"subclass-relationsOnly-owlnets-purified": null
},
"v2.0.0-2021-1-25": {
"instance-inverseRelations-owl": "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_25JAN2021/knowledge_graphs/instance_builds/inverse_relations/owlnets/PheKnowLator_v2.0.0_full_instance_inverseRelations_noOWL_Triples_Identifiers.txt",
"instance-inverseRelations-owlnets": "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_25JAN2021/knowledge_graphs/instance_builds/inverse_relations/owlnets/PheKnowLator_v2.0.0_full_instance_inverseRelations_noOWL_Triples_Identifiers.txt",
"instance-inverseRelations-owlnets-purified": "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_25JAN2021/knowledge_graphs/instance_builds/inverse_relations/owlnets/PheKnowLator_v2.0.0_full_instance_inverseRelations_noOWL_INSTANCE_purified_Triples_Identifiers.txt",
"instance-relationsOnly-owl": "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_25JAN2021/knowledge_graphs/instance_builds/relations_only/owlnets/PheKnowLator_v2.0.0_full_instance_relationsOnly_noOWL_Triples_Identifiers.txt",
"instance-relationsOnly-owlnets": "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_25JAN2021/knowledge_graphs/instance_builds/relations_only/owlnets/PheKnowLator_v2.0.0_full_instance_relationsOnly_noOWL_Triples_Identifiers.txt",
"instance-relationsOnly-owlnets-purified": "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_25JAN2021/knowledge_graphs/instance_builds/relations_only/owlnets/PheKnowLator_v2.0.0_full_instance_relationsOnly_noOWL_INSTANCE_purified_Triples_Identifiers.txt",
"subclass-inverseRelations-owl": "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_25JAN2021/knowledge_graphs/subclass_builds/inverse_relations/owlnets/PheKnowLator_v2.0.0_full_subclass_inverseRelations_noOWL_Triples_Identifiers.txt",
"subclass-inverseRelations-owlnets": "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_25JAN2021/knowledge_graphs/subclass_builds/inverse_relations/owlnets/PheKnowLator_v2.0.0_full_subclass_inverseRelations_noOWL_Triples_Identifiers.txt",
"subclass-inverseRelations-owlnets-purified": "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_25JAN2021/knowledge_graphs/subclass_builds/inverse_relations/owlnets/PheKnowLator_v2.0.0_full_subclass_inverseRelations_noOWL_SUBCLASS_purified_Triples_Identifiers.txt",
"subclass-relationsOnly-owl": "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_25JAN2021/knowledge_graphs/subclass_builds/relations_only/owlnets/PheKnowLator_v2.0.0_full_subclass_relationsOnly_noOWL_Triples_Identifiers.txt",
"subclass-relationsOnly-owlnets": "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_25JAN2021/knowledge_graphs/subclass_builds/relations_only/owlnets/PheKnowLator_v2.0.0_full_subclass_relationsOnly_noOWL_Triples_Identifiers.txt",
"subclass-relationsOnly-owlnets-purified": "https://storage.googleapis.com/pheknowlator/archived_builds/release_v2.0.0/build_25JAN2021/knowledge_graphs/subclass_builds/relations_only/owlnets/PheKnowLator_v2.0.0_full_subclass_relationsOnly_noOWL_SUBCLASS_purified_Triples_Identifiers.txt"
},
... }Update Two: KG Build Types
- As shown in the file above, there are 12 KGs produced each month. The builds differ based on three parameters, each of which are explained below:
- Construction Approach: New data can be added to the core set of ontologies that we use as a base in two ways:
subclass-based(examples here) andinstance-based(examples here). An example of the difference is shown in the image below.
- Relation Strategy: relations can be modeled using a standard approach where a single relation is added between two nodes (i.e.,
relations_only) or by inferring the inverse of the relation, which we accomplish using two strategies (i.e.,inverse_relations) -- see example below. For more details and examples, see here.
- Property Graph Abstraction: We create an abstraction or transformed version of the full OWL graph that decodes the nodes and edges that are needed to support the expressivity of OWL, but which are not biologically meaningful. The standard application of this method results in files that include
_OWLNETS_in the file name and the original full OWL file with be noted withOWLin the filename (for additional information see here).- There is an additional type of output that is produced that takes the OWL-NETS abstraction and makes sure it is completely consistent with the construction approach parameter. The consistent or "purified" output will include
_SUBCLASS_purified_or_INSTANCE_purified_in the file names. For example, if the build isinstance-based, then allrdfs:subClassOfrelations are converted tordf:typeand for all triples where anrdfs:subClassOfrelation occurred we addrdf:typerelations between the object of this triple and all of its ancestors. For asubclass-based build, we implement the same procedure but replace all occurrences ofrdf:typewithrdfs:subClassOf.
- There is an additional type of output that is produced that takes the OWL-NETS abstraction and makes sure it is completely consistent with the construction approach parameter. The consistent or "purified" output will include
- Construction Approach: New data can be added to the core set of ontologies that we use as a base in two ways:
By varying the different combinations of the Construction Approach, Relation Strategy, and Property Graph Abstraction you end up with the following 12 KGs:
- subclass + relations only + owl
- subclass + relations only + owlnets
- subclass + relations only + owlnets - purified
- subclass + inverse relations + owl
- subclass + inverse relations + owlnets
- subclass + inverse relations + owlnets - purified
- instance + relations only + owl
- instance + relations only + owlnets
- instance + relations only + owlnets - purified
- instance + inverse relations + owl
- instance + inverse relations + owlnets
- instance + inverse relations + owlnets - purified
from pheknowlator.
callahantiff
commented on June 12, 2024
1
Awesome @LucaCappelletti94 thanks for your help with this. I will close this issue, but please feel free to re-open if we need to do more work here.
from pheknowlator.
callahantiff
commented on June 12, 2024
Thanks so much @LucaCappelletti94. This sounds totally reasonable. I will take a closer look this weekend and follow-up once I have a better sense. Thank you!
from pheknowlator.
LucaCappelletti94
commented on June 12, 2024
Hello @callahantiff, I am doing a round of updates to the graph retrieval, any news on the PheKnowLator graph availability? I saw it is offered in owl, but we do not currently support the OWL format.
from pheknowlator.
Related Issues (20)
- July 2021 Build -- Delay
- Incorporating Ensmallen -- 🔮 Magic ✨ 🎩
- Script Notebook: OWLNETS_Example_Application.ipynb HOT 1
- New Resource Request: LINCS
- Adding Edge Pattern and Relation Types HOT 3
- requirements.txt HOT 7
- Newline char in node description/definition causes dirty lines in node metadata files HOT 5
- Foreign characters in node labels HOT 18
- Simplify input files -- input yaml HOT 5
- GitHub Actions -- Docker build-push-action failing HOT 3
- Migrate from Google Cloud Container Registry to Artifact Registry
- Create Notebook to Enable pkt use in Efforts like HuBMAP HOT 1
- Create Build Dashboard
- Wiki: Create Endpoint Page
- CI/CD Pipeline: Ensuring Builds Use Most Current Data HOT 3
- Pheknowlator knowledge graph in OBO format HOT 4
- log directory is not created when pkt_kg is imported HOT 1
- Consider switching to https://identifiers.org/ URI or CURIEs HOT 2
- Create Edge-Type Attribute File HOT 15
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pheknowlator.