For MOD imports, we also want to include annotations that use one of the 'high throughput evidence used in manual assertion' evidence codes. These are:

ECO:0006056 | High throughput evidence used in manual assertion | HTP

ECO:0007001 | High throughput mutant phenotype evidence used in manual assertion | HMP

ECO:0007003 | High throughput genetic interaction evidence used in manual assertion | HGI

ECO:0007005 | High throughput direct assay evidence used in manual assertion | HDA

ECO:0007007 | High throughput expression pattern evidence used in manual assertion | HEP

Thx.

In this spreadsheet we have all relations used in annotation extensions, at least in WB and MGI files. For many of these, I can easily map to an RO or BFO term:

regulates_transport_of -> RO:0002011

And the rest are available in the GOREL ontology (produced by the GO pipeline?), with some having xrefs to RO. Unfortunately I only see two of our unmapped relations having RO xrefs (has_agent and regulates_level_of). We'll need to figure out how to translate the other GORELs into GO-CAM if the GOREL relations aren't available for viewing in Noctua.

For CC annotations, the translate method is currently using

Spec says this should be part_of.

GO terms used in exported ttl
use an identifier.org uri like http://identifiers.org/GO:0000302

they should always be using purls like
http://purl.obolibrary.org/obo/GO_0000302

The basic idea here is that when a relationship exists between terms in the ontology, e.g. MF part_of BP, then if there are manual annotations to both (and either no extensions or extensions are all the same), then those annotations could be connected during the import.

We need some illustrative examples here, so holding off on adding this to the to-do list just yet.

Add ECO code filtering option to library like I did here.

Create triples in model for only manual, non-PAINT annotations (experimental evidence codes, plus ISS) for now. Maybe turn the filter into an argument.

@thomaspd @vanaukenk Should the high-throughput codes (HTP, HDA, HMP, HGI, HEP) also be included in these models?

In this model:

http://noctua-dev.berkeleybop.org/editor/graph/gomodel:a4631c87-4d56-4e4c-92cd-ede05da3483b

the inputs for the annotation:

mec-3 contributes_to -> RNA polymerase II regulatory region sequence-specific DNA binding (GO:0000977)

are folded into the mec-3 GP node, but should be folded into the MF node.

The reasoner turns red on this model.

Note also that there are three has_inputs, but the one input that doesn't render a gene name (just shows the URI) is actually a curation mistake and has been removed at the source.

The extensions spreadsheet I've been producing using gpad_extensions_mapper.py has been filtering the input GPAD lines using logic different from that used for filtering GPAD when translating into models. I should reuse the same FilterRule-derived classes in gpad_extensions_mapper.py.

Specifically, extensions generated by GO-CAM-exported GPAD are being included in the extensions spreadsheet.

The gene MGI:1859682 is annotated to GO:0008520 with an IMP evidence code. The original annotation has an allele value in the with field (MGI:2180333). The 'with' field is missing from the model on noctua-dev.

Original annotation in GPAD:
MGI MGI:1859682 enables GO:0008520 MGI:MGI:2178849|PMID:11984597 ECO:0000315 MGI:MGI:2180333 20030212 MGI

For importing the WB annotations, we will need to first filter the gpad file using the 'WB' value in the Assigned_by field.

In the current import on noctua-dev, it looks like we also have manual annotations from other groups, for example, Assigned_by 'UniProt'.

Finish up pulling in these fields from GPAD:

• Contributor (will be in annotation properties eventually)
• Assigned by (e.g. WB, MGI, UniProt)
• with/from (model definition required for this first)
• Date

Contributor should be orcID. In some lines (like protein2go) this is in the curator_uri property in annotation properties column. If curator_uri not present, xref name against users.yaml to try getting orcID. If still can't find orcID, leave blank, one will need to be created.

Currently this method just appends input to http://identifiers.org/. A JSONLD (like go_context.jsonld) should be plugged in somewhere and used for when CURIEs are declared.

E.g.

>>> model = GoCamModel("test", context_jsonld="go_context.jsonld")
>>> model.declare_class("PomBase:SPAC19D5.01")
"http://identifiers.org/pombase/SPAC19D5.01"

Hopefully this will help prevent identifier inconsistencies suppressing GP labels.

For each statement that gets generated from a line of GPAD, add an OWL annotation containing a String capturing the full GPAD line. Something like:
rdfs:comment "GPAD_ORIGIN: WB WBGene00000816 part_of GO:0005634 GO_REF:0000037 ECO:0000501 UniProtKB-KW:KW-0539 20180707 UniProt "

This can be attached to the evidence block. It will be useful for debugging and can be removed once the importer is complete (or left as part of the provenance trail).

Check for consistency.

Maybe also check with GO Rules ?

Gene products annotated to BP terms with the 'acts upstream of' or 'acts upstream of or within' relation should be imported as, for example:

GP <- enabled_by [root MF] -> causally_upstream_of BP

The equivalent model would be used for 'acts upstream of or within' except that the relation between the root MF and the BP would be 'causally upstream of or within'.

See http://noctua-dev.berkeleybop.org/editor/graph/gomodel:4187773d-e2b6-4235-895c-40436435d60d

for how they are currently imported.

We have this in the Google doc but just to note an example here for testing purposes:

The inputs on the mec-3 contributes_to RNA polymerase II regulatory region, sequence-specific DNA binding are pipe-separated and should be split out into separate annotations.

To test that all of the information content in an input GPAD file is indeed captured in the set of GO-CAMs generated from that file, make a module to compare the input GPAD with the GPAD that minerva generates from the GO-CAM.

Close by providing a line by line report of whether each input gpad is recapitulated, deepened, or lost.
Provide model-level report about whether model is (OWL) consistent or not and a count of the input gpad lines in the categories above. e.g. model for Gene A, is consistent, has 4 recapitulations, 2 deepened, 2 lost, and 1 new.

Looks like a heterogeneous doubling-up of prefix prepending for GO class(es?)

<http://identifiers.org/http://purl.obolibrary.org/obo/GO_0003674> a owl:Class .

At least for this MF root term in WB_WBGene00004055.ttl

From evidence rules 2 and 3 in the Google doc, we said that if there was more than one reference separated by a pipe, we would import only one reference using this descending list of preferences:

Order of preference for references:
PMID
GO_REF
doi
Other resource, e.g. MOD paper ID

Do we still want to do this?

Right now, it looks like both IDs (i.e. PMID and WBPaper ID) are being imported from the WB annotations, but only the WBPaper ID is displayed in the graph editor and both are exported in the Annotation Preview.

I can't quite make out how the paper ID is selected for the corresponding form view; both seem to be used but I can't discern a pattern.

Doing some QC on the imports, I realized that WB (and others) have annotations to other children of 'binding' (GO:0005488) that use the IPI (ECO:0000353) evidence code and contain an entry in the With/From field.

To be consistent in how we model 'binding' annotations in GO-CAM, we'll need to broaden our conversion rule so that any annotation to GO:0005488 that has an entry in the With/From field, will convert the With/From value to a 'has input' annotation extension, just like we're doing for 'protein binding' (GO:0005515) and its children.

For an example, see the 'miRNA binding' (GO:0035198) annotation in this alg-1 model.

@dustine32 - let me know if you need any more info or examples. Thx.

Work in progress taking place in https://docs.google.com/document/d/1AgkJqA4-V0HG0SaDiNO9ENTxaQC_1HwsmwyhyHhuxog/edit#

As part of the input code, we want to change annotation extensions that use 'has direct input' to 'has input'.

Loaded models generated from gocamgen somehow break the GPAD export function in minerva that drives the export in Noctua.

Last time GPAD export broke for my generated models it was with the SynGO models. The culprit there ended up being that I was using certain property that wasn't supported by minerva. I'll look up what the property was and see if it's in the gocamgen models.

Right now its just my ORCID and the GO group URL hard-coded in. Should be able to specify these at many levels, e.g. model, axiom, individual, evidence.

We had initially decided to infer this GO-CAM structure from an annotation's aspect:

1. A molecular function annotation is just GP <-enabled_by- MF.
2. A cellular component annotation is just GP -part_of-> CC.
3. A biological process annotation is just GP <-enabled_by- (GO:0003674) -part_of-> BP. In the latter case, both the enabled_by and part_of edge have the same evidence from the original BP annotation.

Since we're now using GPAD, where the relation is explicitly stated, we can just create triples using this instead (after converting to RO of course). So, for this example line (where involved_in translates to RO:0002331):

WB	WBGene00000001	involved_in	GO:0008286	PMID:12393910|WB_REF:WBPaper00005614	ECO:0000316	WB:WBGene00000898		20151214	WB

Triple will be WB:WBGene00000001-RO:0002331->GO:0008286.

So far we've been "collapsing/consolidating," based on certain criteria (e.g. GP + term + extensions are same), multiple GPAD lines into distinct assertion individuals containing multiple evidence. I'd like to get clarification and document this criteria here first, then we can move it to the wiki page.
In my head, this is basically a header vs line situation so I'll present it like so:

Header:

1. GP ID
2. qualifiers aka relations
3. primary term
4. With/From (if primary term is GO:0005515 or descendant)
5. Annotation Extensions

Line:

1. Reference
2. Evidence Code
3. With/From (if primary term is not GO:0005515 or descendant)
4. Source line
5. Date
6. Assigned by
7. Annotation Properties

@vanaukenk @ukemi @thomaspd This mainly came about recently when trying to figure out how to group lines by the with/from field, hence the question mark. Do multiple GPAD lines that differ only in with/from values represent the same assertion individual in GO-CAM or multiple?

After much discussion amongst @pgaudet @thomaspd @ukemi @vanaukenk , we've agreed on a representation for modeling 'regulates o occurs in' when the primary GO annotation is to a [regulation of BP] term.

The representative example is the top line of this model:
http://noctua.geneontology.org/editor/graph/gomodel:5c4605cc00000396

Also shown here:

For a conventional annotation of:
GP acts_upstream_of_or_within [regulation of BP] regulates_o_occurs_in CC

The rule could be written as:
GP <- enabled_by [root MF] -> causally_upstream_of_or_within [regulation of BP] -> regulates [BP] <- occurs_in [CC]

If the original regulation annotation was to a positive or negative regulation term, then we could substitute 'positively regulates' or 'negatively regulates' for 'regulates' in the above model.

See also: geneontology/go-annotation#2626

Currently for MGI, we're filtering out all ISO annotation lines from GPAD. We now want to allow ISO lines to be translated in model if that ISO line has Assigned_by=MGI.

This shouldn't affect other MOD filters. Yet.

From David's email:

The ones to be filtered would be ones with the references:

GO_REF:0000096, MGI: 4834177
GO_REF:0000096; MGI: 4417868

Add step at end of model generation to validate model against the GO-CAM spec here in ShEx format:
https://raw.githubusercontent.com/geneontology/go-shapes/master/shapes/go-cam-shapes.shex

Would be cool if I could integrate some python ShEx parser like PyShEx to look directly at RdfLib graph object in memory. This way I can immediately respond to errors before writing out file. PyShEx has a CLI we can fallback on (checking the written out .ttl file) if using as a library doesn't work out.

First step of this ticket is to get some sort of feedback on models coming out. Then we'll look at some common errors/issues.

As suggested by @goodb .

Will change model ID generation code, which currently grabs a random UUID, to instead simply use the model gene's identifier (e.g. WB:WBGene00003167, MGI:MGI:2159711).

Example model ID: gomodel:WB:WBGene00003167

That extra colon might wreak some havoc so the actual model ID pattern could change. This will help with keeping static URLs to test models after multiple iterations of generating models.

Collect examples of WB and MGI genes and their annotations for testing the import and GPAD generation pipelines. Annotations are in spreadsheets in the January Berkeley meeting folder: https://drive.google.com/drive/folders/1aMocjkYOUSd8rxWXCjGOUwYIu6Qnc-y7

Write a GAF-to-GPAD conversion step (just use ontobio's parser/writer) for whenever GPAD isn't available for a MOD and we have to use GAF.

Perhaps always write out the converted GPAD before subsequent conversion to GO-CAM.

For an annotation of Gene A to 'regulation of X', the models need to be decomposed into a regulatory process and the process that is being regulated. So if a gene product is annotated to 'regulation of leukocyte differentiation' the annoton would look like:
MF enabled_by A, MF-part_of;causally upstream of etc-generic BP, generic BP-regulates-X.

For annotation extensions, when the relation is a property chain like regulates_o_relation-T then the annotation extension should be added to the regulated process X. So the annotation extension would be X-relation-T.

WB:WBGene00006498 (ten-1) should have 17 Biological Process annotations associated with it, but right now, I only see the two CC annotations.

http://noctua-dev.berkeleybop.org/editor/graph/gomodel:e2ebe743-1704-4716-9a86-a9aa6ca7306f

Some examples of missing annotation lines from the snapshot GPAD:

WB WBGene00006498 involved_in GO:0007276 PMID:15936327|WB_REF:WBPaper00025239 ECO:0000315 20090108 WB

WB WBGene00006498 involved_in GO:0008406 PMID:18632986|WB_REF:WBPaper00032026 ECO:0000315 WB:WBVar00091925 20090108 WB

WB WBGene00006498 involved_in GO:0009792 PMID:18632986|WB_REF:WBPaper00032026 ECO:0000316 WB:WBGene00000961 20090108 WB

WB WBGene00006498 involved_in GO:0009792 PMID:18632986|WB_REF:WBPaper00032026 ECO:0000316 WB:WBGene00001328 20090108 WB

WB WBGene00006498 involved_in GO:0009792 PMID:18632986|WB_REF:WBPaper00032026 ECO:0000316 WB:WBGene00002081 20090108 WB

WB WBGene00006498 involved_in GO:0009792 PMID:18632986|WB_REF:WBPaper00032026 ECO:0000316 WB:WBGene00003738 20090108 WB

WB WBGene00006498 involved_in GO:0009792 PMID:22938866|WB_REF:WBPaper00041503 ECO:0000316 WB:WBGene00002251 20121022 WB

WB WBGene00006498 involved_in GO:0009792 PMID:22938866|WB_REF:WBPaper00041503 ECO:0000315 WB:WBVar00091925 20121022 WB

WB WBGene00006498 involved_in GO:0010172 PMID:21795395|WB_REF:WBPaper00039988 ECO:0000316 WB:WBGene00001077 20111103 WB

Translate whatever has_regulation_target annotation extensions satisfy one of the scenarios described in the google doc.

@vanaukenk @ukemi @thomaspd A question about when the use of has_regulation_target is valid: On the extensions2GO-CAM wiki, has_regulation_target is stated as only valid when primary term is an MF. This conforms with my current rule-checking code. Meanwhile, the google doc above lists strategies for how to handle these when the primary term is one of the listed BP terms or its descendants e.g.

3.a. GP-A [regulation of molecular function Z] has_regulation_target GP-B

In searching the GPADs for examples I see has_regulation_target used on annotations to both MF and BP terms:

MGI     MGI:107771      enables GO:0005096      MGI:MGI:3698699|PMID:17116687   ECO:0000314                     20101101        MGI     has_regulation_target(MGI:MGI:97846)|has_regulation_target(MGI:MGI:2180784)
WB	WBGene00000222	involved_in	GO:0006990	PMID:16184190|WB_REF:WBPaper00026830	ECO:0000315		20131015	WB	has_regulation_target(WB:WBGene00002783)

Can someone set me straight on this?

The method input would be an ontobio AssociationSet or association objects. This would update the model object, which could then be written out later.

May need to port over annotation logic from go-pombase.

For a cellular component annotation, the model should just indicate GP<>part_of<>CC, not GP<>enables<>MF<>coccurs_in<>CC.

See model

gomodel:785e6d6b-fbda-4b41-9268-6493c8ee37a1

This ticket is to keep track of the efforts to refactor @dustine32 gpad-to-ttl code to use a reusable gocam data model.

The gocam data model specifications is currently under major discussions for further refinements. Once this is done and the python implementation of that model is completed, we can move forward into integrate it in this gpad-to-ttl code.

Deliverables:

• gocam data model used in this code
• gpad input handler to create gocam
• ttl output handler from any gocam

For genes annotated to root node for all aspects, BP, MF, and CC:

Create a single 'annoton' of:

GP-A<-enabled_by-[root MF]-part_of->[root BP]

and [root MF] -> occurs_in [root CC]

This can happen when there are multiple evidence for the same GP-to-term.

Will need to find existing triple, if exists, matched by GP and term classes (and relation, once we get that fancy) and add the evidence from each identical annotation on that same triple.

A WB example to work with is WB:WBGene00000903 GO:0040024, which has 8 lines in the current wb.gpad file.

When annotation extensions are comma delimited and are from GO_CC, CL and an anatomy ontology then the model should indicate GO_CC<>part_of<>CL<>part_of<>anatomy.

MGI | MGI:1336882 | acts_upstream_of_or_within | GO:0070625 | MGI:MGI:3801544|PMID:18535671 | ECO:0000315 | MGI:MGI:3056083 | 20160406 | MGI | occurs_in(EMAPA:35651),occurs_in(CL:0002064),occurs_in(GO:1990794)

See the bottom of model:
http://noctua.geneontology.org/editor/graph/gomodel:5c4605cc00000315

This can be done relatively easily by calling uri() on the ontobio writer (TurtleRdfWriter). Recent ontobio update points to this new JSONLD. There may still be a few rdflib.term.URIRef() calls that explicitly set prefix (possibly incorrectly).

For context:
geneontology/go-site#617
biolink/ontobio#167

The code I have already written for AssocGoCamModel looks like it should take a list of ontobio associations and generate a single GO-CAM model, though I seem to have left AssocGoCamModel.translate() in an unfinished state:

>>> model.translate()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/ebertdu/go/gocamgen/gocamgen/gocamgen.py", line 257, in translate
    aspect_triples.append(self.emit(term, ENABLED_BY, annoton.enabled_by))
AttributeError: 'AssocGoCamModel' object has no attribute 'emit'

Fixing this method and then model.write() should get us a model we can load to Noctua for viewing.

For @ukemi from our 2019-08-29 call. Here's what I have down:

1. Remove has_input cardinality constraint for MF primary terms
2. Allow negatively_regulates(MF) for binding primary terms
3. Add MF to ACCEPTABLE_NAMESPACES for part_of
4. Allow regulates for MF and BP primary terms

I'll adjust formatted_ext_patterns.tsv accordingly in a new branch, regenerate a new bad_extensions.tsv report, and I'll request PR approval from @ukemi for when he thinks the changes are good.

I spotted this in a MGI model for two annotations to the same ID and term:

MGI:MGI:87859 to GO:0004713

I'm suspecting it's due to emitting evidence URI, dates, comments triples to an axiom for each new evidence rather than appending the evidence URI to the LEGO:evidence property of the single axiom. May the solution is similar to this?

[ rdf:type owl:Axiom ;
   owl:annotatedSource <http://model.geneontology.org/66ab4fd6-68b2-4e3c-aa19-88199eca825e/86ed125d-a654-426d-aaa6-48a30f750a95> ;
   owl:annotatedProperty <http://purl.obolibrary.org/obo/RO_0002333> ;
   owl:annotatedTarget <http://model.geneontology.org/66ab4fd6-68b2-4e3c-aa19-88199eca825e/e46feaf1-2984-43d0-a3ed-d292cead6ef0> ;
   <http://geneontology.org/lego/evidence> <http://model.geneontology.org/66ab4fd6-68b2-4e3c-aa19-88199eca825e/61d635f0-2397-4c0c-908a-740637d14b9d> , <http://model.geneontology.org/66ab4fd6-68b2-4e3c-aa19-88199eca825e/949d06a6-3355-4614-b8a3-2dbdef877c1f> ;
   <http://purl.org/dc/elements/1.1/contributor> "http://orcid.org/0000-0002-6659-0416" ;
   <http://purl.org/dc/elements/1.1/date> "2014-07-14" ;
   rdfs:comment "MGI MGI:87859 enables GO:0004713 MGI:MGI:2384444|PMID:12167702 ECO:0000315   20140714 MGI has_direct_input(MGI:MGI:104741)"
 ] .

Specifically the line:

<http://geneontology.org/lego/evidence> <http://model.geneontology.org/66ab4fd6-68b2-4e3c-aa19-88199eca825e/61d635f0-2397-4c0c-908a-740637d14b9d> , <http://model.geneontology.org/66ab4fd6-68b2-4e3c-aa19-88199eca825e/949d06a6-3355-4614-b8a3-2dbdef877c1f> ;

I'm creating some new annotations in Noctua that are just part of gene-by-gene curation, i.e. not part of a larger GO-CAM model.

I've named this model according to our import conventions, i.e. WB:WBGene00003480.

When we actually import the annotations into Noctua, it would be nice if we could merge other annotations to WBGene00003480 into that existing model.

Would that be possible?

Thx.

As part of the pipeline, we need to output a report that lists the lines in the GPAD file that do NOT get converted according to the current iteration of the rules so curators can check the annotation to see if it's correct (and the rule revised) or should be fixed.

Figure out where these NOT qualifiers go in the model. I'm guessing an axiom is involved. An example GPAD line from wb.gpad:

WB	WBGene00000038	NOT|enables	GO:0003990	PMID:10891266|WB_REF:WBPaper00004251	ECO:0000320			20170131	WB