sul-dlss / cocina-models Goto Github PK

Blocked by #91
Blocking #92

Tracking kevindew/openapi3_parser#13

The geo extension schema needs to be added to openapi in order to complete sul-dlss/dor-services-app#1163.

@arcadiafalcone

10:02 $ exe/generator generate
Traceback (most recent call last):
	5: from exe/generator:9:in `<main>'
	4: from /Users/jcoyne85/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/thor-1.0.1/lib/thor/base.rb:485:in `start'
	3: from /Users/jcoyne85/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/thor-1.0.1/lib/thor.rb:392:in `dispatch'
	2: from /Users/jcoyne85/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/thor-1.0.1/lib/thor/invocation.rb:127:in `invoke_command'
	1: from /Users/jcoyne85/.rbenv/versions/2.7.1/lib/ruby/gems/2.7.0/gems/thor-1.0.1/lib/thor/command.rb:27:in `run'
/Users/jcoyne85/workspace/sul-dlss/cocina-models/lib/cocina/generator/generator.rb:20:in `generate': undefined method `each_key' for Openapi3Parser::Node::Map(#/components/schemas):Openapi3Parser::Node::Map (NoMethodError)

Due to rubocop suggested change.

In order to make sul-dlss/argo#1950 work.

Adding adminMetadata and relatedResource to accommodate existing MODS data.

Blocked by interagent/committee#286

Very few collections in production use sourceId presently (mostly hydrus items), however this is potentially very useful if we use an event driven system to do registration. This allows us to kick off registration asynchronously and then receive events with the sourceId and then find out the druid.

Per slack channel conversation, @andrewjbtw and @jcoyne want sdrPreserve for a file to default to "true".

I'm assuming here that they want shelve for a file to continue defaulting to "false"

related: sul-dlss/common-accessioning/issues/610

Adding to provide consistent way to refer to the source of the metadata (currently recorded in free-text note).

As documented in the README, cutting a new cocina model gem and upgrading all of the relevant applications is extremely time consuming. On the other hand, not cutting a new cocina model is risky because (1) it puts more changes into a single update; (2) it allows time to pass between a change to cocina model and fixes to dependent application; and (3) it makes it difficult/slow to push out a production fix.

Such automation would:

1. Cut a new cocina models gem.
2. Update dor-services client and cut a new gem.
3. Update sdr-client and cut a new gem.
4. Update sdr-api with new gems and openapi.
5. Update dor-services-app with new gems and openapi.
6. ✅ SCRIPTED! Update all other dependent apps with new gems.
   a. script to make PRs to update gems (see sul-dlss/access-update-scripts/pull/181)
   b. script to merge all these PRs (idea: have a way to run "merge_all.rb" to selectively merge these PRs. (see sul-dlss/access-update-scripts/pull/187)
7. ✅ SCRIPTED! Deploy all updated apps.
   script to deploy all these apps PLUS dor-services-app and sdr-api (idea: have a way to run sdr_deploy to selectively deploy these apps) sul-dlss/sdr-deploy#93

This PR added a string or integer value: https://github.com/sul-dlss/cocina-models/pull/144/files

Move DescriptiveStructuredValue from DescriptiveValue to DescriptiveBasicValue.

For parallel title use case.

(Assigned to me)

Otherwise extra properties are caught by dry-struct, not openapi validation.

Role metadata looks like this:

<roleMetadata>
  <role type="dor-apo-manager">
    <group>
      <identifier type="workgroup">sdr:developer</identifier>
    </group>
    <group>
      <identifier type="workgroup">sdr:service-manager</identifier>
    </group>
    <group>
      <identifier type="workgroup">sdr:metadata-staff</identifier>
    </group>
    <person>
      <identifier type="sunetid">kdurante</identifier>
    </person>
  </role>
</roleMetadata>

It is indexed here:
https://github.com/sul-dlss/dor_indexing_app/blob/master/app/indexers/role_metadata_datastream_indexer.rb#L20

and consumed here:
https://github.com/sul-dlss/argo/blob/master/app/helpers/registration_helper.rb#L9

So in this case:

"apo_register_permissions_ssim": [
  "workgroup:sdr:developer",
  "workgroup:sdr:service-manager",
  "workgroup:sdr:metadata-staff",
  "sunetid:kdurante"
]

See https://argo.stanford.edu/view/druid:xt299pt7593

See sul-dlss/argo#2200 for details

Which means this feature of dor-services-app is broken:

https://github.com/sul-dlss/dor-services-app/blob/9cb85b37e05104a99bd747861498ed3ff2376cf2/app/services/cocina/object_creator.rb#L94-L101

Caused by #121

Structural metadata determines how objects will display on the purl. The content type determines which viewer will be displayed, and the resource types plus the organization of files under resources shapes how the content will be displayed.

We should be able to define which resource types can or cannot be used with which content types. For example, a resource type of "page" will only work within the content type of "book". A "page" within an "image" content type will result in an error message in the sul-embed viewer.

If we can define the allowable resource types for each content type, we can validate resource type selections when submitted to SDR and prevent invalid settings from being accessioned. It may take some trial and error to work out what is and is not supported currently in the system. Our method of setting these definitions and validations should be flexible enough to accommodate changes to how the sul-embed viewer displays content.

See: https://github.com/sul-dlss-deprecated/sdr3-models/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc

1. confirm they are not used by dependents
2. rip 'em out

@jcoyne believes @justinlittman found a better way.

We need to remove the inference of resource_type from dor-services-app (https://github.com/sul-dlss/dor-services-app/blob/main/app/services/cocina/to_fedora/resource_type.rb#L16) and make it a stored value in cocina.

Cocina::Models::DRO.new(
        {
          "type": "http://cocina.sul.stanford.edu/models/image.jsonld",
          "externalIdentifier": "druid:bb000kg4251",
          "label": "Roger Howe Professorship",
          "version": 3,
          "access": {
            "access": "world",
            "download": "world",
            "useAndReproductionStatement": "Property rights reside with the repository. Literary rights reside with  the creators of the documents or their heirs. To obtain permission to publish or reproduce, please contact the Public Services Librarian of the Dept. of Special Collections (http://library.stanford.edu/spc)."
          },
          "administrative": {
            "hasAdminPolicy": "druid:ww057vk7675",
            "releaseTags": [
              {
                "who": "cspitzer",
                "what": "self",
                "date": DateTime.new,
                "to": "Searchworks",
                "release": true
              }
            ],
            "partOfProject": "School of Engineering photograph collection"
          }
        }
      )

raises:

Cocina::Models::ValidationError:
       #/components/schemas/ReleaseTag/properties/date expected string, but received DateTime: -4712-01-01T00:00:00+00:00

Notice the DateTime.

Calling .to_h may create a hash with a DateTime.

Those that are not release tags, project tag or process tag.

To that request is a suffix, rather than prefix so that they sort next to the non-request models.

For example, request_dro becomes dro_request.

This needs to be done in cocina-models but also for every consumer of cocina-models, e.g. https://github.com/search?q=org%3Asul-dlss+requestdro&type=Code

Using same approach as in https://github.com/sul-dlss/dor-services-app/blob/master/.circleci/config.yml#L95-L106

The request version of a model (e.g., dro_request) is very similar to the base model (e.g., dro), except, perhaps, a required field is optional. This creates the risk that a field will be added to only one of these models and not the other.

Please write a test/tests to check the attributes of these models to make sure they stay in sync (and possibly reduce duplicate tests).

Currently, catkey and barcode are stored in the identityMetadata as otherId. We need to store these values in the cocina model.

Allow specifying the visibility after an embargo has expired.

So that they can be used with Sorbet.

For workcycle 2.

See:

• #77 (comment)
• #77 (comment)

We want to remove type coercion from string to integer and from string to boolean; this is not strictly backward compatible, so checking dependents will be necessary (and an appropriate release tag)

We currently have these coercible types:

all of these should become Strict::Bool (PR #77 makes description Params::Bool

Also look for Coercible in the code -- anything without a default(nil) should become Strict.

Controlled digital lending is adding file level access rights and needs to be represented in controlled digital lending: sul-dlss/dor-rights-auth#47

http://cocina.sul.stanford.edu/contexts/cocina-base.jsonld should return a JSON ld document similar to http://schema.org/ (when proper Accept header is set)

This allows it to be parsable:

irb(main):001:0> require 'rdf'
=> true
irb(main):002:0> require'json/ld'
=> true
irb(main):003:0> doc = '{"@context":"http://cocina.sul.stanford.edu/contexts/cocina-base.jsonld","@type":"http://cocina.sul.stanford.edu/models/book.jsonld","label":"This is my object","structural":{"hasMember":[{"@context":"http://cocina.sul.stanford.edu/contexts/cocina-base.jsonld","@type":"http://cocina.sul.stanford.edu/models/fileset.jsonld","label":"file1.png","structural":{"hasMember":["foo-file1"]}},{"@context":"http://cocina.sul.stanford.edu/contexts/cocina-base.jsonld","@type":"http://cocina.sul.stanford.edu/models/fileset.jsonld","label":"file2.png","structural":{"hasMember":["bar-file2"]}}]}}'
=> "{\"@context\":\"http://cocina.sul.stanford.edu/contexts/cocina-base.jsonld\",\"@type\":\"http://cocina.sul.stanford.edu/models/book.jsonld\",\"label\":\"This is my object\",\"structural\":{\"hasMember\":[{\"@context\":\"http://cocina.sul.stanford.edu/contexts/cocina-base.jsonld\",\"@type\":\"http://cocina.sul.stanford.edu/models/fileset.jsonld\",\"label\":\"file1.png\",\"structural\":{\"hasMember\":[\"foo-file1\"]}},{\"@context\":\"http://cocina.sul.stanford.edu/contexts/cocina-base.jsonld\",\"@type\":\"http://cocina.sul.stanford.edu/models/fileset.jsonld\",\"label\":\"file2.png\",\"structural\":{\"hasMember\":[\"bar-file2\"]}}]}}"
irb(main):004:0> require 'json'
=> false
irb(main):005:0> input = JSON.parse doc
=> {"@context"=>"http://cocina.sul.stanford.edu/contexts/cocina-base.jsonld", "@type"=>"http://cocina.sul.stanford.edu/models/book.jsonld", "label"=>"This is my object", "structural"=>{"hasMember"=>[{"@context"=>"http://cocina.sul.stanford.edu/contexts/cocina-base.jsonld", "@type"=>"http://cocina.sul.stanford.edu/models/fileset.jsonld", "label"=>"file1.png", "structural"=>{"hasMember"=>["foo-file1"]}}, {"@context"=>"http://cocina.sul.stanford.edu/contexts/cocina-base.jsonld", "@type"=>"http://cocina.sul.stanford.edu/models/fileset.jsonld", "label"=>"file2.png", "structural"=>{"hasMember"=>["bar-file2"]}}]}}
irb(main):006:0>  JSON::LD::API.expand(input)
Traceback (most recent call last):
       16: from /Users/jcoyne85/.rbenv/versions/2.6.4/lib/ruby/gems/2.6.0/gems/json-ld-3.0.2/lib/json/ld/context.rb:253:in `parse'
       15: from /Users/jcoyne85/.rbenv/versions/2.6.4/lib/ruby/gems/2.6.0/gems/json-ld-3.0.2/lib/json/ld/context.rb:390:in `parse'
       14: from /Users/jcoyne85/.rbenv/versions/2.6.4/lib/ruby/gems/2.6.0/gems/json-ld-3.0.2/lib/json/ld/context.rb:390:in `each'
       13: from /Users/jcoyne85/.rbenv/versions/2.6.4/lib/ruby/gems/2.6.0/gems/json-ld-3.0.2/lib/json/ld/context.rb:444:in `block in parse'
       12: from /Users/jcoyne85/.rbenv/versions/2.6.4/lib/ruby/gems/2.6.0/gems/json-ld-3.0.2/lib/json/ld/context.rb:444:in `call'
       11: from /Users/jcoyne85/.rbenv/versions/2.6.4/lib/ruby/gems/2.6.0/gems/json-ld-3.0.2/lib/json/ld/api.rb:523:in `documentLoader'
       10: from /Users/jcoyne85/.rbenv/versions/2.6.4/lib/ruby/gems/2.6.0/gems/rdf-3.0.12/lib/rdf/util/file.rb:297:in `open_file'
        9: from /Users/jcoyne85/.rbenv/versions/2.6.4/lib/ruby/gems/2.6.0/gems/rdf-3.0.12/lib/rdf/util/file.rb:119:in `open_url'
        8: from /Users/jcoyne85/.rbenv/versions/2.6.4/lib/ruby/2.6.0/net/http.rb:605:in `start'
        7: from /Users/jcoyne85/.rbenv/versions/2.6.4/lib/ruby/2.6.0/net/http.rb:919:in `start'
        6: from /Users/jcoyne85/.rbenv/versions/2.6.4/lib/ruby/2.6.0/net/http.rb:930:in `do_start'
        5: from /Users/jcoyne85/.rbenv/versions/2.6.4/lib/ruby/2.6.0/net/http.rb:945:in `connect'
        4: from /Users/jcoyne85/.rbenv/versions/2.6.4/lib/ruby/2.6.0/timeout.rb:103:in `timeout'
        3: from /Users/jcoyne85/.rbenv/versions/2.6.4/lib/ruby/2.6.0/timeout.rb:93:in `block in timeout'
        2: from /Users/jcoyne85/.rbenv/versions/2.6.4/lib/ruby/2.6.0/net/http.rb:946:in `block in connect'
        1: from /Users/jcoyne85/.rbenv/versions/2.6.4/lib/ruby/2.6.0/net/http.rb:949:in `rescue in block in connect'
JSON::LD::JsonLdError::LoadingRemoteContextFailed (loading remote context failed: http://cocina.sul.stanford.edu/contexts/cocina-base.jsonld: Failed to open TCP connection to cocina.sul.stanford.edu:80 (getaddrinfo: nodename nor servname provided, or not known))

APOs have a section in the administrativeMetadata that records descriptive metadata format and source. Users supply this information when filling out the Register APO and Edit APO forms.

  <descMetadata>
    <format>MODS</format>
    <source>DOR</source>
  </descMetadata>

We are currently not mapping this data to Cocina. But I'm not sure what this data actually does. It seems like any item could have Symphony-originated metadata or user-supplied metadata but I could be missing something.

If this data isn't used, we should take it out of the interface rather than map it. If it does get used we should map it.

Blocks sul-dlss/dor-services-app#2418

This will replace the default collection from dor-services

When a collection is created in argo, the APO for that collection is updated to make the new collection a “default collection” (//administrativeMetadata/registration/collection). These nodes are what governs what collections show up when registering items.

See: https://github.com/sul-dlss/argo/blob/main/app/controllers/registrations_controller.rb#L119
https://github.com/sul-dlss/dor-services/blob/1743ea8c144467d0167c333d34cfb62b9d2eb7ec/lib/dor/datastreams/administrative_metadata_ds.rb#L119-L143
https://github.com/sul-dlss/argo/blob/main/app/controllers/collections_controller.rb#L25

I think all objects must have an APO. Is that right @andrewjbtw ?

When I look at the Cocina model for an object and don't see all the data mapped there, should I assume it's not mapped?

Example:

• This item has a geoMetadata datastream: https://argo.stanford.edu/view/bq841qm7759
• The Cocina model for that item has no "geographic" subsection (though this is part of the Cocina API)
• There is geographic data within the "description" subsection (from the MODS?)

Does that mean the data is not mapped yet? This item has been released to Earthworks, so the data published to the Purl needs to include geodata. If we took Fedora out of the process, would the Purl still publish with the right data?

Should be "naf", not "lcnaf". Checked tests - not referenced, no test update needed.

• Fixtures
• OpenAPI

(assigned to me)