mff-uk / dataspecer Goto Github PK

Related work https://linkml.io/

Prefixes are specified as a part of schema definition with psm:prefixName and ``` psm:prefixIRI`` predicates. All prefixes are optional, meaning that they are used in output jsonld context only if they were applied to some IRI in the context.

With importing context all prefixes are imported as well, meaning contexts once declared in the imported context are not re-declared.

Changes introduced in e78cd93 breaks the build because of the unknown type being too strict.

See https://www.typescriptlang.org/play?#code/MYGwhgzhAEAiYBcwAUIFsCCUD2wCWie2AdgKLEAm0A3gL4BQwJECciK6WEuBCRZlACoBPAA4BTaAF5oAIlkBuevXEAPUdgBOrAGYBXYsD4loeCPCSpMOfIRLkKACnrRom8dz2bg4gFzQDAGtibAB3YgAaegBKf3dPb0kzNktOG15+BxoXN3EEL2JchJ9oAEIpGWI9EBAc1wAyeqLsLx8AfgA6BDEPTrxDED0KD0cLDmtuW2MBChEJaKVaIA

Following PSM operations are required for now.

Schema

• Create schema
• Modify schema label and description
• Set schema root (only at creation)

Class

• Create (interpreted) class
• Modify class label and description
• Modify class technical label
• Change order of class parts

Attribute

• Create (interpreted) attribute
• Modify attribute label and description
• Modify attribute technical label
• Delete attribute (+delete itself from class parts)

Association

• Create (interpreted) association
• Modify association label and description
• Modify association technical label
• Delete association (+delete itself from class parts)

As of now, it is possible to request all resources, load them and find those with desired type. This is not optimal solution when searching for schema root. A solution is to allow user of the interface to request only resources of particular RDF type.

As of now each resource has a string, that is used to determine the object type at runtime - classic TypeScript approach. This string is artificial, we should utilize RDF types instead.

Following PIM operations are required for now.

Schema

• Create schema
• Modify schema label and description

Class

• Create class
• Modify class label and description
• Delete class

Attribute

• Create attribute
• Modify attribute label and description
• Modify attribute datatype
• Delete attribute

Association

• Create association
• Modify association label and description
• Delete association

The PimMemoryStore does not delete resources as specified by an operation.

As of now each operation produce general object that capture the change in the model. It would be easier to add operation specific results - for example DataPsmCreateClasss can return an extra field with name of the created class.

As of now there is CoreModelReader and CoreModelWriter. As there is now no single model, it would be better to rename them to CoreResourceReader and CoreResourceWriter.

We need an interface that allows us to write objects as RDF.
edit: We also need the implementation.

Implement a proxy for httpFetch to limit the number of concurrent requests.

As of now the operation result contains list of changed and deleted properties. We need to add list of new properties. This information is easily available to the operations and will make use of the operation results much easier.

As of now there is not interface for text output. Bikeshed nad ReSpec employ NodeJs WriteStream and consequently they can not be used in browser.

A solution is to introduce a new interface, e.g.

interface TextSink {
  write(content:string) : Promise<void> ;
  flush() : Promise<void> ;
 }

As a next step a browser and NodeJs version should be implemented. The browsers version can utilize string concatenation while the NodeJs version can use the NodeJs ```WriteStream``.

Once #7 is solved, we need to implement this interface by all adapters to allow for RDF IO operations.

data-psm stands for data platform specific model so it make sense to shorten that to dpsm instead of the original one.

We need to make sure that we provide IO interfaces as argument in all relevant places.

In sgov-adapter.ts .

Although JS code runs in a single thread it is possible to call a single function multiple times using async/wait and Promises. This can be problem for the CoreModelWriter as we do not support running of multiple operations simultaneously. This must be at least document so the user is aware of this.

In addition we may try to introduce some mechanism that would make sure that operations are evaluated one at a time. We just nee to keep in mind, that the Store, implementing the ````CoreModelWriter``` may be on a server. So the functionality may need to be close to the store.

As of now we use CIM only as a source of data to pre-fill PIM operations. But there are some common properties, like color or "group" that we may want to use across multiple applications. A solution is to introduce a shared CIM interface.

The following files are breaking the build because of bugs:

• lib/platform-independent-model/adapter/rdf/pim-resource-adapter.ts
• lib/io/rdf/sparql/sparql-rdf-source.ts
• lib/io/rdf/jsonld/jsonld-adapter.ts
• lib/data-psm/adapter/rdf/data-psm-resource-adapter.ts

We need to agree on common approach to using Undefined , Null and exceptions in the API. The objective of this issue is to collect ideas and decide on how we will use those. Once decided another issue should be created to implement those decision and describe them in the readme file / wiki.

Edit: In addition we may need to think about handling "expected" errors as invalid data, invalid operations or unavailable service. As typescript does not support exception type declaration the alternative is to return error object as an alternative to the regular function result. In order to make it easy to work with other code, there can be a wrapper that will with getOrThrow method.

Entity ze slovník.gov.cz oproti obecným entitám mají navíc atribut glosář. Obecně pak různé CIM mohou entitám přiřazovat libovolné (předem definované pro CIM) atributy. Ty je potřeba ukládat v rámci PIM modelu, neboť PIM musí být od CIMu datově nezávislý.

Tedy je potřeba rozmyslet, jak budou fungovat adaptéry na ukládání modelu, který tímto obsahuje data specifická pro konkrétní CIM.

The goal is to implement a read-only store for different sources described by URL. For now, we only consider:

• Static file located on the Internet in RDF format (Turtle for example)

We need to be able to query CoreResourceReader for resources with particular interpretation or ends of the association.
Possibilities are:

• Return all resources with arbitrary predicate and object of given value
• Let user specify RDF predicate and object
• Let user query by example using model objects

We need to be able to bundle and execute the transformers using command lined interface.

Add interface and implementation that would allow us to load / save stores to given URL.
The URL may represent a simple REST API (GET, POST) or potentially storage of certain type LDP, Solid.

As stated in #12 there is need to optimize reading of remote resources. While the proposed solution address situations when user request multiple properties, it may not always be optimal from the code perspective.

A solution is to introduce a cache that would read the whole object at once, using this cache for future request. We may still implement this to not force user to request all properties at one.

As a result user can still request properties one at the time while not producing another SPARQL query for each call.

As of now some architectural decisions are in the readme file. Those must be moved to the Wiki together will all relevant information and decision related to this repository.

As of now RdfSource allow use to read one property at a time. When reading multiple properties for a single object such approach is sub-optimal.

In addition we can also add new methods that allow user to request multiple properties, e.g.

properties(iri: string[], predicate: string): Promise<{[predicate:string] : RdfObject[]}>;

and similar for reverse property.

The default implementation may utilize the property and reverseProperty operations. In the final step an optimization can be implemented for SparqlSource.

Once #3 is finished, we need to update the ReSpec and Bikeshed writers to utilize the new interface.

Each operation should validate the prerequisites in scope of the schema. In the current state not everything that should be validated is, the purpose is to validate all that is possible. As a result if the schema is valid before the operation it must be valid after the operation.

As a part of this issue we may also introduce a better way how to handling invalid state, i.e. using exceptions of particular type rather than return statements that clutter the code with if-return statements.

As we have data-psm we should have pim.

The goal is to implement a Webpack loader for *.sparql files that minimizes and transforms them into a parametrized function replacing the template variables inside them.

It is useful to have ability to read PIM/PSM using CoreModelReader interface. However, this interface is now implemented only by PimMemoryStore, which does not allow for easy modification. A solution is to introduce new store that would implement CoreModelReader using user provided list of resources as a database.

If the dataPsmMoveAfter is null the item should be moved to the first place.

Implement ClassReference and actions for it.

Comming in TypeScript 3.7 the ?? is binary operator that returns the second value iff the first one is null or undefined. This is better then || that returns the second argument iff the first one is false; this includes false, 0, "".

We also need to make sure to use

"compilerOptions": {
    "strict": true,
    "target": "ES2020"
  }

as a configuration to propagate ?? to ES as it is now part of ES2020.

We can use decorators to set object types, generate "is" functions or load data from RDF. This would allow us to decrease and simplify model code.

https://www.typescriptlang.org/docs/handbook/decorators.html

We need to be able to build bundles of the code to allow to import in other projects.

Once #35 is finished we need to migrate also all that is needed to run Bikeshed and ReSpec generators.

Merge develop into main and use main branch instead. As new features are developed in feature branches the main should not be broken so often.

We may need to measure performance in order to be able to optimize. This is not easy due to a heavy use of async calls. A solution might be to design a wrappers that could be used with the classes from this repository that would allow to measure execution time of particular operations.

While using GitHub wiki provide us with some advantages like menu, etc .. it is not suitable for rapidly changing software as it is not easy to connect commit and wiki version. A solution is to move the wiki into the code.

As of now model-driven-data host a single project. It may be beneficial to create a monorepository.
This would allow us to have repository for different generators (xsd, json-schema, ... ) in addition it would support development of UI directly within this repository.

As of now the object model is not working, we need to update it to work with the current data-psm and pim classes in the develop.

We need more restrictive Linter rules to increase code style consistency.