mff-uk / dataspecer Goto Github PK
View Code? Open in Web Editor NEWHome Page: https://dataspecer.com
License: MIT License
Home Page: https://dataspecer.com
License: MIT License
Related work https://linkml.io/
Prefixes are specified as a part of schema definition with psm:prefixName
and ``` psm:prefixIRI`` predicates. All prefixes are optional, meaning that they are used in output jsonld context only if they were applied to some IRI in the context.
With importing context all prefixes are imported as well, meaning contexts once declared in the imported context are not re-declared.
Changes introduced in e78cd93 breaks the build because of the unknown
type being too strict.
Following PSM operations are required for now.
Schema
Class
Attribute
Association
As of now, it is possible to request all resources, load them and find those with desired type. This is not optimal solution when searching for schema root. A solution is to allow user of the interface to request only resources of particular RDF type.
As of now each resource has a string, that is used to determine the object type at runtime - classic TypeScript approach. This string is artificial, we should utilize RDF types instead.
Following PIM operations are required for now.
Schema
Class
Attribute
Association
The PimMemoryStore
does not delete resources as specified by an operation.
As of now each operation produce general object that capture the change in the model. It would be easier to add operation specific results - for example DataPsmCreateClasss can return an extra field with name of the created class.
As of now there is CoreModelReader
and CoreModelWriter
. As there is now no single model, it would be better to rename them to CoreResourceReader
and CoreResourceWriter
.
We need an interface that allows us to write objects as RDF.
edit: We also need the implementation.
Implement a proxy for httpFetch to limit the number of concurrent requests.
As of now the operation result contains list of changed and deleted properties. We need to add list of new properties. This information is easily available to the operations and will make use of the operation results much easier.
As of now there is not interface for text output. Bikeshed nad ReSpec employ NodeJs WriteStream
and consequently they can not be used in browser.
A solution is to introduce a new interface, e.g.
interface TextSink {
write(content:string) : Promise<void> ;
flush() : Promise<void> ;
}
As a next step a browser and NodeJs version should be implemented. The browsers version can utilize string concatenation while the NodeJs version can use the NodeJs ```WriteStream``.
Once #7 is solved, we need to implement this interface by all adapters to allow for RDF IO operations.
data-psm
stands for data platform specific model
so it make sense to shorten that to dpsm
instead of the original one.
We need to make sure that we provide IO interfaces as argument in all relevant places.
In sgov-adapter.ts
.
Although JS code runs in a single thread it is possible to call a single function multiple times using async
/wait
and Promises
. This can be problem for the CoreModelWriter
as we do not support running of multiple operations simultaneously. This must be at least document so the user is aware of this.
In addition we may try to introduce some mechanism that would make sure that operations are evaluated one at a time. We just nee to keep in mind, that the Store, implementing the ````CoreModelWriter``` may be on a server. So the functionality may need to be close to the store.
As of now we use CIM only as a source of data to pre-fill PIM operations. But there are some common properties, like color or "group" that we may want to use across multiple applications. A solution is to introduce a shared CIM interface.
The following files are breaking the build because of bugs:
lib/platform-independent-model/adapter/rdf/pim-resource-adapter.ts
lib/io/rdf/sparql/sparql-rdf-source.ts
lib/io/rdf/jsonld/jsonld-adapter.ts
lib/data-psm/adapter/rdf/data-psm-resource-adapter.ts
We need to agree on common approach to using Undefined , Null and exceptions in the API. The objective of this issue is to collect ideas and decide on how we will use those. Once decided another issue should be created to implement those decision and describe them in the readme file / wiki.
Edit: In addition we may need to think about handling "expected" errors as invalid data, invalid operations or unavailable service. As typescript does not support exception type declaration the alternative is to return error object as an alternative to the regular function result. In order to make it easy to work with other code, there can be a wrapper that will with getOrThrow
method.
Entity ze slovník.gov.cz oproti obecným entitám mají navíc atribut glosář. Obecně pak různé CIM mohou entitám přiřazovat libovolné (předem definované pro CIM) atributy. Ty je potřeba ukládat v rámci PIM modelu, neboť PIM musí být od CIMu datově nezávislý.
Tedy je potřeba rozmyslet, jak budou fungovat adaptéry na ukládání modelu, který tímto obsahuje data specifická pro konkrétní CIM.
The goal is to implement a read-only store for different sources described by URL. For now, we only consider:
We need to be able to query CoreResourceReader
for resources with particular interpretation or ends of the association.
Possibilities are:
We need to be able to bundle and execute the transformers using command lined interface.
Add interface and implementation that would allow us to load / save stores to given URL.
The URL may represent a simple REST API (GET, POST) or potentially storage of certain type LDP, Solid.
As stated in #12 there is need to optimize reading of remote resources. While the proposed solution address situations when user request multiple properties, it may not always be optimal from the code perspective.
A solution is to introduce a cache that would read the whole object at once, using this cache for future request. We may still implement this to not force user to request all properties at one.
As a result user can still request properties one at the time while not producing another SPARQL query for each call.
As of now some architectural decisions are in the readme file. Those must be moved to the Wiki together will all relevant information and decision related to this repository.
As of now RdfSource
allow use to read one property at a time. When reading multiple properties for a single object such approach is sub-optimal.
In addition we can also add new methods that allow user to request multiple properties, e.g.
properties(iri: string[], predicate: string): Promise<{[predicate:string] : RdfObject[]}>;
and similar for reverse property.
The default implementation may utilize the property
and reverseProperty
operations. In the final step an optimization can be implemented for SparqlSource.
Once #3 is finished, we need to update the ReSpec and Bikeshed writers to utilize the new interface.
Each operation should validate the prerequisites in scope of the schema. In the current state not everything that should be validated is, the purpose is to validate all that is possible. As a result if the schema is valid before the operation it must be valid after the operation.
As a part of this issue we may also introduce a better way how to handling invalid state, i.e. using exceptions of particular type rather than return statements that clutter the code with if-return statements.
As we have data-psm we should have pim.
The goal is to implement a Webpack loader for *.sparql
files that minimizes and transforms them into a parametrized function replacing the template variables inside them.
It is useful to have ability to read PIM/PSM using CoreModelReader
interface. However, this interface is now implemented only by PimMemoryStore
, which does not allow for easy modification. A solution is to introduce new store that would implement CoreModelReader
using user provided list of resources as a database.
If the dataPsmMoveAfter
is null the item should be moved to the first place.
Implement ClassReference and actions for it.
Comming in TypeScript 3.7 the ??
is binary operator that returns the second value iff the first one is null
or undefined
. This is better then ||
that returns the second argument iff the first one is false; this includes false
, 0
, ""
.
We also need to make sure to use
"compilerOptions": {
"strict": true,
"target": "ES2020"
}
as a configuration to propagate ??
to ES as it is now part of ES2020.
We can use decorators to set object types, generate "is" functions or load data from RDF. This would allow us to decrease and simplify model code.
https://www.typescriptlang.org/docs/handbook/decorators.html
We need to be able to build bundles of the code to allow to import in other projects.
Once #35 is finished we need to migrate also all that is needed to run Bikeshed and ReSpec generators.
Merge develop into main and use main branch instead. As new features are developed in feature branches the main should not be broken so often.
We may need to measure performance in order to be able to optimize. This is not easy due to a heavy use of async calls. A solution might be to design a wrappers that could be used with the classes from this repository that would allow to measure execution time of particular operations.
While using GitHub wiki provide us with some advantages like menu, etc .. it is not suitable for rapidly changing software as it is not easy to connect commit and wiki version. A solution is to move the wiki into the code.
As of now model-driven-data host a single project. It may be beneficial to create a monorepository.
This would allow us to have repository for different generators (xsd, json-schema, ... ) in addition it would support development of UI directly within this repository.
As of now the object model is not working, we need to update it to work with the current data-psm and pim classes in the develop.
We need more restrictive Linter rules to increase code style consistency.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.