Comments (4)
I have made some progress on this. I would love input on ideas with regards to matching the file containing the license.
I don't want to open each file - this would be too heavy given the runtime of the license matcher. Right now, I'm using a regular expression:
(licens.*)|(reuse)|(copy)
But I think more searchterms can be added to this, I'm also considering adding different languages for license.
With regard to the serialization of the resulting license triples - I am using schema.org/license for the predicate, and spdx ontology for the object. The subject is still a question: What should the object be, to which we attach all our properties? I was thinking something like: "some_repo_related_prefix:[insert_repos_name]". This instance can then be rdf:type'd as a schema.org/CodeRepository.
Check out my branch! (License_scanner)
from gimie.
But I think more searchterms can be added to this, I'm also considering adding different languages for license.
I agree checking LICENS., COPY. and REUSE.* would already be a good start. We can always think about optimizing this later.
With regard to the serialization of the resulting license triples
It will be easier to maintain and test if we separate the code that gets the license (path -> URI) from the code that materializes the triple or serializes the RDF. It's probably easier if the URI is simply kept as an attribute of LicenseMetadata
in this part of the code.
What should the object be, to which we attach all our properties?
For remote repositories, I guess it would make sense to use the remote URL ? For local repositories, indeed we likely need to create some prefix... I'm not sure about the requirements for this, but AFAIK it should :
- be globally unique
- yield the same URI if we run gimie twice on the same repo
- yield different URI if we run gimie twice on different repos with the same name
Maybe something along the lines local://repo-name/first-commit-hash
could do the trick, but I think this would require some separate discussion.
from gimie.
Just to come back to this point, I think something like this could work:
If it's an online repository, we can use the repository URL as a URI.
Advantages:
- The URI is immediately dereferencable to the actual thing we are trying to describe in triples. For physical objects like a building this is not possible, and you always need some kind of "representation" in the form of a URI, but for an online repository it's pretty elegant.
- No need for uuid's or other unique-making identifiers to be postfixed or prefixed.
- If we use our own URI like www.example.com/myrepository this should be hosted somewhere, and redirect to the online repo. Using the actual online repository URL circumvents this.
Disadvantages:
- I'm not sure how GitHub deals with persistence. If you change the name of your repo, does the old name become available again for others to use? And even if that's not the case, when GitHub/lab/whatever is down or the repo is removed or name is changed, the URI is no longer dereferencable... That would kind of defeat the purpose.
For a local repository, we could probably just do indeed a local://repo-name/ and hash combination, I like this idea, but would maybe advocate truncating such a hash, like 5 characters or something. This would still be plenty robust in our use case I think (collision factor for 6 character hash at 100 000 entries is already like 17%, so adding the reponame as a further "hash" would reduce that number even more. And that is under the assumption of 100k entries. our scenario is probably more likely to stay in the realm of 100's or at most 1000's of tools.
The shorter and more readable the URI, the easier it is to use in queries. This SHOULD not be an important factor in deciding a URI, but seeing as labels and definitions/comments are not always added, having a somewhat readable meaningful URI can be useful in small scale projects like this.
I'll do some experimentation with truncating a hash algorithm and adding it to the make-a-graph part of the script. This kind of URI generation should probably be a method for some class so it can be re-used everytime we want to make a triple.
from gimie.
Thanks ! Sounds good, feel free to create a new issue to define a URI :)
If you change the name of your repo, does the old name become available again for others to use?
AFAIK, the old URL redirects to the new one as long as it is left unused. As names are scoped by user/organization, this is usually not a big problem.
This kind of URI generation should probably be a method for some class so it can be re-used everytime we want to make a triple.
Indeed, that would make sense
from gimie.
Related Issues (20)
- Incorrect value for codeRepository
- Licenses not being picked up correctly HOT 2
- Extract license when unavailable HOT 3
- provide generic file object
- Reduce extractor complexity
- Add Parser concept HOT 1
- [ IP] publishing date - not pulled out of gimie HOT 2
- Fix docker push CI HOT 5
- Implement license detection for GitExtractor
- make list_files recursive HOT 9
- write contribution guide
- enforce conventional PR titles
- prevent docs rebuild on non-default branches HOT 1
- Implement license matcher HOT 2
- Optimize Github Actions HOT 1
- Optimize gimie container size
- License identifier not matched correctly HOT 1
- License (SPDX) maintenance strategy HOT 1
- Move code out of __init__ files
- graphql error on some large repos
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gimie.