Giter VIP home page Giter VIP logo

cocoda-mappings's Introduction

This repository contains concordances between knowledge organization systems (KOS) and scripts to harvest and convert these concordances.

This work is part of project coli-conc.

  • the base directory contains PHP scripts to show concordances at https://coli-conc.gbv.de/concordances/

  • directory wikidata contains scripts to harvest mappings from Wikidata

  • directory csv contains concordances and mappings from different sources in CSV format

  • directory zbw contains scripts to convert mappings provided by ZBW

See file README.md in each particular directory for further documentation.

SOURCES AND ISSUE TRACKER

https://github.com/gbv/cocoda-mappings/

REQUIREMENTS

  • wdmapper
  • PHP >= 7.0 with ext-gd enabled
  • Catmandu (for Perl scripts)
$ sudo apt-get install php-gd
$ composer install

cocoda-mappings's People

Contributors

davidbrohrer avatar dennistobola avatar hornmo avatar janaag avatar nichtich avatar niklasben avatar stefandesu avatar zuphilip avatar

Stargazers

 avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cocoda-mappings's Issues

Sparklines updated?

The sparklines are a great way to illustrate the growth for a property or perhaps also a mapping.

However, it seems that they are not created anew after changes. (e.g., in the last days, I have added lots of GND and RePEc ids, but the sparklines did not change).

Versioning of mappings

Committing to a dedicated git repository seems to be the easiest way. For better diffs, sort mappings before commiting!

Add concordance metadata such as date

ddc_bk_chem - ?
ddc_bk_pol - ?
ddc_rvk_1000 - 2013
ddc_rvk_100 - 2013
ddc_rvk_biblinf - 2009
ddc_rvk_ethik - 2015
ddc_rvk_medizin - 2011
ddc_rvk_philo_psych
ddc_rvk_recht - 2012
gnd_ddc_crisscross - 2010
rvk_bk_recht - 2013
rvk_bk_wiwi - 2013
rvk_ddc_ethik - 2015
rvk_ddc_philo_psych
rvk_gnd_ubregensburg - 2015
stw_gnd_exact_zbw

Add MSC concordances

We have some MSC concordances to be added. MSC is revised every 10 years so it must be tracked which version to map against.

Add type of KOS and allow filtering by KOS type

General KOS types (thesaurus, classification...) are given by BARTOC. These types can be shown like tags at each KOS. A list of checkboxes at the top could allow showing/hiding concordances because the current list becomes too long.

Timeout for Geonames

More than 1.5 Million mappings but still possible to download on local machine:

$ time wdmapper get P1566 -g '' -o P1566.txt

real	1m27.590s
user	0m52.872s
sys	0m5.252s

On the server, however the call dies after half an hour with segfault or similar. Further debugging required. The SPARQL query is

SELECT ?item ?target WHERE { ?item wdt:P1566 ?target }

Looks like a bug in wdmapper (?)

Add documentation for kos-registry

kos-registry is now included as a git submodule, but there are no instructions that it is necessary to fetch that. So we need something like this:

# Either
git submodule init
git submodule update
# Or
git clone --recurse-submodules https://github.com/gbv/cocoda-mappings.git

Add mapping URIs

Imported mappings from concordances should get an URI to be annotateable. Could be based on mapping content identifier plus partOf (see gbv/jskos#68)

csv: file 'kos.yaml' doesn't exist

Since moving kos-registry to a submodule, make in the csv folder won't run anymore because file 'kos.yaml' doesn't exist. Either the Makefile has to be adjusted to use the file from ../kos-registry/, or we need a symlink.

Missing properties

wdmapper does not harvest mappings if

  • property lacks URL template (P1630): P1691, P1190, P667, P1189, P2347, P1762, P714, P2452, P305
  • property has multiple URL templates (P1630): P486, P1256, P1257
  • property is not of type external-id or string: P2643

csv: Add documentation about dependencies and usage

For make to work in the csv directory, some perl dependencies are necessary. Maybe include something like cpanm Catmandu Catmandu::MARC. Also include documentation about what make and make ndjson does (and that it exists in the first place).

Default sort of wikidata mappings table

Short of filing an issue for sortability, I detected in the code that the table already is sortable - which is great. Two additonal suggestions:

  • default sort by KOS name (which perhaps what most people would expect. Another reasonable default could be size descending)
  • add some indicator, that the columns are sortable, and which sort is active

Change RVK URIs

Use %20 instead of _ and respect spaces in number ranges.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.