julielab / jcore-base Goto Github PK
View Code? Open in Web Editor NEWBase modules of JCoRe
License: BSD 2-Clause "Simplified" License
Base modules of JCoRe
License: BSD 2-Clause "Simplified" License
... as the title says.
Either it has to do with some missing infos in the cpe itself, or with the unconventional way of handling shared resources.
Pipeline crashes, because it doesn't find the config file (that's only a deduction and nothing that is reported).
Right now, the .gitignore
file of the jcore-base
parent folder only specifies "ignores" for its own level and every subfolder has it's own .gitignore
that specifies some similar "ignores" (like .classpath
, .settings
etc., i.e. IDE specific files that appropriately shouldn't be committed).
However, it would be nice, to have all these "global ignores" in the parent .gitignore
and leave the project .gitignores
for local "ignores" only. You can do this with the asteriks
A slash followed by two consecutive asterisks then a slash matches zero or more directories. For example, "a/**/b" matches "a/b", "a/x/b", "a/x/y/b" and so on.
postponed until we have some more wo_manpower
right now the recently added consumer just outputs sentences (one per line) and tokens (whitespace separated).
it also needs a feature to add POS information to each token if available (& desired) so that the POS tag is appended to each token with a designated delimiter (e.g. _
or |
).
Furthermore README :)
@egrygorova: Please contact me when you're done with your former task so we can tackle this one.
Its currently set to log4j
The tests don't run because resource files are missing. I set the tests to ignore for the moment, which obviously shouldn't stay that way.
Right now, the Postprocessing done by JSBD is hardcoded, as well as an abbrevation lexicon.
The postprocessing steps were probably fitted to "biomedical english abstracts". This results in some issues with "medical german discharge summaries etc."
At least the abbrevation lexicon should be "outsourced".
I get an error in Eclipse for this project (but there isn't any further indication to what this error might be). When I try to deploy jcore-base-SNAPSHOT it throws an error as well, when coming to the point of preparing the assigner.
Is it just me?
Please write the README for JEmAS.
As of Mar 24, 2017 our licensing policy for
"
There is an ongoing group-intern debate on whether source code (e.g., the JCoRe code we distribute via GitHub) must comply with the licenses of the employed libraries if those are not included in the distribution and instead only referenced as a maven dependency (that would mean that using the API in the source makes the the source a derivative) or the source code rather constitutes a work of its own and only building the project merges source and libraries (thus constituting a derivative).
"
Anyways, I would recommend setting the BSD-2 clause license now and a prominent note at the top of the REAMDE that third party libraries might have different licenses -- better: make a list of all the third-party dependencies and their licenses respectively.
due to the special version of the uea-stemmer packaged in jcore-dependencies, old models are not compatible anymore (can't be loaded due to differences in the uea objects)
When using the BioNLP09 consumer from a CPE descriptor in the following way
<casProcessor deployment="integrated" name="BioNLP09Writer"> -->
<descriptor>
<import name="de.julielab.jcore.consumer.bionlp09event.desc.jcore-bionlp09event-consumer" />
</descriptor>
<deploymentParameters />
<configurationParameterSettings>
<nameValuePair>
<name>Directory</name>
<value>
<string>st09-bionlp09-format</string>
</value>
</nameValuePair>
</configurationParameterSettings>
<errorHandling>
<errorRateThreshold action="terminate" value="10/1000" />
<maxConsecutiveRestarts action="terminate"
value="30" />
<timeout max="100000" default="-1" />
</errorHandling>
<checkpoint batch="1" time="1000ms" />
</casProcessor>
on initialization of the CPE via the runCPE.sh script I get the message
<stack trace>
Caused by: java.lang.Exception: The component BioNLP09Writer cannot be created. (Thread Name: main)
When indexing for semedico, apparently every document print a warning like this:
11:30:21.091 [[Procesing Pipeline#2 Thread]::] WARN d.j.j.c.e.AbstractFieldsGenerator - Section annotation in document 4321653 occured with begin=8006 and end=9270 (document text length: 9270). Ignoring
Check if the offsets of the sections are correct.
The test fails due to the existence of .gitignore in the input directory. Has been fixed in the 2.1.x-bugfixes branch which would have to be merged.
There is no information about it.
Please write the README for this type system.
Currently only the reference ID is set.
Basically the version mentioned in all UIMA descriptors are wrong (still at 2.0.0 or even 2.0.0-SNAPSHOT). There should be some sort of script that automatically adapts those versions for all component and type descriptors.
The consumer does apparently not create the directories to the output directory itself. This is weird when using the default configuration. It should just work out-of-the-box, the output directories should automatically be created, if not existing.
Just convert it to the JCoRe conventions.
Create a descriptor
Sometimes it is important to have access to the exact CAS string as it is stored within UIMA. For instance, when outputting the text with annotations in an offset-based manner, the original document string must not be changed.
JSBD should be able to be given a number of annotation types where sentences should always end (titles, sections etc). This would fix a range of issues in full texts.
Too complicated to write the README properly.
Please review the README.
added a script that converts sgml to xml files expected by the reader; however the script is just rudimentary right now, but works for all files of the following structure:
<DOC>
<DOCID> nyt960214.0765 </DOCID>
<STORYID cat=a pri=r> A4505 </STORYID>
<SLUG fv=ttx-z> BC-<COREF ID="1">PANTEX</COREF>-<COREF ID="3">FLIGHTS</COREF>-TEX </SLUG>
<DATE> <COREF ID="104">02-14</COREF> </DATE>
<NWORDS> 0535 </NWORDS>
<PREAMBLE>
[...]
The script takes as argument the name of the file to convert
python muc7_SGML2XML.py training.tr.keys.980410
and produces a file with the same name but an additional ".xml" ending.
But:
it seems the reader doesn't annotate coreferences in the CAS? Need to investigate!
The current readme is some copy of the JSBD readme.
In the type system (jcore-morpho-syntax-types.xml), "lemma" is a feature of the annotation type token. The value of the lemma feature has to be of type lemma which in turn has only one feature of type string.
Why not deleting this intermediate lemma type and set the lemma feature of the type token to be a string right away?
Please write the README for jCore utilities.
The folder structure in jcore-base/jcore-opennlp-token-ae/ does not conform to most of the other components.
Also, the readme file needs editing ;-)
mstparser now depends on trove4j and not jules-trove
therefore we need a new test-model
this will also affect the mstparser projects in jcore-projects
we need a new genia model there as well
make sure that
jcore-parent
JULIE Coordination Resolution
is addedOpenNLP Constituency Parser
is addedThe log message "No organisms" is actually quite helpful during development. However, it is currently just a system.out.println. Please make that a logging message.
I am looking for the type de.julielab.jcore.types.AbstractPart
mentioned in jcore-type-priorities.xml
, but I cannot find it in the type systems. Did I miss it?
@JULIELab/core Just a sidenote (but an important one): if you add a component to the repository or even if you "only" add features to a component, please create/update the readme's accordingly.
changed the class from Consumer to generic AE
need to fix some tests
Please review those README's, thank you.
they both utilize now the "official mallet" artifact from Maven Central and not our "homebrewn" mallet version. The new performance for the FraMed corpus is documented. Need to check for e.g. Genia as well?!
All (?) AEs have some "ae" in the name, but the stanford lemmatizer has not. Is there a particular reason for that? I think it was jcore-stanford-lemmatizer-ae in the past it has been changed. AE or not AE, that is the question!
We have the writer in JCoRe, but we cannot easily read the XMIs. There is an XMI reader in Jules...
Just for the sake of completion and consistency (as discussed the other day): we should outsource the lexicon (plus descriptor) to a jcore-projects
project; similar to e.g. jcore-jsbd
has two referencing projects medical and biomedical.
so that they're able to read all st13 information as well
please provide some information about that component, hence I can make a README for it
due to changes in the names for some packages, we need to retrain the models (biomed & medical) and deploy them with new projects for 2.3.0
add conll consumer to jcore base
jcore-opennlp-token-ae depends on trove4j 3.0.3
jcore-jtbd-ae depends on trove4j 2.0.2 through jcore-mallet 2.0.9
... causes java.lang.ClassNotFoundException: gnu.trove.TObjectIntHashMap when both tokenizers are used in one project.
For easier reference add the descriptor path of a component (e.g. de.julielab.jcore.ae.[...]
) to the top line of each readme.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.