ezpaarse-project / ezpaarse Goto Github PK
View Code? Open in Web Editor NEWezPAARSE can ingest your (proxy) log files and show how users access suscribed electronic ressources.
Home Page: http://www.ezpaarse.org
License: Other
ezPAARSE can ingest your (proxy) log files and show how users access suscribed electronic ressources.
Home Page: http://www.ezpaarse.org
License: Other
When the format of the first line can't be recognized, the server ends the response but the client doesn't do anything, and the progress bar freezes at its current progress.
The other errors are correctly handled on client side, probably because the response is immediate in those cases. In case of unknown format, the request has to be read a bit before the error can be sent.
Manuel cancel is needed to stop the process
Because of the lint hook, each git commit is very slow (about 5 seconds on my machine)
I like to commit lot of atomic things but this slow step makes you want to commit only once a lot of things into the same commit.
Moreover, it checks also none ezpaarse code, for example platform one (which is not link checked on its side) so it's frustrating to be not allowed to abort its own commit cause someone else did not respect lint in the platforme code.
Trying to install ezpaarse based on the Readme instructions, I encounter an error message :
ln: impossible de créer le lien symbolique '/home/ezpaarse/test/ezpaarse/build/nvm/bin/latest': Aucun fichier ou dossier de ce type
Makefile:98 : la recette pour la cible « nodejs » a échouée
It can be fixed by manually running the command:
mkdir build/nvm/bin/
from the install directory but it should be done by the makefile. Sorry for just opening the issue and not submitting a patch but my Makefile skills are below the required level :)
With a source directory containing files like " ezproxy.log-20141231.gz" the regex used to detect whether the file should be processed or not does not match
Happened several times while processing ~30 gzipped files with chrome. Worked fine with firefox.
In order to clean up users that have not been using ezpaarse for a long time.
the routes described in the Knowledge bases management section have changed
/parsers/status
/pkb/status lead to 404 or "Cannot PUT /pkb/status" and "Cannot PUT /parsers/status" messages.
There is now only one route available : /platforms/status on which we can GET or POST.
For example :
'curl -X GET -u "admin:password" http://localhost:59599/platforms/status'
yields the following response :
{"current":"b87ecaa","head":"b87ecaa","tag":"","from-head":"uptodate","from-tag":"uptodate","local-commits":false,"local-changes":false}
Travis didn't build the commit dedicated to the migration to node 6, so it's not clear if the problem has been introduced by node 6 or by the modules update. I can't reproduce the problem on my machine, so it's seems to be specific to the Travis environment.
@tporquet I have log file which contains 1500 lines and when i inserted that file into ezpaarse it only processed 500. How can I know which lines didn't get processed?
As of now there's an error in the logs but nothing in the report.
Docker builds are failing at Docker Hub, breaking docker-compose.yml which is pointing to an image that never built.
Are there any plans for ezpaarse to support the new R5 spec ?
No accent in english
Use global replace to be sure typos are all correted
So it could be easier to have quick feedback when a parser is changed or when a pkb is added or updated.
Moreover, ezpaarse unittest are huge and a refactoring like that could be a small step to improve the lisibility and usability of the unittest.
Hello,
Considering the following log file extract:
bounday-issue-nok.txt
*
Log extractor will fail as first line doesn't fit expected format (no event found)
Now let's move the first line at position 2
bounday-issue-ok.txt
Extraction works (116 events found)
This was encountered using predefined setting "France - Université Lille 2"
Best,
Antoine
Currently the rtypes from fields.json are pulled into the ezPaarse documentation. This is great and I support this mirroring, but I'm wondering if we can update the rtype descriptions in fields.json to support English descriptions. This would make it easier for English speaking users of ezPaarse. These are basic descriptions of the rtype fields, I may be able to get away with translating these programmatically and creating a pull request. Should I leave in the French rtype descriptions as well, or just convert it to English? I would like to find a way to describe rtypes in fields.json in both English and French if possible, unless it would make the fields.json file difficult to read.
If the server reaches the timeout (2min of inactivity by default) when uploading multiple files, multiparty fires an error in an internal stream which has no listener. This obviously result in a crash.
The incriminated line:
https://github.com/andrewrk/node-multiparty/blob/master/index.js#L197
Hi,
While processing the sample file available for download. I am getting the below error:
4003 : Line format was not recognized
Can you kindly assist. Thank you.
Under node v10, ezPAARSE startup hangs up after building the domains matching list. No exception thrown. Works fine with node v9.
When sending a file with an unknown format several times in a row, one request over two behaves correctly, and the other just freeze and doesn't produce any output on server side. But closing ezpaarse does terminate the request.
Reproduced in chrome and firefox.
The very first user is in user group instead of admin
That could be pretty useful for debugging purposes.
Right now the report is downloaded only if a code 200 is returned, making it difficult to debug automated jobs.
For bibliolog-like applications
Problème rencontré à Montpellier (l'instance ezpaarse tourne mais le client web affiche une page blanche) et reproduit en faisant :
--> refuse en disant "bower ESUDO Cannot be run with sudo"
Il faut forcer bower en utilisant l'option --allow-root
--> en tenir compte dans le make
Currently, we browse the crossref results once for each EC until we find a matching DOI, which means that we loop paquetSize / 2
times in average.
A better implementation would be to iterate once over the results to make a DOI<->Result
map, and then iterate once over the ECs and use this map to merge the results. This would greatly decrease the number of loops.
Sometimes the app crash on startup with EADDRINUSE althought the port is open.
When the request comes from the browser, regularly send something to keep the connection alive.
It would be nice if we could use the card ID to fill the form and generate a test file.
The recognized fields are annoying to fill and they are not used anymore
I think we should mention mongodb as soon as the readme.md, instead of only in https://github.com/ezpaarse-project/ezpaarse/blob/master/doc/quickstart.md (which is 3 clicks away).
This project downloads nvm (?) and uses node v8.6.0.
Is this a hard dependency, or can it run on more recent versions of node?
If it will work on more recent versions of node, how to configure this?
I see relevant-looking stuff in config.json
and bin/buildnode
but would appreciate advice.
As of now, only the first line is used to test the log format. But sometimes, the very first line can contain anomalies for some reason, causing the job to fail. ezPAARSE should try to parse the N first lines. This number would be customizable via a header (something like ezPAARSE-Max-Parse-Fails
) with a default value around 10-20.
make test
devrait lancer tous les tests
make test platformX
ne devrait lancer que les tests pour platformX
Hi,
thanks for this great tool, very useful!
We use Graylog https://www.graylog.org to visualize and ingest the date produced by ezpaarse.
Is it possible to specify format and fields for geolocation ?
Graylog produce natively a map widget based on latitude,longitude but it must be formatted on an unique field containing "latitude" + "," + "longitude"
It is possible to do this ?
Thanks in advance !
Johan
Université de Rennes 1
The crossref middleware checks the DOI pattern to crossref from firing an error. DOIs with a bad pattern should be available somewhere.
When running docker-compose up -d
, an error is thrown: Error response from daemon: manifest for ezpaarseproject/ezpaarse:3.0.5 not found
.
A quick look at https://hub.docker.com/r/ezpaarseproject/ezpaarse/tags doesn't show a 3.0.5
tag. Build does run when one of the older tagged builds is supplied to docker-compose.yml
It should be turned off by default
Some new headers are not listed
So it could give us info about where is located the user thus help us to customize our SAV answer.
Maybe the geoloc (town and country) could also be great.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.