orapi's People
orapi's Issues
Error in series lookup
Selecting certain series leads to an error on the Series overview page.
Reason: Error in lookup function to get the events of a series
https://github.com/WolfgangFahl/ConferenceCorpus/blob/727afb6f5c9b2e3807260a6161a27531b2f77dd2/corpus/event.py#L489
uses two different lookup tables to get the series one is by acronym and one by pageTitle due to the missing acronym issue both tables deviate causing a lookup error
Fix loading of custom or backup over ConferenceCorpus
ConferenceCorpus currently is focused on loading predefined datasources. With the different backups of openresearch we need to load the or backup from given or-wikis such as orfixed without messing up the CC database
keep uploaded excel files for further reference
keep a history of excel sheets being used and allow to download them later (for admins : all - for users: own)
Validation phase example check Homepage
Cases:
- Homepage exists
- Homepage exists but wrong content (define wrong e.g. acronym or title can not be found)
- Homepage does not exist
- check internet archive at time of creation of record or start date of event
Extend csv upload with publish entities feature
After updating the cache the changes should be published to the wiki. This functionality might be integrated into ConferenceCorpus.
k-cap 2015
according to https://www.k-cap.org/kcap15/index.html the location is Pallisades, NY, not Pallisades TX
Migrate Wiki API Wrapper Classes to py-3rdparty-mediawiki
Move the following classes to py-3rdparty-mediawiki and adjust orapi accordingly to function as before.
Line 8 in 9f8eb19
Line 76 in 9f8eb19
Line 99 in 9f8eb19
Proper MenuBar
Switch to MenuBar Widget for
- proper handling of basedUrls
- highlighting of active page
iswc upload fails
File "/home/th/.local/lib/python3.8/site-packages/corpus/smw/topic.py", line 112, in pushToWiki
self.updateWikiText(overwrite=overwrite)
File "/home/th/.local/lib/python3.8/site-packages/corpus/smw/topic.py", line 87, in updateWikiText
self.wikiFile.updateTemplate(self.entity.templateName, wikiSonRecord, overwrite=overwrite)
File "/home/th/.local/lib/python3.8/site-packages/wikifile/wikiFile.py", line 245, in updateTemplate
WikiFile.update_arguments(template, args, overwrite, prettify)
File "/home/th/.local/lib/python3.8/site-packages/wikifile/wikiFile.py", line 311, in update_arguments
template.set_arg(key, value+postfix, preserve_spacing=False)
TypeError: unsupported operand type(s) for +: 'int' and 'str
ISWC-Updated.ods
'
DropZoneField not dependent on BaseUrl
Line 419 in 0b1c004
it should be possible to start with an empty series
if no events are in a series derive column headers from some sample default series instead.
Add CI
- add CI
- add server startup to test setup
- setup wikidocker to test the upload
date handling e.g. conversion to ISO-Dates
- https://confident.dbis.rwth-aachen.de/or/index.php?title=ICSE
- https://confident.dbis.rwth-aachen.de/orapi/api/series/ICSE?source=orclone
is an example for a conference series with dates that need some love.
It would be good if the date enhancing would be a standard enhancing feature to make sure the dates show up properly in
the spreadsheet.
E.g.
- 23/04/1995 -> 1995-04-23
- 4/06/2000 -> 2000-06-04
- 2021/05/22 -> 2021-05-22
orapi missing in List_of_OsProjects so tickets can not be referenced
Excel support
This is a method for creating excel compatible output ... Could we implement that?
df=pd.DataFrame(lod)
writer = pd.ExcelWriter(f'{eventSeriesAcronym}.xlsx', engine='xlsxwriter')
df.to_excel(writer, index = False)
writer.save()
None handling
https://confident.dbis.rwth-aachen.de/or/index.php?title=ACSAC
has quite a few invalid "None" entries at different positions even in a URL. This behavior should be avoided.
Upload doesn't seem to have an effect - feedback for upload is hidden
Publish series does not publish series
created new items but these were not uploaded to orclone.
see https://confident.dbis.rwth-aachen.de/or/index.php?title=ICADL
Location publishing omitts Region
Location information are extracted from ["Country", "Region", "City"] but region information is stored under the template param State.
Line 281 in 6c3a47b
Add CI tests
Test environment:
- mock pushing a few event pages into the target wiki
Tests:
- test csv download from source wiki
- test mocked csv upload โ test if changes are applied
- add option to start orapi without user verification (for tests)
internet archive
shows links and different symbol
typo upldated
Dryrun! (not upldated)
Add wikidata Q-Ids to location entities
City, Region and Country should also have a corresponding CityWikidataId, ... column
red links in validation phase
for non existing pages show a red link instead of a blue link
Add commandline interface to start orapi
required params:
- wikiId: id of the wiki for which the api should be provided
- wikiTextPath: path to the location of the wikiMarkupFiles to init CC
- might switch to live extraction (directly query pages from the wiki)
Upload does not work with message "string does not contain a date"
This message is not helpful since it does not say which string is the problem. See the attached excel file that the date columns are purposely undefined in some cases which shouldn't be a problem
WISE.xlsx
Convert dates to iso-format
After extraction of date values they should be converted to ISO-format
Complete Refactoring to pyOnlineSpreadSheetEditing
-
SpreadSheed classes moved to pyOnlineSpreadSheetEditing migrate the use of those classes to this project in orapi
Line 16 in 9f8eb19
-
Integrate the Enhancement and Validation pipeline of TableEditing to orapi
wikidata synchronization support
see also WolfgangFahl/ProceedingsTitleParser#62
it should be possible to synchronize a wikidata series with an OpenResearch series.
Examples see: dicussions in
- SmartDataAnalytics/OpenResearch#208
- SmartDataAnalytics/OpenResearch#192
- SmartDataAnalytics/OpenResearch#143
- SmartDataAnalytics/OpenResearch#162
- SmartDataAnalytics/OpenResearch#127
- SmartDataAnalytics/OpenResearch#181
- SmartDataAnalytics/OpenResearch#215
- SmartDataAnalytics/OpenResearch#241
- SmartDataAnalytics/OpenResearch#257
Location type is not set / included in the reduced result format
Integration tests with docker
- ls an integration of wikidocker into the CI useful/necessary?
Personally i think this should only be done if there is problem in the interaction with the components. Otherwise each component should do it's own reliability tests.
Publish progress as links to source/preview and target wiki
Currently Publish completed is shown and a json rep which is not sorted.
The list and the upload should be sorted by year / ordinal going from the oldest to the newest so that the upload is finished when the most recent event is uploaded.
Two - Three links should be displayed.
- source wiki
- preview wiki (optional and currently probably useful)
- target wiki (important)
There should also be a delta view option / display giving a hint on how big the chances have been or inspecting these. For the target and preview wiki this can easily be done with a link to the history page of the wiki involved. See original Y-Principle generator written in Java for a potential UI for this.
Add Metadata to excel exports
i.e. the excel sheets should contain metadata about the used properties e.g. a mapping between column names and names of the properties in potential target datasets such as openresearch, wikidata, etc.
- display of the metadata at the frontend?
K-CAP export - garbled columns issue
Download of https://confident.dbis.rwth-aachen.de/orfixed/index.php?title=K-CAP and opening with Excel leads to some gabled columns:
abstract Tableediting
enhancing, checking and spreadsheet editing should have a testable and useable API
Adjust Event series to new naming scheme
Line 127 in 08fd11a
Change to IsA::EventSeries
Spreadsheet editing allow deleting values
If a -
is in a cell delete the corresponding property value
K-CAP red links in locations
https://confident.dbis.rwth-aachen.de/or/index.php?title=K-CAP has red links - will republishing fix this?
Why do the red links not show in the https://confident.dbis.rwth-aachen.de/or/index.php?title=List_of_Locations?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.