tholzheim / orapi Goto Github PK

Selecting certain series leads to an error on the Series overview page.

Reason: Error in lookup function to get the events of a series
https://github.com/WolfgangFahl/ConferenceCorpus/blob/727afb6f5c9b2e3807260a6161a27531b2f77dd2/corpus/event.py#L489
uses two different lookup tables to get the series one is by acronym and one by pageTitle due to the missing acronym issue both tables deviate causing a lookup error

ConferenceCorpus currently is focused on loading predefined datasources. With the different backups of openresearch we need to load the or backup from given or-wikis such as orfixed without messing up the CC database

keep a history of excel sheets being used and allow to download them later (for admins : all - for users: own)

Cases:

• Homepage exists
• Homepage exists but wrong content (define wrong e.g. acronym or title can not be found)
• Homepage does not exist
  • check internet archive at time of creation of record or start date of event

After updating the cache the changes should be published to the wiki. This functionality might be integrated into ConferenceCorpus.

according to https://www.k-cap.org/kcap15/index.html the location is Pallisades, NY, not Pallisades TX

Move the following classes to py-3rdparty-mediawiki and adjust orapi accordingly to function as before.

Switch to MenuBar Widget for

• proper handling of basedUrls
• highlighting of active page

File "/home/th/.local/lib/python3.8/site-packages/corpus/smw/topic.py", line 112, in pushToWiki
self.updateWikiText(overwrite=overwrite)
File "/home/th/.local/lib/python3.8/site-packages/corpus/smw/topic.py", line 87, in updateWikiText
self.wikiFile.updateTemplate(self.entity.templateName, wikiSonRecord, overwrite=overwrite)
File "/home/th/.local/lib/python3.8/site-packages/wikifile/wikiFile.py", line 245, in updateTemplate
WikiFile.update_arguments(template, args, overwrite, prettify)
File "/home/th/.local/lib/python3.8/site-packages/wikifile/wikiFile.py", line 311, in update_arguments
template.set_arg(key, value+postfix, preserve_spacing=False)
TypeError: unsupported operand type(s) for +: 'int' and 'str
ISWC-Updated.ods
'

if no events are in a series derive column headers from some sample default series instead.

• add CI
• add server startup to test setup
• setup wikidocker to test the upload

• https://confident.dbis.rwth-aachen.de/or/index.php?title=ICSE
• https://confident.dbis.rwth-aachen.de/orapi/api/series/ICSE?source=orclone

is an example for a conference series with dates that need some love.
It would be good if the date enhancing would be a standard enhancing feature to make sure the dates show up properly in
the spreadsheet.

E.g.

• 23/04/1995 -> 1995-04-23
• 4/06/2000 -> 2000-06-04
• 2021/05/22 -> 2021-05-22

https://rq.bitplan.com/index.php/List_of_OsProjects

This is a method for creating excel compatible output ... Could we implement that?

df=pd.DataFrame(lod)
writer = pd.ExcelWriter(f'{eventSeriesAcronym}.xlsx', engine='xlsxwriter')
df.to_excel(writer, index = False)
writer.save()

https://confident.dbis.rwth-aachen.de/or/index.php?title=ACSAC
has quite a few invalid "None" entries at different positions even in a URL. This behavior should be avoided.

ICADL.xlsx

ICADL.xlsx

created new items but these were not uploaded to orclone.
see https://confident.dbis.rwth-aachen.de/or/index.php?title=ICADL

Location information are extracted from ["Country", "Region", "City"] but region information is stored under the template param State.

Test environment:

• mock pushing a few event pages into the target wiki

Tests:

• test csv download from source wiki
• test mocked csv upload → test if changes are applied
  • add option to start orapi without user verification (for tests)

shows links and different symbol

Dryrun! (not upldated)

City, Region and Country should also have a corresponding CityWikidataId, ... column

for non existing pages show a red link instead of a blue link

required params:

• wikiId: id of the wiki for which the api should be provided
• wikiTextPath: path to the location of the wikiMarkupFiles to init CC
  • might switch to live extraction (directly query pages from the wiki)

This message is not helpful since it does not say which string is the problem. See the attached excel file that the date columns are purposely undefined in some cases which shouldn't be a problem
WISE.xlsx

After extraction of date values they should be converted to ISO-format

• SpreadSheed classes moved to pyOnlineSpreadSheetEditing migrate the use of those classes to this project in orapi
  

  orapi/orapi/spreadsheet.py

  Line 16 in 9f8eb19

  class SpreadSheetType(Enum):

• Integrate the Enhancement and Validation pipeline of TableEditing to orapi

see also WolfgangFahl/ProceedingsTitleParser#62

it should be possible to synchronize a wikidata series with an OpenResearch series.

Examples see: dicussions in

• SmartDataAnalytics/OpenResearch#208
• SmartDataAnalytics/OpenResearch#192
• SmartDataAnalytics/OpenResearch#143
• SmartDataAnalytics/OpenResearch#162
• SmartDataAnalytics/OpenResearch#127
• SmartDataAnalytics/OpenResearch#181
• SmartDataAnalytics/OpenResearch#215
• SmartDataAnalytics/OpenResearch#241
• SmartDataAnalytics/OpenResearch#257

see https://confident.dbis.rwth-aachen.de/orapi/location/US/CA?reduce

• ls an integration of wikidocker into the CI useful/necessary?
  Personally i think this should only be done if there is problem in the interaction with the components. Otherwise each component should do it's own reliability tests.

Currently Publish completed is shown and a json rep which is not sorted.

The list and the upload should be sorted by year / ordinal going from the oldest to the newest so that the upload is finished when the most recent event is uploaded.

Two - Three links should be displayed.

• source wiki
• preview wiki (optional and currently probably useful)
• target wiki (important)
  There should also be a delta view option / display giving a hint on how big the chances have been or inspecting these. For the target and preview wiki this can easily be done with a link to the history page of the wiki involved. See original Y-Principle generator written in Java for a potential UI for this.

i.e. the excel sheets should contain metadata about the used properties e.g. a mapping between column names and names of the properties in potential target datasets such as openresearch, wikidata, etc.

• display of the metadata at the frontend?

Download of https://confident.dbis.rwth-aachen.de/orfixed/index.php?title=K-CAP and opening with Excel leads to some gabled columns:

enhancing, checking and spreadsheet editing should have a testable and useable API

Change to IsA::EventSeries

If a - is in a cell delete the corresponding property value

https://confident.dbis.rwth-aachen.de/or/index.php?title=K-CAP has red links - will republishing fix this?
Why do the red links not show in the https://confident.dbis.rwth-aachen.de/or/index.php?title=List_of_Locations?