Goobi.Presentation supports searches using SOLR. The SOLR query string needs special handling for these characters: +-&|!(){}[]^"*?:. Some of them cannot be used in the search field: If a user supplied string contains one of the characters +-!(){}[]^":, the search will fail and a TYPO3 error page "Oops, an error occurred!" is shown. With TYPO3 debugging enabled, call stack information is shown:

Uncaught TYPO3 Exception
#400: '400' Status: Bad Request (More information)

Apache_Solr_HttpTransportException thrown in file
/var/www/html/typo3conf/ext/dlf/lib/SolrPhpClient/Apache/Solr/Service.php in line 340.

24 Apache_Solr_Service::_sendRawGet("http://dlf:dlf@localhost:8080/solr/dlfCore2/select?wt=json&json.nl=map&q=%5C&start=0&rows=50000")


/var/www/html/typo3conf/ext/dlf/lib/SolrPhpClient/Apache/Solr/Service.php:

01170:   if ($method == self::METHOD_GET)
01171:   {

01172:    return $this->_sendRawGet($this->_searchUrl . $this->_queryDelimiter . $queryString);

01173:   }
01174:   else if ($method == self::METHOD_POST)


23 Apache_Solr_Service::search("\", 0, 50000, array)


/var/www/html/typo3conf/ext/dlf/common/class.tx_dlf_solr.php:

00281: 
00282:   // Perform search.

00283:   $results = $this->service->search((string) $query, 0, $this->limit, $this->params);

00284: 
00285:   $this->numberOfHits = count($results->response->docs);


22 tx_dlf_solr::search("\")


/var/www/html/typo3conf/ext/dlf/plugins/search/class.tx_dlf_search.php:

00554: 
00555:    // Perform search.

00556:    $results = $solr->search($query);

00557: 
00558:    $results->metadata = array (

The METS application profile Version 2.3 describes the physical document structure as follows:

The ORDERLABEL attribute can contain the pagination of the
individual page according to the original.
The LABEL attribute can include the foliation of the individual
page or the specification of record side and track number.

In Kitodo.Presentation, the function _getPhysicalPages() ignores the LABEL and sets "label" as ORDERLABEL if present.

https://github.com/kitodo/kitodo-presentation/blob/master/dlf/common/class.tx_dlf_document.php#L1930

I would suggest, to use LABEL if present, else ORDERLABEL if present, else empty string.

Extract from the OAI PMH Protocol Version 2.0:

3.1.1 HTTP Request Format

OAI-PMH requests must be submitted using either the HTTP GET or POST methods.
POST has the advantage of imposing no limitations on the length of arguments.
Repositories must support both the GET and POST methods.

Goobi.Presentation currently fails to support POST methods.
This was detected and reported as an error by the BASE OAI-PMH Validator (http://oval.base-search.net/).

In TYPO3 7 the class loading changed completely. The file ext_autoload.php is ignored but a section "autoload" in ext_emconf.php could be used to provide class locations and help the autoloader.

Even better is to provide a composer.json file with autoload information. This makes it possible to use Kitodo.Presentation in composer mode. Running TYPO3 in composer mode makes it easier for deployment and class loading should be even faster.

Read the blogpost of Helmut Hummel: Changes in class loading in TYPO3 7LTS
and COMPOSER AND TYPO3 7LTS.

This issue is related to #156.

The image position and zoom level ist stored in a session cookie. If the position values are outside the current viewport, the main image won't be seen.

This happens e.g. in case of different image dimensions in one digital object.

Example:
http://digital.slub-dresden.de/werkansicht/dlf/6439/

Currently the zoom slider works only if the zoom is changed step by step. If you move the slider back and forth multiple times, the results are erratic and not as expected.

In eID script for fetching fulltext files, the makeinstance call of class "language" causes an PHP exception. This class is not found in 7.6 like this.

Solution: remove the language label part of this script. It's not used anyway.

The repository itself contains the directory dlf in which the whole extension is located.

If you want to install it on your local TYPO3 installation you have to clone the repository directly inside the ext directory. Otherwise you end up with a structure like ext/dlf/dlf which does not work.

The drawback of cloning into ext is that any other file/extension that is in there would be in your local kitodo-presentation repository as well (as untracked files). That doesn’t make much sense since it should be encapsulated on its own.

You can potentially clone the repo somewhere and then use a symbolic link in the ext directory. But even then you’d need an additional (superfluous) directory hierarchy for the extension because the .git is a lever higher than the extension itself.

The solution would be to move everything out of the dlf directory into the root of the repository. That would be a drastic change to the repository structure so I don’t know the possible repercussions to that move.

Setting of fields inside palettes are not shown in current TCA configuration in TYPO3 7.6. This is due to the rewritten parser. But indeed, is seems to be a missconfiguration for long time.

By adding the reserved field name "--palette--", the palettes are shown in TYPO3 6.2 and 7.6

TCA Reference 6.2 types->showitem
TCA Reference 7.6 types->showitem

On indexing documents with fulltexts, the fulltext is added by parsing the ALTO-file by an XPath-expression in class.tx_dlf_alto.php.

This XPath expression doesn't respect a possible ComposedBlock around the TextBlock. In this example only "Heinz Niemann" gets into the index.

After adding Map features within the Page View Plugin the following JavaScript error appears:
"ReferenceError: layers is not defined" within https://github.com/kitodo/kitodo-presentation/blob/master/dlf/plugins/pageview/tx_dlf_pageview.js#L101-L103
The documentation of the method tells me that it is only for fulltext and image manipulation. The Map Features like "OverviewMap" and "ZoomPanel" are not mentioned. Is that just needless code at this place? The createControls_ method is by the way called within the init() method.
https://github.com/kitodo/kitodo-presentation/blob/master/dlf/plugins/pageview/tx_dlf_pageview.js#L305-L307
I comment out that piece of code and everthing is still working without JavaScript errors.

The display of page numbers in the table of contents does not work correctly in Goobi.Presentation.

How it is:
It seems as if all page numbers are displayed in the middle of the table of contents.

How it should be:
Either the page numbers should be displayed for each element (chapter, titele page, ...), or no page numbers should be displayed at all.

Provide the possibility to deselect each selected facet for backward facetting (if option is set it in the backend)

You can set the desired page by an integer value or by given the value of the ID attribute of the physical StructMap.

The plugins pagegrid and toc don't support the second case. Page = 1 is used as fallback instead.

As of version 3.0.1 of EXT:t3jquery (release 2015-06-29) usage of class tx_t3jquery is deprecated. This is logged to deprecation log.

Current t3jquery-documentation is outdated but the code is hostet on GitHub [1].

In Kitodo.Presentation a helper function in class.tx_dlf_helper.php adds jQuery by EXT:t3jquery to the page header. E.g. the pageview plugin is calling this function.

Solution: Use \T3Ext\T3jquery\Utility\T3jqueryUtility class instead.

[1] https://github.com/typo3-themes/t3jquery

In Kitodo.Presentation 2.0 with OpenLayers 3.0 it's not possible anymore to use "Right Click" to save the original image. Only a low quality version "Download.png" is offered.

Users of the previous version are used to this unspecified feature. Even though the filegroup "DOWNLOAD" is intended for this purpose.

This has to be discussed and implemented.

Page views with zoom don't work: neither the controls for zooming nor the page image are shown.

The Firefox web developer console shows these errors:

TypeError: P is undefined Class.js:64:3
TypeError: OpenLayers.Control.ZoomIn is not a constructor ZoomPanel.js:47:12

ZoomIn.js depends on Button.js which is missing.

Several Kitodo.Presentation plugins use the old pi_wrapInBaseClass() methold to wrap the plugin output inside a plugin specific <div>.
E.g. the output of the metadata plugin ends up in

<div class="tx-dlf-metadata">
...
</div>

This div is hard coded in pi_wrapInBaseClass() and cannot be removed by typoscript. This is a limitiation in special design cases. E.g. if you want to use several plugins in a <ul>-list.

Proposal: Remove pi_wrapInBaseClass() in all plugins and put the generated div inside the plugin templates for compatibility reasons.

Affected plugins are:

• metadata
• toolbox
• tool fulltext
• tool imagemanipulation
• tool pdf
• navigation
• audioplayer
• collection
• listview
• metadata
• pagegrid
• pageview
• search
• statistics
• toc
• validator

Not affected:

• newspaper
• feed
• oai

Link to pi_wrapInBaseClass() in TYPO3 API 6.2: http://api.typo3.org/typo3cms/62/html/_abstract_plugin_8php_source.html#l00787

I suggest adding automated static code analysis for pull requests and commits, for example using Scrutinizer. Examples:

• https://scrutinizer-ci.com/g/UB-Mannheim/kitodo-presentation/
• https://scrutinizer-ci.com/g/UB-Mannheim/PalMA/

Example of PR with Scrutinizer:

• UB-Mannheim/PalMA#31

When searching for text with special characters, that text is modified in the search field:

• Enter search string with special characters, e.g. a:b
• The search is executed, and the search field now contains a:b instead of a:b

So pressing the search button a 2nd time will give a different result.

As mentioned in issue#21 search truncation with * does not work any longer (since fix#34).
Using '*' still works - but 'fragment*' does not work.
This kind of search is very important for our users.

If Kitodo.Presentation is installed/configured to run on a subdirectory of the Webserver the following line https://github.com/kitodo/kitodo-presentation/blob/master/dlf/plugins/pageview/class.tx_dlf_pageview.php#L185
creates an invalid URL. I think it is possible to avoid this behaviour just by deleting the leading '/'. Am I right or will then arise other problems?

The current version of TYPO3 is 7 LTS. It is recommended for new projects and will get security updates until November 2018.

The current Goobi.Presentation supports TYPO3 6.2 LTS which gets maintenance until March 2017. According to dlf/ext_emconf.php, the exact range of versions supported by Goobi.Presentation is from 4.5.0 up to 6.2.99.

As all TYPO3 versions before TYPO3 6.2 LTS are completely unsupported, I suggest dropping support for those versions in the next version of Goobi.Presentation, too. This includes TYPO3 4.5 LTS, 6.0, 6.1 and others.

Goobi.Presentation currently fails to work with TYPO3 7 LTS because some deprecated interfaces were removed (maybe there are more reasons). There is an extension "compatibility6" which provides some backwards-compatibility, but I don't know whether it is sufficient for Goobi.Presentation, and people suggest not to use it because it is bad for the performance of TYPO3. Therefore code changes for Goobi.Presentation are needed to fully support TYPO3 7 LTS. These changes will be much easier if we can drop support for anything before TYPO3 6.2 LTS.

The check to see if CORS is enabled in dlf/plugins/pageview/tx_dlf_utils.js uses an AJAX-Requets to see if it is possible to fetch certain resources. This test sometimes succeeds although the corresponding request was not completed successfully.

.error(function(data, type) { if (type === 'error') { response = false; } });

In the error-case there is a specific check for the return type of 'error'. The jQuery-Documentation does contain more than this value when an error occurred (see: http://api.jquery.com/jQuery.ajax/).

"success", "notmodified", "nocontent", "error", "timeout", "abort", or "parsererror"

So, if the request has been blocked and was therefore being aborted the error-callback in the check does not set the response-value to false. Eliminating the if-clause should prevent this false-positive. We are not interested why the request fails specifically, because at that point we are not able to do anything about it anyway.

The TYPO3 project re-worked the copyright/license headers, see https://forge.typo3.org/issues/59535.
I suggest that Goobi.Presentation should do something similar (for the same reasons).

See also the discussion on https://forge.typo3.org/issues/56776.

Several fixes which are needed for TYPO3 6.2 are missing in v1.3.0.
We should have a working version in the TYPO3 extension repository.

The current used build of OpenLayers 3 in Kitodo.Presentation the FullScreen control is not included. It should be included in ol3-dlf.json and a new build should be added.

Of course this control must be added in pageview plugin as well to be usable.

Submitting an empty search field leads to the following TYPO3 exception:
#400: '400' Status: Bad Request (More information)

Apache_Solr_HttpTransportException thrown in file
[...]/typo3conf/ext/solr/Resources/Private/Php/SolrPhpClient/Apache/Solr/Service.php in line 611.

--> obviously the SolrPhpClient library of EXT:solr (3.3.1) is used and not the one of Goobi.Presenation

If the location of the METS-file contains special characters as "Umlaute", the METS-file is not loaded and all plugin output is empty.

Example: http://digital.slub-dresden.de/fileadmin/data/330699008/heßFleh_330699008_mets.xml

If you enable the developer log, you receive the error message

[tx_dlf_document->load(http://digital.slub-dresden.de/fileadmin/data/322151767/meißunsem_322151767_mets.xml)] Invalid file location "http://digital.slub-dresden.de/fileadmin/data/322151767/meißunsem_322151767_mets.xml" for document loading

This is caused by the validation inside GeneralUtility::isValidUrl() used in tx_dlf_document::load().

The location should be (raw)urlencoded after reading from the database.

To guide users in citations it would be helpful to extend the generated pdfs with a preceding page that displays available metadata in table format and a recommended citation (e.g. "SLUB Dresden, Mscr.Dresd.M.32, digital.slub-dresden.de/id272362328 (CC-BY-SA 4.0)")

Feature request at meeting SUB HH and SLUB DD, April 29th, 2015.

Problem description:
From several persons, criticism regarding the display of the title of periodical issues in result lists arose. The problem is the display of [kein Titel] instead of the periodical's title [1].

In long result lists, this becaomes quite confusing. The RSS-Feed faced the similar problem, which was solved by Goobi.Presentation CE 1.1.b2 [2]. In the release notes, it seems as if the problem should be solvable in the presentation, too:

"- Für Bände ohne eigenen Titel kann nun wahlweise der Titel der übergeordneten Einheit in eckigen Klammern angezeigt werden. Das ist beispielsweise in den RSS-Feeds interessant, um dort "leere" Einträge zu vermeiden, kann aber auch im Metadaten-Plugin verwendet werden. Das Feature lässt sich über die Plugin-Konfiguration individuell ein- und ausschalten."

Solution description:
In the result lists, periodical issues should be labeled with the periodical's title, except when the issue has an own title.

[1]

[2]

TYPO3 switched to PSR-2 (see issue 70515). PSR-2 is a standard (Coding Style Guide) recommended for PHP code.

I suggest using this standard for Goobi.Presentation, too. A related issue is #65.

One major change is replacing tab characters by spaces. Obviously this is a general trend for free software projects.

See also the German article in Wikipedia.

In the toolbox plugin, multiple tools may be included. The template file of the toolbox may be set by typoscript or flexform. If none is set, the default template will be used.

The tools may have their own template files which may be set by typoscript or flexform. If none is set, the default template should be used.

This order will fail, if a custom toolbox template is set as the configuration will be merged:

// Merge configuration with conf array of toolbox.
$this->conf = tx_dlf_helper::array_merge_recursive_overrule($this->cObj->data['conf'], $this->conf);

Finally, all tools use the toolbox template instead of their own.

Idea to fix this:

• Rename the configuration option in the tools. e.g. toolTemplateFile instead templateFile
• others?

A Workaround is possible, by setting the tools template file by typoscript:

plugin.tx_dlf_toolsPdf.templateFile = EXT:dlf/plugins/toolbox/tools/pdf/template.tmpl

This is not a bug report, but a suggestion for a new feature.

When a user accesses a website using https, images and other files are usually still fetched with http instead of https because the METS/MODS files typically contain URLs using http.

Example: https://digital.slub-dresden.de/werkansicht/dlf/98053/1/0/

Web browsers report for that page something like "This connection is insecure. Parts of this page are not secure." which is correct.

There are several possible solutions to get a 100 % secure connection.

1. Use URLs with https in the METS/MODS files. This requires touching all existing METS/MODS files. In addition, those files will then always be fetched using https, even if the user accesses the website with http.
2. Use HTTP Strict Transport Security. Then http is no longer possible.
3. Modify the URLs read from METS/MODS before using them.
4. Use relative URLs in the METS/MODS files. The URLs for all files referred to in a METS/MODS file would be relative to the URL of the METS/MODs file itself.

Solution 3 would need code in Kitodo.Presentation which fixes the URLs, for example by doing a simple string substitution. For the example, replacing http://digital.slub-dresden.de/ by a simple / would be sufficient.

Solution 4 might violate standards and break existing applications like the DFG viewer.

The old tca.php called dynamicConfigFile from ext_tables.php is deprecated and will be removed in TYPO3 8. It has several disadvantes like no caching, difficult to read, split one configuration to different files.

See this blogposts for background and solutions:

https://usetypo3.com/good-practices-in-extensions.html
https://scripting-base.de/blog/cleaning-the-hood-tca.html

The format string %u within function strftime is not available on Windows. Should be replaced by %w. https://github.com/kitodo/kitodo-presentation/blob/master/dlf/plugins/newspaper/class.tx_dlf_newspaper.php#L222-L236

The following error is thrown in class.ext_update.php:

ERROR: Unknown column 'tx_dlf_metadata.xpath' in 'where clause'

The field 'xpath' doesn't exists. @sebastian-meyer: can we just remove this part of the query?

Background: This query is called on every access of the Extension Manager as of TYOPO3 6.2.12. The EM checks if the available update scripts have some update tasks to do. If not, the update icon is not shown.

With change "Breaking: #70132 - FormEngine custom functions" , the way itemProcFunc is used in Kitodo.Presentation plugins doesn't work anymore in TYPO3 7.6. It is not possible to access "pi_flexform" anymore from user function.

User functions are used, to get records only of the configured storage folder ("pages" / startingpoint).

This is a common problem with TYPO3 plugins. This could be solved by using foreign_table and foreign_table_where. But this way, we cannot access configured fields (e.g. "pages") of current flexform. That's why itemProcFunc has been used so far.

Further investigation is needed.

This is a blocker for using all features of Kitodo.Presentation with TYPO3 7.6

Currently it is not possible to translate the labels like "Title" of the tx_dlf_metadata table or "Strukturtyp" of the tx_dlf_structures.

Usually you copy a table row and insert it again on the same page. You edit the copy and change the language to the overlay language AND you select the default label.
If this is done, the Metadata-entry is displayed twice: One time with the default label. The second time with the dummy index_name ("title0).

Pointers to PDF files (mets:fptr) can be contained in the physical or in the logical structural map (mets:structMap).

The DFG viewer and Goobi.Presentation offer a download link for the whole work, no matter whether the PDF file pointer is part of the physical or of the logical structural map.

They don't offer a download link if the file pointer is only part of the logical structural map (<mets:structMap TYPE="LOGICAL">). This is the standard used by Visual Library, see http://digi.bib.uni-mannheim.de/ubmadesb/oai/?verb=GetRecord&metadataPrefix=mets&identifier=17198 for an example.

Update 2015-04-15:

File pointers in the logical structural map are typically used for PDF files which cover a whole chapter,
but sometimes also only a single page, so maybe depending on the page range a "Download single page" or a "Download part of work" might be adequate.

This would be a (useful) new feature for both DFG viewer and Goobi.Presentation, not a bug fix.

The pagegrid plugin offers thumbnails of neighbour pages. A pagebrowser navigation let' you switch to the next bunch of thumbnails.

On using this pagebrowser navigation, the selected image get's lost and image number 1 is shown.

The desired behaviour would be:

• page 1: thumbnails 1 to 20 (depending on setting) --> tx_dlf[page]=1
• page 2: thumbnails 21 to 30 --> tx_dlf[page]=21
  ...

The TCA definition in tca.php is not compatible with TYPO3 7.6.

1. All select types need to define a renderType like "selectBox"
2. The config "form_type" has gone. This was used to render input fields as read-only. Now these fields are rendered read-write

This TYPO3 extension should include an OpenOffice manual.

Otherwise TYPO3 extension manager / Get Extensions / (filter dlf) / Show all versions of dlf / Manual Read online fails with this dead link: http://docs.typo3.org/typo3cms/extensions/dlf/

Maybe it is sufficient to add a rudimentary dlf/doc/manual.sxw which refers to the available websites.

If using TYPO3 behind a HTTP-Proxy, Kitodo.Presentation will fail to load the METS-files. The global proxy settings of TYPO3 won't be applied as the pure PHP file_get_contents() method is used.

TYPO3 GeneralUtilities provides the method getUrl() which is available in 6.2, 7.8 and 8.x. This ensures the usage of given proxy settings and may use curl inside. The result should be the same.

Files to be adapted:

• plugins/pageview/class.tx_dlf_fulltext_eid.php
• common/class.tx_dlf_document.php
• common/class.tx_dlf_indexing.php
• hooks/class.tx_dlf_em.php
• hooks/class.tx_dlf_tcemain.php

The simple search returns always an empty result if option "fulltext" enabled and the search is limited to current collection.

Example:
http://digital.slub-dresden.de/kollektionen/73/

Search for e.g. "Schmidt Pieschen" which is found without limiting to collection on page http://digital.slub-dresden.de/kollektionen/

Importing data from Visual Library fails with out-of-memory:

/var/lib/typo3-dummy/typo3/cli_dispatch.phpsh dlf index -pid 3 -core 1 -doc "http://digi.bib.uni-mannheim.de/oai/?verb=GetRecord&metadataPrefix=mets&identifier=17208"

The XML data mets:mptr causes an endless recursion because it refers to the same URL.

<mets:mptr LOCTYPE="URL" xlink:href="http://digi.bib.uni-mannheim.de/oai/?verb=GetRecord&metadataPrefix=mets&identifier=17208"/>

Goobi.Presentation CE 1.3.0 which is also the latest version available from TYPO3 Extension Repository breaks the TYPO3 extension manager when it is installed. The breakage is reported in the Apache2 error.log:

PHP Fatal error:  require_once(): Failed opening required 'common/class.tx_dlf_helper.php' (include_path='/var/www/html/typo3/contrib/pear/:.:/usr/share/php:/usr/share/pear') in /var/www/html/typo3temp/ExtensionManager/UpdateScripts/ext_updatef6b292186acec66f73619c4c401cd408.php on line 25, referer: https://localhost/typo3/backend.php

This issue was already fixed in commit 8cf6ded, but there is currently no released version which includes that fix.

As a workaround, it is possible to edit (or remove) the buggy file typo3temp/ExtensionManager/UpdateScripts/ext_update*.php.

The box with extended metadata cannot be opened in Kitodo.Presentation, when the button to open the box covers the "navigation area" (see image below).

This happens when the metadata preview beneath the title is too long. Until now, the only known way to open the metadata box is to scale down or up the browser's content until the button moves out of the navigation area.

It would be more usable, if either

• a line break is used if the metadata preview covers the navigation area (metadata in two lines) or
• if the lenghts of the metadata preview is limited in a way that it never covers the navigation area (not all metadata is shown)

Example: http://digital.slub-dresden.de/id274276038

The user selection of the radio-buttons "Metadata" vs. "Fulltext" is lost on the result listing.

The query string is not prefilled in the search field after search. This should be the desired behaviour as "well known search engines" work like this - and the SLUB-cataloge as well.

While indexing some parts of our collection, we noticed a strange phenomenon: we got lots of entries in the table of libraries ("Mandanten" in German TYPO3, table tx_dlf_libraries in MySQL).

The relevant code in function tx_dlf_document.save looks for the correct library (columns pid and index_name in table tx_dlf_libraries) and inserts a new library if it did not find one.

Usually it expects an empty index_name (Are other value supported, too? Is $metadata['owner'][0] ever set?). If a new library is inserted, it gets the correct pid, but an index_name which is null (!= empty string). When the same document is indexed again, that null index_name won't match the DB search criteria, so one more library is inserted. This can be repeated ad infinitum.