islandora / crayfish Goto Github PK

View Code? Open in Web Editor NEW

8.0 35.0 29.0 2.2 MB

A collection of Islandora microservices, lovingly known as Crayfish.

License: MIT License

PHP 99.89% Shell 0.11%

islandora

crayfish's Introduction

Crayfish

Introduction

A collection of Islandora 8 microservices, lovingly known as Crayfish. Some of the microservices are built specifically for use with a Fedora Repository and API-X, while others are just for general use within Islandora 8.

Requirements

The minimum requirements for any microservice are

PHP 7.3+
Composer

Many microservices have extra installation requirements. Please see the README of each microservice for additional details.

Services

Crayfish contains the following services

CrayFits: FITS as a microservice.
Homarus: FFmpeg as a microservice.
Houdini: ImageMagick as a microservice.
Hypercube: Tesseract as a microservice.
Milliner: Microservice that converts Drupal entities into Fedora resources.
Recast: Microservice that remaps Drupal URIs to add Fedora to Fedora links based on associated Drupal URIs in RDF.

See the individual services for more information on their endpoints.

Security

Crayfish microservices use JWTs to handle authentication like the rest of the Islandora 8. It is disabled by default. To enable, set security enabled to true in cfg/cfg.php for any microservice. You can also set the path to an xml configuration file for security a la Syn with the security config parameter.

Development

If you would like to contribute, please get involved by attending our weekly Tech Call. We love to hear from you!

If you would like to contribute code to the project, you need to be covered by an Islandora Foundation Contributor License Agreement or Corporate Contributor License Agreement. Please see the Contributors pages on Islandora.ca for more information.

We recommend using the islandora-playbook to get started. If you want to pull down the submodules for development, don't forget to run git submodule update --init --recursive after cloning.

Maintainers

Jonathan Green

This project has been sponsored by:

American Philosophical Society
Born-Digital
discoverygarden inc.
LYRASIS
McMaster University
PALS
University of Limerick
University of Manitoba
UPEI
Simon Fraser University
York University

License

MIT

crayfish's People

Contributors

Stargazers

Watchers

crayfish's Issues

[BUG] Hypercube broken pipe error with PHP deprecation error on PHP 8.1+ISLE

What steps does it take to reproduce the issue?

Create a site with the ISLE Site Template, which runs on PHP 8.1.

With Islandora Text Extraction enabled, verify that a media derivative context is configured with a reaction that carries out a text extraction action.

This is configured in the starter site by default.

Create a Repository Item node of type Paged Content, then add one or more members with model Page.

Add media to those pages of type File and upload a TIFF with embedded text.

When does this issue occur?
Which page does it occur on?
What happens?

No Extracted Text media is created.

The Hypercube logs show the following errors:


isle-site-template-hypercube-dev-1  | 172.19.0.15 - - [25/Sep/2023:20:23:09 +0000] "GET / HTTP/1.1" 500 84265 "-" "Apache-HttpClient/4.5.13 (Java/17.0.8)" "-"
isle-site-template-hypercube-dev-1  | [2023-09-25 20:23:10] request.INFO: Matched route "hypercube_convert". {"route":"hypercube_convert","route_parameters":{"_route":"hypercube_convert","_controller":"App\\Islandora\\Hypercube\\Controller\\HypercubeController::ocr"},"request_uri":"http://hypercube:8000/","method":"GET"} []
isle-site-template-hypercube-dev-1  | [2023-09-25 20:23:10] security.INFO: Guard authentication successful! {"token":"[object] (Symfony\\Component\\Security\\Guard\\Token\\PostAuthenticationGuardToken: PostAuthenticationGuardToken(user=\"admin\", authenticated=true, roles=\"authenticated, fedoraadmin\"))","authenticator":"Islandora\\Crayfish\\Commons\\Syn\\JwtAuthenticator"} []
isle-site-template-hypercube-dev-1  | [2023-09-25 20:23:10] php.INFO: Deprecated: fopen(): Passing null to parameter #3 ($use_include_path) of type bool is deprecated {"exception":"[object] (ErrorException(code: 0): Deprecated: fopen(): Passing null to parameter #3 ($use_include_path) of type bool is deprecated at /var/www/crayfish/Hypercube/vendor/guzzlehttp/psr7/src/StreamWrapper.php:45)"} []
isle-site-template-hypercube-dev-1  | [2023-09-25 20:23:10] request.CRITICAL: Uncaught PHP Exception ErrorException: "Notice: fwrite(): Write of 1024 bytes failed with errno=32 Broken pipe" at /var/www/crayfish/Hypercube/vendor/islandora/crayfish-commons/CmdExecuteService.php line 82 {"exception":"[object] (ErrorException(code: 0): Notice: fwrite(): Write of 1024 bytes failed with errno=32 Broken pipe at /var/www/crayfish/Hypercube/vendor/islandora/crayfish-commons/CmdExecuteService.php:82)"} []
isle-site-template-hypercube-dev-1  | 172.19.0.15 - - [25/Sep/2023:20:23:10 +0000] "GET / HTTP/1.1" 500 84265 "-" "Apache-HttpClient/4.5.13 (Java/17.0.8)" "-"
   ^[^[

To whom does it occur (anonymous visitor, editor, administrator)?

Editor,, Administrator

What did you expect to happen?

Text extractions are successfully executed.

Which version of Islandora are you using?

ISLE Site Template run via automatic setup script on ~ Sept 21, 2023.

Crayfish container version:
"Image": "sha256:399ac4c2df9f71f0609b0a115b34072e964bbbda2f947a4a442c52bddbc71da2",
"ResolvConfPath": "/var/lib/docker/containers/6dfe5dd791d1d89d14e32fbc2722a253f2381b9779708051a490dc7de21b1c29/resolv.con

Any related open or closed issues to this bug report?

Screenshots:

[BUG] - Uncaught Exception: Unrecognized option "syn_config" under "crayfish_commons"

What steps does it take to reproduce the issue?
Follow the instructions here to set up the action and reaction for hocr generation.

NOTICE: PHP message: 2024-01-30T19:13:22+00:00 [critical] Uncaught Exception: Unrecognized option "syn_config" under "crayfish_commons". Available options are "apix_middleware_enabled", "fedora_base_uri".
2024/01/30 19:13:22 [error] 606#606: *135 FastCGI sent in stderr: "PHP message: 2024-01-30T19:13:22+00:00 [critical] Uncaught Exception: Unrecognized option "syn_config" under "crayfish_commons". Available options are "apix_middleware_enabled", "fedora_base_uri"" while reading upstream, client: 172.22.0.6, server: , request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/var/run/php-fpm82/php-fpm82.sock:", host: "hypercube:8000"
172.22.0.6 - - [30/Jan/2024:19:13:22 +0000] "GET / HTTP/1.1" 500 137332 "-" "Apache-HttpClient/4.5.13 (Java/17.0.9)" "-"

Related Alpaca trace

org.apache.camel.http.common.HttpOperationFailedException: HTTP operation failed invoking http://hypercube:8000/ with statusCode: 500
	at org.apache.camel.component.http.HttpProducer.populateHttpOperationFailedException(HttpProducer.java:362)
	at org.apache.camel.component.http.HttpProducer.process(HttpProducer.java:246)
	at org.apache.camel.support.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:66)
	at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:169)
	at org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$RedeliveryTask.redeliver(RedeliveryErrorHandler.java:779)
	at org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(DefaultReactiveExecutor.java:148)
	at org.apache.camel.impl.engine.DefaultReactiveExecutor.scheduleMain(DefaultReactiveExecutor.java:60)
	at org.apache.camel.processor.Pipeline.process(Pipeline.java:147)
	at org.apache.camel.impl.engine.CamelInternalProcessor.process(CamelInternalProcessor.java:312)
	at org.apache.camel.component.jms.EndpointMessageListener.onMessage(EndpointMessageListener.java:141)
	at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:736)
	at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:696)
	at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:674)
	at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:318)
	at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257)
	at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1237)
	at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1227)
	at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1120)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:840)
	Suppressed: org.apache.camel.http.common.HttpOperationFailedException: HTTP operation failed invoking http://hypercube:8000/ with statusCode: 500
		... 21 common frames omitted
		Suppressed: org.apache.camel.http.common.HttpOperationFailedException: HTTP operation failed invoking http://hypercube:8000/ with statusCode: 500
			... 21 common frames omitted
			Suppressed: org.apache.camel.http.common.HttpOperationFailedException: HTTP operation failed invoking http://hypercube:8000/ with statusCode: 500
				... 21 common frames omitted
				Suppressed: org.apache.camel.http.common.HttpOperationFailedException: HTTP operation failed invoking http://hypercube:8000/ with statusCode: 500
					... 21 common frames omitted
					Suppressed: org.apache.camel.http.common.HttpOperationFailedException: HTTP operation failed invoking http://hypercube:8000/ with statusCode: 500
						at org.apache.camel.component.http.HttpProducer.populateHttpOperationFailedException(HttpProducer.java:362)
						at org.apache.camel.component.http.HttpProducer.process(HttpProducer.java:246)
						at org.apache.camel.support.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:66)
						at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:169)
						at org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$RedeliveryTask.doRun(RedeliveryErrorHandler.java:714)
						at org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$RedeliveryTask.run(RedeliveryErrorHandler.java:623)
						... 16 common frames omitted
nat@LIB-6HSWM53:~/tools/isle-dc$

When does this issue occur?
Upload a page repository item.
Which page does it occur on?
Any new page item.
What happens?
The hocr text file does not get created.
To whom does it occur (anonymous visitor, editor, administrator)?
editor/admin
What did you expect to happen?
The hocr file to be created.

Which version of Islandora are you using?

Islandora Starter Site
ISLE-DC TAG=3.0.0

Any related open or closed issues to this bug report?

Screenshots:

[FEATURE] Add CrayFits to Crayfish

Overview of feature request

UPEI's Robertson Library has been holding onto the code for Crayfits. It is currently being installed in both provisioning systems, ISLE-DC and Playbook.

We propose to contribute the code of CrayFits to Islandora, to live within the Crayfish repository.

This would be accompanied by:

instructions for existing sites to update Crayfish and remove the separate CrayFits service (both on ISLE and Playbook
an update to Islandora to include the repository known as islandora_fits as a submodule of Islandora (ticket)
an update to the Playbook to not install the separate repositories (Crayfits/islandora_fits), and to configure/use the code within Crayfish/Islandora
an update to ISLE to do the same

Any related open or closed issues to this feature request?
Islandora/islandora#906