A Java microservice for transforming a JSON entity stream. This service is designed to be used as a microservice system for the HTTP transform in a Sesam service instance.

It decodes binary content or downloads files via URLs, and then extracts textual information using the Apache Tika library (http://tika.apache.org/). The resulting text is inserted as a string property into the entities and returned to the client.

You will need a working Java 8 dev environment and Maven 3.x to build this service.

cd content-extraction-service
mvn clean install
docker build -t sesam/content-extraction-service:latest .
docker run -it -p 4567:4567 sesam/content-extraction-service:latest

Get the IP from docker:

docker inspect -f '{{.Name}} - {{.NetworkSettings.IPAddress }}' content-extraction-service

JSON entities can be posted to 'http://localhost:4567/transform'. The result is streamed back to the client. Exchange "localhost" with the Docker IP if running in Docker.

$ curl -s -XPOST 'http://localhost:4567/transform' -H "Content-type: application/json" -d '[{ "_id": "jane", "url": "http://some-url-to-a-file/file.pdf"}]' | jq -S .
[
  {
    "_id": "jane",
    "url": "http://some-url-to-a-file/file.pdf",
    "_content": "The content in text form"
  }
]

Note the example uses curl to send the request and jq prettify the response.

You can configure the service with the following environment variables:

When running in Docker you can either specify this in a file (see https://docs.docker.com/compose/env-file/) or on the command line with "docker run .. -e VAR1=VAL1 -e VAR2=VAL2 .."