Microservice that calls a URL (with optional payload) and able to store the result in a configurable property.

• can be used as sink or transform
• entity level customization
• transform results are streamed back by default and streaming can be turned off.
• Listens on port 5001 by default
• can be configured to tolerate errors - to avoid pump execution failure

service_config_property, if specified must point to a dict in the incoming entities. The dict fields must be a subset of environment variables. refer to "Environment Parameters" section for their explanations and eligable values.

*: can be overruled by utilizing entity level customization via service_config_property envvar/query param.

AUTHORIZATION, if specified, can have following structures

No auth:

"AUTHORIZATION": None

Basic Auth

"AUTHORIZATION": {
  "type": "basic",
  "basic": ["my username", "my password"]
}

Oauth2:

"AUTHORIZATION": {
  "type": "oauth2",
  "oauth2": {
    "client_id": "my oauth2 client",
    "client_secret": "my oauth2 secret",
    "token_url": "my oauth2's token url"
  }
}

System:

{
  "_id": "my-rest-transform-system",
  "type": "system:microservice",
  "docker": {
    "environment": {
      "HEADERS": {
        "Accept": "application/json; version=2",
        "Authorization": "token my-travis-token"
      },
      "URL": "https://my_domain/my_script_root",
      "DO_STREAM": false,
      "PROPERTY": "mytransformresponse",
      "TOLERABLE_STATUS_CODES": "404|400"
    },
    "image": "sesamcommunity/sesam-rest-transform",
    "port": 5001
  }
}

Pipe:

{
  "_id": "my-transform-pipe",
  "type": "pipe",
  "source": {
    "type": "dataset",
    "dataset": "my-source"
  },
  "transform": [{
    "type": "dtl",
    "rules": {
      "default": [
        ["copy", "_id"],
        ["add", "id", "_S._id"]
      ]
    }
  }, {
    "type": "http",
    "system": "my-rest-transform-system",
    "url": "/transform?path=/mypath/to/myresource?myid={{ id }}"
  }, {
    "type": "dtl",
    "rules": {
      "default": [
        ["copy", "_id"],
        ["if",
          ["eq", "_S.mytransformresponse.transform_succeeded", false],
          ["comment", "optionally, do the tolerated error handling here, or alternatively in a succeeding pipe"],
          ["add", "details", "_S.mytransformresponse"]
        ]
      ]
    }
  }]
}

Following combinations are all equivalent for the examplified case: