Summary
During #532 we identified a guide cloud/operator/migration-guide.adoc that needs to be revisited.

Goal
Improve the guide to contain version we maintain
Groom the migration process

Summary
Holistically review every single guide. Test that it works and that the UX is smooth.
Take into consideration, the whole picture of Sonataflow.
This task will be split per chapter.

Goal
Review each chapter of our guides, look for everything that is unclear, find discrepancies.
Ensure the use case in the guide is possible to achieve, if there is no example for the use
case, validate it and create the example.

Tasks

• Setup -> #588
• Getting Started -> #583
• Core Concepts -> #589
• Tooling -> #590
• Service Orchestration -> #591
• Event Orchestration -> #592
• Security -> #593
• Executing, Testing and Troubleshooting -> #594
• Persistence -> #595
• Cloud -> #596
• Job Service -> #597
• Data Index -> #598

In https://sonataflow.org/serverlessworkflow/latest/getting-started/create-your-first-workflow-service.html#proc-boostrapping-the-project

kn workflow create \
--name serverless-workflow-hello-world \
--extension quarkus-jsonp,quarkus-smallrye-openapi \
--quarkus-platform-group-id=io.quarkus.platform \
--quarkus-version=2.16.7.Final

error : unknown flag: --extension

There are items that depend on changes in other repositories.

We should somehow consume them so that there are no changes needed in docs, when the dependent item is updated/changed.

Goal

• Groom the documentation around serverless to be ready for next release.
• Remove old stuff
• Reorg Quarkus guides to separate section under /use-cases/advanced-developer-use-cases
• Introduce couple of new guides that were spit from original ones

When I try to set a timeout to 2month: P2M I get an error:

[ERROR]         [error]: Build step org.kie.kogito.quarkus.common.deployment.KogitoAssetsProcessor#generateSources threw an exception: org.kie.kogito.codegen.process.ProcessCodegenException: Processes with errors are Invalid process: '/home/kogito/serverless-workflow-project/src/main/resources/m2k.sw.yml'. Found error: java.lang.IllegalArgumentException: An invalid "eventTimeout": "P2M" configuration was provided for the state "WaitForSaveTransformationCompletionEvent" in the serverless workflow: "Move2Kube workflow".When configured, it must be set with a greater than zero ISO 8601 time duration. For example PT30S. Or a valid expression, for example $CONST.myDuration, where 'myDuration' is defined in the constant section of the workflow

but according to https://en.wikipedia.org/wiki/ISO_8601#Durations it should be fine

To resolve ambiguity, "P1M" is a one-month duration and "PT1M" is a one-minute duration (note the time designator, T, that precedes the time value).

According to @wmedvede

a P1M (on month) in the context of the workflows can't be translated into how much time to wait.
When we say 1 month, are we talking about January -> 31 days, February -> 29/28 days, etc.

So this needs to be documented

We have a workflow definition that used to work before migrating to Apache to deploy on sonataflow.org. This DNS points to https://apache.github.io/incubator-kie-kogito-docs, which has this repo as the website source.

We have opened an ASF INFRA ticket: https://issues.apache.org/jira/browse/INFRA-25091 but it seems that it's in the parking lot. We might have to implement this workflow manually.

The use-cases & advanced-developer-use-cases is a bit counter intuitive as we do not have any other use cases there.
Intention was to have also advanced use case without java and for Administrator persona there in the future.

Goal
Revisit the section to make the chapters more intuitive.

Related to apache/incubator-kie-kogito-serverless-operator#312

After the mentioned feature is merged, we need a guide explaining to cluster admins how to set this new configuration and the implications of doing it.

Currently, some users can't have Docker installed locally, so they will likely use Podman instead.

Sometimes, compatibility problems may arise like this: containers/podman#15499.

We need a troubleshooting guide to add Podman configuration information to help users use Podman.

Summary
During #532 it has become clear we have very bloated guides doing lots of things. As per @kaldesai suggestions,
we should be streamlining these to be smaller and more focused.

Goal
Streamline create-your-first-workflow-service-with-kn-cli-and-vscode to 4 guides

• Create guide
• Run guide
• Deploy guides
• Undeploy guide

when running kogito examples such as serverless-workflow-custom-function-knative or serverless-workflow-loanbroker-showcase with minikube/kubernetes, I can call the service from my local host to kickoff the workflow, but the internal service to service calls fail with connection refused.

I assume since kourier load balancer external-ip is 127.0.0.1, thats where its picking that ip. Not sure what I need to do to fix this issue. The workflow json function is referencing "operation": "knative:custom-function-knative-service",

Example,

Caused by: java.util.concurrent.CompletionException: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: custom-function-knative-service.default.127.0.0.1.sslip.io/127.0.0.1:80 at io.smallrye.mutiny.operators.uni.UniBlockingAwait.await(UniBlockingAwait.java:79) at io.smallrye.mutiny.groups.UniAwait.atMost(UniAwait.java:65) at org.kie.kogito.addons.quarkus.knative.serving.customfunctions.PlainJsonKnativeServiceRequestClient.sendRequest(PlainJsonKnativeServiceRequestClient.java:86) at org.kie.kogito.addons.quarkus.knative.serving.customfunctions.KnativeServiceRequestClient.execute(KnativeServiceRequestClient.java:56) at org.kie.kogito.addons.quarkus.knative.serving.customfunctions.PlainJsonKnativeServiceRequestClient_ClientProxy.execute(Unknown Source) at org.kie.kogito.addons.quarkus.knative.serving.customfunctions.KnativeServerlessWorkflowCustomFunction.execute(KnativeServerlessWorkflowCustomFunction.java:58) at org.kie.kogito.addons.quarkus.knative.serving.customfunctions.KnativeServerlessWorkflowCustomFunction_ClientProxy.execute(Unknown Source) at org.kie.kogito.addons.quarkus.knative.serving.customfunctions.KnativeWorkItemHandler.internalExecute(KnativeWorkItemHandler.java:42) at org.kie.kogito.serverless.workflow.WorkflowWorkItemHandler.executeWorkItem(WorkflowWorkItemHandler.java:40) at org.kie.kogito.addons.quarkus.knative.serving.customfunctions.KnativeWorkItemHandler_ClientProxy.executeWorkItem(Unknown Source) at org.jbpm.process.instance.LightWorkItemManager.internalExecuteWorkItem(LightWorkItemManager.java:79) at org.jbpm.workflow.instance.node.WorkItemNodeInstance.lambda$internalTrigger$0(WorkItemNodeInstance.java:161) at org.jbpm.workflow.instance.node.WorkItemNodeInstance.processWorkItemHandler(WorkItemNodeInstance.java:174) ... 90 more

Have a guide in the docs/website about Knative Serving Revisions. This should help administrators to leverage from Knative platform to control which Workflow version to direct users. A/B and Canary tests are the perfect scenarios for this guide. Page: https://github.com/kiegroup/kogito-docs/blob/main/serverlessworkflow/modules/ROOT/pages/cloud/versioning-workflows-in-knative.adoc

We have to remove references from Infinispan, MongoDB, and Oracle (everything that's not Postgres) from our documentation since we are not distributing these images anymore.

Guide user on environment setup and explain why is each item needed.
Start with minimal setup.
Add more complex tools later in guide.

Summary
We used to have previews of the docs, when doing contribution for reviews.

Goal
Implement such previews if possible in Apache community

Add Newsletter subscription use case to documentation.

Goal
Implement a guide covering the $title topic for serverless documentation

Reference: https://github.com/apache/incubator-kie-kogito-docs/blob/main/serverlessworkflow/modules/ROOT/pages/cloud/quarkus/build-workflow-images-with-tekton.adoc
This file does not contain anything, it is just placeholder. Leaving it here to indicate intended location

Goal
Holistic review of non-quarkus event orchestration chapter

tooling guides mention and use serverless workflow editor.

We should be using SonataFlow editor

Goal is to replace and groom the tooling sections so that they are up to date with our naming

In Deploying a New Workflow section, there are two broken links:

Deploying the greeting Workflow example

kubectl apply -f https://raw.githubusercontent.com/kiegroup/kogito-serverless-operator/main/test/testdata/sw.kogito_v1alpha08_sonataflow_devmode.yaml -n <your_namespace>

The link should be replaced with https://github.com/kiegroup/kogito-serverless-operator/blob/main/test/testdata/sonataflow.org_v1alpha08_sonataflow_devmode.yaml

At the end of the same section, there is a similar issue with

For example using VSCode, there are the commands needed:

curl -S https://raw.githubusercontent.com/kiegroup/kogito-serverless-operator/main/config/samples/sw.kogito_v1alpha08_sonataflow_devmode.yaml > workflow_devmode.yaml
code workflow_devmode.yaml
kubectl apply -f workflow_devmode.yaml -n <your_namespace>

In this case, there is no sonataflow.org_v1alpha08_sonataflow_devmode.yaml in /config/samples folder. Since the guide require sonataflow.org/profile: dev the only applicable config seems to be sonataflow.org_v1alpha08_sonataflow_devmodeWithConfigMapAndExternalResource.yaml
Is it correct?

I'd be happy to open a PR to fix this, but I need a bit of guidance here :)

Title

In https://sonataflow.org/serverlessworkflow/latest/cloud/operator/install-serverless-operator.html#_uninstall_3, the command to uninstall the operator is:

kubectl delete -f https://raw.githubusercontent.com/kiegroup/kogito-serverless-operator/<version>.x/operator.yaml

When version should be replaced by the version get by

$ kubectl get deployment sonataflow-operator-controller-manager -n sonataflow-operator-system -o jsonpath="{.spec.template.spec.containers[?(@.name=='manager')].image}"
quay.io/kiegroup/kogito-serverless-operator:1.43

So based on what is written in the doc:

The operator manager image reflects the current operator’s version. Replace the major and minor version names in the command below. For example, if the image version is 1.41.0 use 1.41

The command should be

kubectl delete -f https://raw.githubusercontent.com/kiegroup/kogito-serverless-operator/1.43/operator.yaml

But that is not working as we got a 404, the correct command seems to be:

kubectl delete -f https://raw.githubusercontent.com/kiegroup/kogito-serverless-operator/v1.43.0/operator.yaml

So, using the same URL as the one when installing the operator:

kubectl create -f https://raw.githubusercontent.com/kiegroup/kogito-serverless-operator/v1.43.0/operator.yaml

So either the placeholder should be kubectl delete -f https://raw.githubusercontent.com/kiegroup/kogito-serverless-operator/v<version>/operator.yaml without any changes on the minor version.
Or, the aforementioned URL should be made available

Currently, the guide use either kn-cli starting guide or quarkus one.

Goal
Unify the usage of getting started in quarkus and non-quarkus guides

When running kn service list after following the instructions from Serverless Workflow plug-in for Knative CLI "RevisionMissing : Configuration "my-project" does not have any ready Revision." is shown.

Steps to reproduce:

kn workflow create
cd new-project
kn workflow build --image my-project
kn workflow deploy
kn service list

Output:

NAME                              URL                                                                    LATEST   AGE     CONDITIONS   READY   REASON
my-project                        http://my-project.default.10.99.29.228.sslip.io                                 3m42s   0 OK / 3     False   RevisionMissing : Configuration "my-project" does not have any ready Revision.

This issue blocks https://issues.redhat.com/browse/KOGITO-7694.

Title

Goal
Holistic review of non-quarkus getting-started

Goal
Holistic review of non-quarkus setup guides

Summary
During #532 it become clear this guide needs some love.

Goal
Improve https://sonataflow.org/serverlessworkflow/main/core/understanding-workflow-error-handling.html to be shorter and more focused.
Consider splitting into core-concepts and example

Holistic review of non-quarkus core-concepts chapter

Based on this project we can turn all that information in a practical guide to give users a starting point for working with Tekton and ArgoCD using the operator's gitops profile.

Holistic review of non-quarkus tooling chapter

Hi, in the webpage:
https://kiegroup.github.io/kogito-docs/serverlessworkflow/main/getting-started/create-your-first-workflow-service.html
the url: https://github.com/kiegroup/kogito-examples/tree/main/serverless-workflow-examples/serverless-workflow-hello-world
is HTTP 404 Not Found

Summary
As we are shifting towards more java-less guides we need to introduce such examples
where no Quarkus and Java is exposed to the user.

Goal

• Introduce getting-started-with-events guide for core/handling-events-on-workflows.adoc
• Introduce getting-started-with-callbacks guide for core/working-with-callbacks.adoc
• Update reference to a guides without quarkus for core/understanding-jq-expressions.adoc
• Introduce getting-started-with-inputs for core/defining-an-input-schema-for-workflows.adoc
• Introduce getting-started-with-outputs for core/defining-an-input-schema-for-workflows.adoc
• Review any other guide and ensure we reference non-quarkus examples in non-quarkus section

There is only workflow document.

Fix the outdated guides. We now use Github Issues.

Goal
Holistic review of non-quarkus security chapter

Page: https://github.com/kiegroup/kogito-docs/blob/main/serverlessworkflow/modules/ROOT/pages/integrations/camel-k-integration.adoc

Possible reference: https://github.com/knative-sandbox/eventing-kogito/tree/main/examples/camel-telegram

Summary
This guide is no longer up to date as it references older version and information.

Goal
Fix the guides to be up to date

Summary
Currently our serverless docs are referencing Minikube. In community we already switched to test on Kind.
It would be nice to unify this and use something we test on daily in guides

Goal
Exchange Minikube with Kind in documentation of serverless

I am trying the Hello World Sample for OpenShift Serverless Logic under https://openshift-knative.github.io/docs/docs/latest/serverless-logic/getting-started/create-your-first-workflow-service.html

The build step is fine but when running quarkus dev I run into the following issue relating to an image pull:

quarkus dev
[INFO] Scanning for projects...
[INFO]
[INFO] --------------< org.acme:serverless-workflow-hello-world >--------------
[INFO] Building serverless-workflow-hello-world 1.0.0-SNAPSHOT
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- quarkus-maven-plugin:2.16.7.Final:dev (default-cli) @ serverless-workflow-hello-world ---
[INFO] Invoking org.apache.maven.plugins:maven-resources-plugin:2.6:resources @ serverless-workflow-hello-world
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO] Invoking io.quarkus.platform:quarkus-maven-plugin:2.16.7.Final:generate-code @ serverless-workflow-hello-world
[INFO] Invoking org.apache.maven.plugins:maven-compiler-plugin:3.10.1:compile @ serverless-workflow-hello-world
[INFO] Nothing to compile - all classes are up to date
[INFO] Invoking org.apache.maven.plugins:maven-resources-plugin:2.6:testResources @ serverless-workflow-hello-world
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /Users/fch/dev/serverless-workflow-hello-world/src/test/resources
[INFO] Invoking io.quarkus.platform:quarkus-maven-plugin:2.16.7.Final:generate-code-tests @ serverless-workflow-hello-world
[INFO] Invoking org.apache.maven.plugins:maven-compiler-plugin:3.10.1:testCompile @ serverless-workflow-hello-world
[INFO] Nothing to compile - all classes are up to date
Listening for transport dt_socket at address: 5005
2023-06-13 13:28:06,190 INFO  [org.kie.kog.cod.api.uti.AddonsConfigDiscovery] (build-44) Performed addonsConfig discovery, found: AddonsConfig{usePersistence=false, useTracing=false, useMonitoring=false, usePrometheusMonitoring=false, useCloudEvents=true, useExplainability=false, useProcessSVG=false, useEventDrivenDecisions=false, useEventDrivenRules=false}
2023-06-13 13:28:06,632 INFO  [org.kie.kog.cod.cor.uti.ApplicationGeneratorDiscovery] (build-39) Generator discovery performed, found [processes]
2023-06-13 13:28:07,029 INFO  [doc.1.0]] (build-49) Creating container for image: testcontainers/sshd:1.1.0
2023-06-13 13:28:07,034 INFO  [org.tes.uti.RegistryAuthLocator] (build-49) Failure when attempting to lookup auth config. Please ignore if you don't have images in an authenticated registry. Details: (dockerImageName: testcontainers/sshd:1.1.0, configFile: /Users/fch/.docker/config.json. Falling back to docker-java default behaviour. Exception message: /Users/fch/.docker/config.json (No such file or directory)
2023-06-13 13:28:07,051 INFO  [doc.3.4]] (build-49) Creating container for image: testcontainers/ryuk:0.3.4
2023-06-13 13:28:07,172 INFO  [doc.3.4]] (build-49) Container testcontainers/ryuk:0.3.4 is starting: bc7292d6f2866551cc311bcc7ecc9fb401785b76cbb0cc9862c1ad397685b130
2023-06-13 13:28:07,909 INFO  [doc.3.4]] (build-49) Container testcontainers/ryuk:0.3.4 started in PT0.865798S
2023-06-13 13:28:07,948 INFO  [doc.1.0]] (build-49) Container testcontainers/sshd:1.1.0 is starting: 95d59ae804b4ccc8a10fe285f0f82a2078890d1e83ee62c95cd251b01e3c6885
2023-06-13 13:28:08,340 INFO  [doc.1.0]] (build-49) Container testcontainers/sshd:1.1.0 started in PT1.454878S
2023-06-13 13:28:08,523 INFO  [doc.red.io/.28]] (docker-java-stream--1713804985) Starting to pull image
2023-06-13 13:28:08,791 ERROR [com.git.doc.api.asy.ResultCallbackTemplate] (docker-java-stream--1713804985) Error during callback: java.lang.NullPointerException: Cannot invoke "String.matches(String)" because the return value of "com.github.dockerjava.api.model.PullResponseItem.getStatus()" is null
	at com.github.dockerjava.api.command.PullImageResultCallback.checkForDockerSwarmResponse(PullImageResultCallback.java:48)
	at com.github.dockerjava.api.command.PullImageResultCallback.onNext(PullImageResultCallback.java:35)
	at org.testcontainers.images.LoggedPullImageResultCallback.onNext(LoggedPullImageResultCallback.java:48)
	at org.testcontainers.images.TimeLimitedLoggedPullImageResultCallback.onNext(TimeLimitedLoggedPullImageResultCallback.java:73)
	at org.testcontainers.images.TimeLimitedLoggedPullImageResultCallback.onNext(TimeLimitedLoggedPullImageResultCallback.java:24)
	at org.testcontainers.shaded.com.github.dockerjava.core.exec.AbstrAsyncDockerCmdExec$1.onNext(AbstrAsyncDockerCmdExec.java:41)
	at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder$JsonSink.accept(DefaultInvocationBuilder.java:315)
	at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder$JsonSink.accept(DefaultInvocationBuilder.java:298)
	at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder.lambda$executeAndStream$1(DefaultInvocationBuilder.java:275)
	at java.base/java.lang.Thread.run(Thread.java:833)

Note that the images testcontainers/ryuk:0.3.4 and testcontainers/sshd:1.1.0 were pulled and the containers successfully started.

I have macOS with podman 4.5.1. and a rootful VM. I have also set ryuk.container.privileged=true following these instructions.

Document and give example of how a JSON string shall be formatted in data field when using data in context and jq expression

transition:
      nextState: WaitForSaveTransformationCompletionEvent
      produceEvents:
        - eventRef: saveTransformationEvent
          data: '{"gitRepo":".repositoryURL", "branch": ".targetBranch", "token": ".token", "workspaceId": ".workspaceId", "projectId": ".projectId", "transformId": ".transformId ", "workflowCallerId": "$WORKFLOW.instanceId"}'

was not working as the data was not parsed as expected when publishing event:

ProcessInstanceDataEvent {specVersion=1.0, id='338d205f-c266-42c5-9187-bd988dd1f756', source=/process/m2k, type='save-transformation', time=2024-01-30T08:48:05.895695548Z, subject='null', dataContentType='null', dataSchema=null, data={"gitRepo":".repositoryURL","branch":".targetBranch","token":".token","workspaceId":".workspaceId","projectId":".projectId","transformId":".transformId ","workflowCallerId":"$WORKFLOW.instanceId"}, kogitoProcessInstanceId='6311bf4e-c490-4473-8e22-bba027730530', kogitoRootProcessInstanceId='null', kogitoProcessId='m2k', kogitoRootProcessId='null', kogitoAddons='null', kogitoIdentity='null', extensionAttributes={kogitoproctype=SW, kogitoprocinstanceid=6311bf4e-c490-4473-8e22-bba027730530, kogitoprocist=Active, kogitoprocversion=1.0, kogitoprocid=m2k}}

Instead the data string shall be:

data: "{gitRepo:.repositoryURL|sub(\"http(s)?://\";\"ssh://\"), branch: .targetBranch, token: .token, workspaceId: .workspaceId, projectId: .projectId, transformId: .transformId, workflowCallerId: $WORKFLOW.instanceId}"

When published:

ProcessInstanceDataEvent {specVersion=1.0, id='293b7596-1253-4c98-ac12-4e9825f01bce', source=/process/m2k, type='save-transformation', time=2024-01-30T13:24:14.595998574Z, subject='null', dataContentType='null', dataSchema=null, data={"gitRepo":"ssh://bitbucket.org/gfarache31/m2k-test","branch":"aaaaaaasssss","token":null,"workspaceId":"b93980cb-3943-4223-9441-8694c098eeb9","projectId":"9b305fe3-d441-48ce-b01b-d314e86e14ec","transformId":"723dce89-c25c-4c7b-9ef3-842de92e6fe6","workflowCallerId":"7ddb5193-bedc-4942-a857-596b31f377ed"}, kogitoProcessInstanceId='7ddb5193-bedc-4942-a857-596b31f377ed', kogitoRootProcessInstanceId='null', kogitoProcessId='m2k', kogitoRootProcessId='null', kogitoAddons='null', kogitoIdentity='null', extensionAttributes={kogitoproctype=SW, kogitoprocinstanceid=7ddb5193-bedc-4942-a857-596b31f377ed, kogitoprocist=Active, kogitoprocversion=1.0, kogitoprocid=m2k}}

We need to start enhancing the new features and what we know about troubleshooting the operator build and deploy process using the preview profile.

Goal
Holistic review of non-quarkus service orchestration chapter

Currently the prerequisites mention different technologies and it is a mess overall.

Goal
Review prerequisites for non-quarkus guides.
Review prerequisites for quarkus guides.
Unify the environment configuration for non-quarkus guides
Unify the environment configuration for quarkus guides

Goal
Holistic review of non-quarkus Executing, Testing and Troubleshooting chapter

Summary
There is a TODO in https://github.com/apache/incubator-kie-kogito-docs/blob/main/serverlessworkflow/modules/ROOT/nav.adoc to implement Java Workflow Library

Goal
Implement the guide or chapter with more guides