Support Zeebe StreamActivatedJobs API about camunda-8-js-sdk HOT 11 CLOSED

@aleksander-dytko an update on the status of this. This has been more challenging than I had anticipated. My revised time to deliver support for this feature in the JS SDK is the second week of June.

from camunda-8-js-sdk.

OK, I got it working outside the test suite. There seems to be some interaction in the way the test suite mechanics are wired up.

from camunda-8-js-sdk.

Hi @jwulf, do I assume correctly that this is targeted for 8.5 release?

from camunda-8-js-sdk.

Hi @aleksander-dytko, it will probably soon come after the 8.5 release, asynchronously. There is a lot of work across the entire SDK for the 8.5 release.

from camunda-8-js-sdk.

Thanks @jwulf. Do you know the rough timeline for this?
The customer raised a Feature Request here and I wanted to provide them with an update

from camunda-8-js-sdk.

@jwulf @akeller do we know when this item will be delivered? I wanted to update the customer in the feature request

from camunda-8-js-sdk.

This feature is under development, I estimate that it will be available at the end of May. 

In the meantime, would you be able to share the current worker creation parameters? There may be some tuning that can be done with the existing long polling that could mitigate the resource exhaustion. 

Some ideas that occur to me are to extend the long poll period, and also to add a randomised offset to it so that different workers that are started at the same time do not all poll at once.

from camunda-8-js-sdk.

At the moment, this feature fails in CI against both SM and SaaS. Example: https://github.com/camunda/camunda-8-js-sdk/actions/runs/9313149620

On my local machine, it fails or succeeds with no discernable pattern.

I am using Wireshark to see if I can debug it at the network level.

The stream is opened, but sometimes nothing comes over it locally. On CI, nothing comes over it.

CI is running Ubuntu. Locally, I am running MacOS. I am installing Ubuntu locally to see what happens there.

from camunda-8-js-sdk.

@pepopowitz and I paired on this today. This appears to be an issue with the underlying JS gRPC library. We haven't been able to isolate the circumstances surrounding the failure to get it to stably reproduce.

Today, it worked for some time, and then started failing. I had thought that restarting the SM stack caused it to work for some time, but even that didn't work.

Here is a network packet capture from Wireshark for the failing call from the JS client:

The data packet that is being sent is an End Stream command.

With GRPC trace debugging turned on, this is what happens:

For some reason, the stream is terminated by the client immediately.

Contrast this with the network capture of the same request from the Java client (working, and in parallel to the JS client):

from camunda-8-js-sdk.

In the test, it needs a two second delay after starting the stream before starting a process instance.

from camunda-8-js-sdk.

@aleksander-dytko support for the streaming jobs API is available in the latest packages: 8.5.5 and 8.6.0.

Documentation:

https://camunda.github.io/camunda-8-js-sdk/classes/index.Zeebe.ZeebeGrpcClient.html#streamJobs

Here is a potential "gotcha" using this feature, documented in the README:

Job Streaming

The Zeebe gRPC API supports streaming available jobs, rather than polling for them.

The ZeebeGrpcClient method StreamJobs allows you to use this API.

Please note that only jobs that become available after the stream is opened are pushed to the client. For jobs that were already activatable before the method is called, you need to use a polling worker.

In this release, this is not handled for you. You must both poll and stream jobs to make sure that you get jobs that were available before your application started as well as jobs that become available after your application starts.

In a subsequent release, the ZeebeWorker will transparently handle this for you.

from camunda-8-js-sdk.