I would like to make my function to poll the target queue every 100ms, so to achieve that I used to use maxPollingInterval settings and set it to 00:00:00.100, with QueueBatch installed, this doesn't seem to work anymore. any idea?

Thanks

Thx for adding batch support for message queues!

I'm trying to get QueueBatchTrigger to fire using WebJobs 3.0.4 but it won't.
If I change it to the default QueueTrigger it triggers message by message and yes I do have messages in my queu.

What am I doing wrong?

Program.cs

static async Task Main(string[] args)
        {
            var builder = new HostBuilder()
                .UseEnvironment("Development")
                .ConfigureWebJobs(b =>
                {
                    b.AddAzureStorageCoreServices()
                    .AddAzureStorage()
                    .AddServiceBus()
                    .AddEventHubs()
                    .AddTimers();
                })
                .ConfigureAppConfiguration(b =>
                {
                    // Adding command line as a configuration source
                    b.AddCommandLine(args);
                })
                .ConfigureLogging((context, b) =>
                {
                    b.SetMinimumLevel(LogLevel.Debug);
                    b.AddConsole();
                })
                .ConfigureServices((context,services) =>
                {
                    // add some sample services to demonstrate job class DI
                    services.Configure<QueuesOptions>(options =>
                    {
                        //maximum number of queue messages that are picked up simultaneously to be executed in parallel (default is 16)
                        options.BatchSize = 15;
                        //Maximum number of retries before a queue message is sent to a poison queue (default is 5)
                        options.MaxDequeueCount = 5;
                        //maximum wait time before polling again when a queue is empty (default is 1 minute).
                        options.MaxPollingInterval = System.TimeSpan.FromSeconds(60);
                    });

                })
                .UseConsoleLifetime();

            var host = builder.Build();
            using (host)
            {
                await host.RunAsync();
            }
        }

ProductQueu.cs

public class ProductQueu
    {
        public static void QueuListener([QueueBatchTrigger("product")]IMessageBatch batch)
        {
            Console.WriteLine(batch);
        }
    }

How does FastQueue achieves "up to 20x" faster than SDK provided client? What is the secret sauce?

Would it have better performance by having a shared HttpClient object so that it reuses port for getting messages instead of allocate and dispose each time?

First, a piece of solid work! Two things:

1. Looking at src/QueueBatch abstractions, they don't know anything about Azure Cloud queues. Neither does src/QueueBatch/Impl/Queues/FastCloudQueue.cs (just a quick glance, I suppose there are more). Do you think it's reasonable to make the lib more pluggable, etc. QueueBatch.Abstractions along with QueueBatch.Azure, QueueBatch.Aws, QueueBatch.RabbitMq etc.? This could make the whole a highly performant solution of any queue provider out there.

2. I'm also wondering whether it's worthwhile as it could also be that the performance improvement trick is strictly related with Azure queue specific stuff. Is it? Well, from what you said in https://github.com/Scooletz/QueueBatch#why it shouldn't be the case.

Just an idea, what do you think?

Hey @Scooletz

Could be a silly question, lets say my function receives a batch of 32 messages as an example, when it comes to calculate Azure Function pricing, will it be counted as executed 32 times or only once? My guess is latter, just want to be sure.
Thanks

So that I can process multiple messages in parallel and also mark them processed in parallel.

I see that the number of messages that are retrieved in one call is set to 32 (or CloudQueueMessage.MaxNumberOfMessagesToPeek, which I assume is 32 also).

Do you see any issue in extending the functionality to make this number configurable?

I struggled a bit to figure out how to actually get a hold of my queued message. Maybe clarify in docs (not sure if what I got is the best way...) what to do with a Memory<byte> object? Maybe my lack of c# knowledge but I think a clarification might be useful?!

so I 'd add a message as usual:

 await batchQueue.AddMessageAsync(new CloudQueueMessage(JsonConvert.SerializeObject(myDTO)));

and then consume it in my trigger

public static async Task RunAsync([QueueTrigger("batch-queue")]MyDTO myDTO, ILogger log)
{
        // use myDTO here ...
}

using BatchQueueTrigger

public static async System.Task RunAsync([QueueBatchTrigger("batch-queue")]IMessageBatch batch,  ILogger log)
{
        foreach (var msg in batch.Messages)
        {
                MyDTO dto = JsonConvert.DeserializeObject<MyDTO>(System.Text.Encoding.Default.GetString(msg.Payload.ToArray()))
        }
        batch.MarkAllAsProcessed();
}

is there a better way?

I see in issue #13 that QueueBatch doesn't respect host.json for the maxPollingInterval setting. Does it also ignore the rest of the host.json settings for queues as well?

"visibilityTimeout": "00:00:05",
"batchSize": 16,
"maxDequeueCount": 5,
"newBatchThreshold": 8,
"messageEncoding": "base64"

In particular I'm interested in setting the # of batches that can be retrieved and processed in parallel. The equivalent setting for single messages is the batchSize parameter. Also the newBatchThreshold for getting a new batch when the # processed gets down to a certain threshold. Is there an equivalent for these settings using the attribute or does QueueBatch already respect these settings from host.json?

Hey

I have scenario where handles messages in parallel, so the way I have this done is to wrap them in their own Task which calls MarkAsProcessed at the end, then await Task.WhenAll(allTasks).

It works fine with happy path, however if one of task causes exception, the function will again pick up all processed messages which seems wrong to me. so I had a look at the source code, and realize it doesn't dobatch.Complete in case where exception occurred

Do you think it is a valid feature/bug to add/fix? if so, I don't mind create a PR for it.

Thanks