Provider for Redis / Expired Jobs about hangfire.tags HOT 11 CLOSED

@markalanevans correct. Our hangfire is only keeping 10K job records as history, but the tag records are only being cleaned up after 24hrs. This was destroying our system with Busy errors, as there were millions of these tag records :) To get us through till our next deployment window tonight, I'm manually deleting them every 30min or so. And tonight we will be disabling till we get a better solution.

from hangfire.tags.

Good question. It’s been a while since I’ve touched that code, but I think the Redis provider itself passes an expiry date when setting values. Redis itself will clean up keys at the specified expiry date.

I have to admit that I’m not entirely sure, but I’ll have to dove in a bit more to find out.

Are you currently experiencing issues with this behavior?

from hangfire.tags.

We haven't installed it yet. We just came across the project and started to evaluate it and while reading the docs that was highlighted as a potential issue.

If you happen to have a quick answer that's great, if not we will leave that as an open question on our end to test.

Thank you very much for making this.

from hangfire.tags.

We are using this project and wondering about the same issue. Is there any harm in us manually cleaning up the tags folder so it only keeps a few thousand? Is there any harm to hangfire proper, if we delete a tag that is tied to jobs that haven't been cleaned up by hangfire?

from hangfire.tags.

@matthewjcooper so is it not auto cleaning up tags that are tied to expired jobs?

from hangfire.tags.

So, I've checked a few things here. The "issue" is that Redis does not support an expiry value on a single value in a set of values.

What we could do instead, is to register a special set which contains keys and use the expire timestamp as scores. We would also need a special worker process that runs sometimes to clean up the expired keys.

An alternative is doing this manually, like I think @matthewjcooper does.

from hangfire.tags.

Is there any harm in us manually cleaning up the tags folder so it only keeps a few thousand? Is there any harm to hangfire proper, if we delete a tag that is tied to jobs that haven't been cleaned up by hangfire?

No and no.

from hangfire.tags.

I've just created a branch (redis-expire) which contains some cleanup code for tags in Redis.

This will:

a) Use a "global" SET where the KeyValue combination of the tag is a value, and the expiry date is the score (in seconds since Epoch). Tags that do not expire are not in this list.
b) Added an ExpireTagWatcher which runs hourly and at the start of the application, and will cleanup the global SET in step a, but also cleanup every tag which is expired according to the global SET in step a.
c) Set the current date in tags:expire_min_ticks when started for the first time. It will use this value as the minimum date when filtering tags that should be removed. This means that tags from BEFORE this change WILL NOT be cleaned up!

@matthewjcooper Is it possible that you test this branch? It's not on nuget, so you need to compile it yourself.

from hangfire.tags.

@erwin-faceit would it be possible to build a pre-release NuGet package for your change for testing? I'd be willing to test it, I only have VS2022 and can't build due to the 4.5 Net build dependency.

from hangfire.tags.

Here you go!

https://www.nuget.org/packages/Hangfire.Tags/1.8.4-beta.1
https://www.nuget.org/packages/Hangfire.Tags.Pro.Redis/1.9.1-beta.1
https://www.nuget.org/packages/Hangfire.Tags.Redis.StackExchange/1.9.1-beta.1

from hangfire.tags.

Our internal tests are working for these changes, so I'll merge the code to the master branch, and release accordingly.

from hangfire.tags.