Add "throttle" Method for Rate Limiting about python-diskcache HOT 6 CLOSED

"throttle" seems to be a good one-word name. Other considered: "meter", "pace", "tempo", "rhythm". Some googling shows that throttling requests is more common than other terms.

from python-diskcache.

See also the stampede lock code in the diskcache directory.

from python-diskcache.

Tested pseudocode:

def throttle(count, seconds):
    """Throttle function calls to `count` times per `seconds`.

    """
    rate = count / seconds

    def decorator(func):
        last = time.time()
        tally = count

        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            nonlocal last, tally
            while True:
                now = time.time()
                extent = now - last
                last = now
                tally += extent * rate
                if tally > count:
                    tally = count - 1
                    break
                elif tally < 1:
                    time.sleep((1 - tally) / rate)
                else:
                    tally -= 1
                    break
            func(*args, **kwargs)
        return wrapper
    return decorator

I'm doubtful this can be made multi-threaded with the atomic methods provided by diskcache. But maybe it can be good-enough.

from python-diskcache.

Design notes: throttle should be method on Cache objects that accepts key, count and seconds. The throttle method should use a transaction to calculate how long to sleep before asking again. If returned value is zero then perform action immediately.

Maybe there should be a Throttler object that uses the throttle method instead.

FanoutCache should also have throttle method but acts instead as decorator and handles retries for Timeout error. Django Cache ha same method that behaves identically.

Maybe have rate-limit method and throttle. One acts as low-level sleeper function, other acts as decorator. Yes, better to have separate methods.

from python-diskcache.

It would be nice to have an API for measuring the rate of something. Use case idea: better health checks. I think it could be implemented as a leaky-bucket with an un-bounded size. Return value is bucket size. Measure in calls per second.

from python-diskcache.

This is now included in the recipes. Commit reference bf7196d

from python-diskcache.