Skip to content

Make it easier to configure rate limiter to never cross max_burst admitted cells in a specific period of time #219

@kskalski

Description

@kskalski

I spent some time debugging why rate limiter allowed total n larger than max_burst within first minute when using Quota::per_minute(max_burst).

It seems this is actually an allowed and correct behavior according to the algorithm and careful reading of documentation. What happened was basically: some of the cost cells got replenished before client requested admitting the subsequent max_burst + Xth cell.

Maybe the documentation could be still improved, e.g. stating explicitly that above scenario is possible, but I'm wondering about the best way to configure rate limiter for a use case where the "moving window" admittance of cells in a period specified when creating quota is always below max_burst.
My current simple approach is "down-scaling" the period, i.e. something like this:

 // `governor` counts cost accumulated *minus* replenished in time, so it won't
 // strictly forbid going over max burst in specified period of time (during
 // steady ramp-up initial requests will be forgotten before full period).
 // Downscaling quota to smaller granularity period mostly prevents this problem.
 let divisor = 10.min((minute_limit + rate_limits::MAX_COST - 1) / rate_limits::MAX_COST);
 let max_period_burst =
     rate_limits::Cost::new(minute_limit / divisor).unwrap_or(rate_limits::Cost::MIN);
 let unit_replienish_in = std::time::Duration::from_millis(60000 / minute_limit as u64);
 governor::Quota::with_period(unit_replienish_in).unwrap().allow_burst(max_period_burst)

One idea that would make it easier is a version of above code to be added to quota, so I could do something like
governor::Quota::per_minute(minute_limit).with_granularity(Duration::from_secs(10))

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions