Skip to content

Add supporting data structures to rate limit library#7088

Open
cplaursen wants to merge 8 commits into
xapi-project:feature/throttling2from
cplaursen:feature/throttling2
Open

Add supporting data structures to rate limit library#7088
cplaursen wants to merge 8 commits into
xapi-project:feature/throttling2from
cplaursen:feature/throttling2

Conversation

@cplaursen
Copy link
Copy Markdown
Contributor

The current rate limiting library is missing a way to assign incoming requests to corresponding rate limiters. We also need usage tracking in order to let users make decisions about who they should rate limit. This PR adds a table for mapping requests to rate limiters, and a data structure for keeping usage statistics, together with comprehensive unit tests.

@cplaursen cplaursen changed the base branch from master to feature/throttling2 May 21, 2026 09:28
@cplaursen cplaursen force-pushed the feature/throttling2 branch from 8db10a6 to 249292e Compare May 21, 2026 09:48
Comment thread ocaml/libs/rate-limit/test/test_lru.ml Outdated
Comment thread ocaml/libs/rate-limit/linked_list.ml Outdated
cplaursen added 4 commits May 22, 2026 09:35
Signed-off-by: Christian Pardillo Laursen <christian.pardillolaursen@citrix.com>
Signed-off-by: Christian Pardillo Laursen <christian.pardillolaursen@citrix.com>
We need an in-memory key-value store to catalogue callers for the purposes of
rate limiting and usage tracking. We want to support the specification of
callers via prefixes, which means that we cannot use a regular hashtable.

Instead, we can use an association list with an LRU cache, which should amortise
usage as we expect most calls to come from the same small group of callers.

Signed-off-by: Christian Pardillo Laursen <christian.pardillolaursen@citrix.com>
This commit adds a thread-safe counter that tracks API call usage, with the
objective of exposing a counter to an RRD.

Signed-off-by: Christian Pardillo Laursen <christian.pardillolaursen@citrix.com>
@cplaursen cplaursen force-pushed the feature/throttling2 branch from 249292e to 8b14089 Compare May 22, 2026 08:39
Comment thread ocaml/libs/rate-limit/test/test_rate_limit.ml
Comment thread ocaml/libs/rate-limit/linked_list.mli
module D = Debug.Make (struct let name = "caller_table" end)

module Key = struct
type t = {user_agent: string; client_ip: string}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd prefer to avoid using strings with '*', it seems needlesly flexible, and I'd prefer to have a more restrictive type representation which is easier to compare with its intent, and harder to misuse.

type match =
    Full of string
  | Prefix of string
  | Wildcard

type t = {user_agent: match; client_ip: match}

let matches_field ~pattern ~target = 
  match pattern with
  | Wildcard, _ ->
      true
  | _, Wildcard ->
      false
  | Prefix pattern , Prefix target | Prefix pattern, Full target ->
      String.starts_with ~prefix:pattern target
  | Full pattern, Full target ->
      String.equal pattern target

let compare_match a b = match a, b with [...]

let ( <?> ) a b = if a = 0 then b else a

let compare a b = match_compare a.user_agent b.user_agent <?> match_compare a.client_ip b.client_ip

This will need 3 functions to create each of the types of matches, but this is usually a good compromise, do let me know if you think it makes the rest of the code too unwieldy and this approach is not feasable

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this was a good idea. I've trimmed it down to Full or Prefix, since wildcards are just Prefix "". I've left out the functions to create the matches, and expect anyone using this library to just use the type constructors.


val get_uuid : t -> string

val get_last_called : t -> Mtime.span
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can the semantics of this value be explicitly defined for users of the module? I would prefer if this returned how much time has passed since the the last call was made, but then it's not clear how it's going to be used.

For reference, when it becomes get_time_since_last_called, Mtime_clock.counter can be used with (Mtime_clock.count last_called), and update it when it's called also becomes trivial, removing the need for the unit-tests by always using a new counter: (Mtime_clock.counter ()).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't really need to store the time in memory for my purposes, so I'll get rid of this for now and we can re-introduce if and when it's needed.

Copy link
Copy Markdown
Member

@robhoes robhoes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally this looks fine to me. Now let's see how it will actually get used :)

cplaursen added 4 commits May 28, 2026 14:11
Signed-off-by: Christian Pardillo Laursen <christian.pardillolaursen@citrix.com>
Signed-off-by: Christian Pardillo Laursen <christian.pardillolaursen@citrix.com>
Signed-off-by: Christian Pardillo Laursen <christian.pardillolaursen@citrix.com>
Signed-off-by: Christian Pardillo Laursen <christian.pardillolaursen@citrix.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants