Skip to content

Remove boxed timeout future allocation#815

Open
simbiont666 wants to merge 3 commits intocloudflare:mainfrom
simbiont666:timeout-noalloc-future
Open

Remove boxed timeout future allocation#815
simbiont666 wants to merge 3 commits intocloudflare:mainfrom
simbiont666:timeout-noalloc-future

Conversation

@simbiont666
Copy link

Hi!

This PR removes heap allocation in pingora-timeout by replacing boxed timeout futures with concrete future types.
Behavior stays the same.

Use concrete future types in ToTimeout/Timeout instead of Box<dyn Future>
@drcaramelsyrup
Copy link
Collaborator

Without having looked into too much detail yet, we were wondering if you've ran some basic benchmarks to show improvements if any.

@simbiont666
Copy link
Author

@drcaramelsyrup

The original timeout benchmarks hit the immediately-ready path, so they did not capture the case where the wrapped future is polled as Pending at least once. I added one to cover that path

so..
results on my machine (base M4 Mini), 5 runs each:

timeout-noalloc-future
pingora timeout pending-once 20.444917ms total, 204ns avg per iteration
pingora timeout pending-once 9.817333ms total, 98ns avg per iteration
pingora timeout pending-once 9.63175ms total, 96ns avg per iteration
pingora timeout pending-once 9.457833ms total, 94ns avg per iteration
pingora timeout pending-once 9.575208ms total, 95ns avg per iteration

tokio timeout pending-once 9.119375ms total, 91ns avg per iteration
tokio timeout pending-once 6.196458ms total, 61ns avg per iteration
tokio timeout pending-once 5.382125ms total, 53ns avg per iteration
tokio timeout pending-once 5.568625ms total, 55ns avg per iteration
tokio timeout pending-once 5.797833ms total, 57ns avg per iteration

BASELINE (9ac75d0)
pingora timeout pending-once 22.12925ms total, 221ns avg per iteration
pingora timeout pending-once 10.712041ms total, 107ns avg per iteration
pingora timeout pending-once 10.881959ms total, 108ns avg per iteration
pingora timeout pending-once 10.841833ms total, 108ns avg per iteration
pingora timeout pending-once 10.953334ms total, 109ns avg per iteration

tokio timeout pending-once 8.404417ms total, 84ns avg per iteration
tokio timeout pending-once 6.20375ms total, 62ns avg per iteration
tokio timeout pending-once 5.321458ms total, 53ns avg per iteration
tokio timeout pending-once 5.06025ms total, 50ns avg per iteration
tokio timeout pending-once 6.092583ms total, 60ns avg per iteration

I’m still new to this project, so I can’t confidently estimate how frequent this “at least one pending poll” path is in production traffic yet. But even though the tokio variant is still faster overall, fast timeout became about ~10% faster.
Workspace-level benchmarks did not show a meaningful overall change (differences were mostly within measurement noise).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants