Add OFFLOCK pool concurrency policy backed by RouteSegmentedConnPool

[arturobernalg]

Wire PoolingHttpClientConnectionManager to use RouteSegmentedConnPool when OFFLOCK is selected.

[garydgregory]

What "OFFLOCK" mean?

[arturobernalg]

What "OFFLOCK" mean?
HI @garydgregory. OFFLOCK is selected, PoolingHttpClientConnectionManager switches to RouteSegmentedConnPool, which keeps per-route state in independent segments, avoids a single global lock and offloads slow graceful closes to a bounded executor. The goal is to reduce contention in highly concurrent workloads while still honouring the same per-route / total caps as today.
The name was meant as “off-lock / low-lock” policy, but if you prefer a more explicit name like SEGMENTED (or similar),

[garydgregory]

What "OFFLOCK" mean?
HI @garydgregory. OFFLOCK is selected, PoolingHttpClientConnectionManager switches to RouteSegmentedConnPool, which keeps per-route state in independent segments, avoids a single global lock and offloads slow graceful closes to a bounded executor. The goal is to reduce contention in highly concurrent workloads while still honouring the same per-route / total caps as today.
The name was meant as “off-lock / low-lock” policy, but if you prefer a more explicit name like SEGMENTED (or similar),

Thank you for the explanation @arturobernalg

So the bottom line is that this is not a "lock
-free" implementation ("offlock" is very odd phrasing IMO.) This makes me feel that "off" in the name is inappropriate.

Any other name would be better IMO.

[arturobernalg]

What "OFFLOCK" mean?
HI @garydgregory. OFFLOCK is selected, PoolingHttpClientConnectionManager switches to RouteSegmentedConnPool, which keeps per-route state in independent segments, avoids a single global lock and offloads slow graceful closes to a bounded executor. The goal is to reduce contention in highly concurrent workloads while still honouring the same per-route / total caps as today.
The name was meant as “off-lock / low-lock” policy, but if you prefer a more explicit name like SEGMENTED (or similar),

Thank you for the explanation @arturobernalg

So the bottom line is that this is not a "lock -free" implementation ("offlock" is very odd phrasing IMO.) This makes me feel that "off" in the name is inappropriate.

Any other name would be better IMO.

OFFLOCK selects the segmented pool implementation and routes slow graceful closes off the hot pool locks to reduce lock contention. The name reflects that intent;

[ok2c]

@arturobernalg Please consider writing up a short connection management guide for the website

[rschmitt]

OFFLOCK is selected, PoolingHttpClientConnectionManager switches to RouteSegmentedConnPool, which keeps per-route state in independent segments, avoids a single global lock and offloads slow graceful closes to a bounded executor.

I believe this technique is commonly known as lock striping.

[arturobernalg]

@arturobernalg Please consider writing up a short connection management guide for the website

@ok2c done

[ok2c]

@arturobernalg You may want to cherry-pick this change-set to 5.6.x

[rschmitt]

OFFLOCK selects the segmented pool implementation and routes slow graceful closes off the hot pool locks to reduce lock contention.

Wasn't this feature previously implemented for the STRICT and LAX concurrency policies? If this is a feature we could eventually implement in strict and/or lax pools then I think we should be hesitant to expose OFFLOCK as part of the public API. There are other ways to expose an opt-in experimental feature that we plan to eventually make the default behavior.

[ok2c]

@rschmitt We will not know how well OFFLOCK performs and whether it is ready to become the default pool implementation unless it gets tested in read-world settings. But let's be realistic. People are not going to use or even experiment with it unless it has been made available in GA release and exposed as a part of public APIs.