switch HA-proxy tests to h1load client #69

Sashan · 2025-11-27T15:37:15Z

This change simplifies current HA-proxy test set up. Testing no longer requires apache/nginx server as backend. Instead of using siege as a client the test uses
h1load [1].

The pull request also install httpterm [2] http/1.1 server. It's unused currently.

The HA-proxy configuration for testing matches the configuration used in 'State of SSL stacks' write up.

The h1load client currently runs with options as follows :
h1load
-l \ # long results, output expected by h1load shell script
-P \ # report also percentiles for gathared data
-d ${TEST_TIME} \ # test duration, TEST_TIME is 10secs
-c 500 \ # 500 concurrent connections
-t ${THREAD_COUNT} \ # gather data for 1, 2, 4, 8, 16, 32, 64 threads
-u \ # use runtime instead of system time
${BASE_URL}${PORT} # url where to connect to

The options above is just the initial version.

[1] https://github.com/wtarreau/h1load

[2] https://github.com/wtarreau/httpterm

[3] https://www.haproxy.com/blog/state-of-ssl-stacks

nhorman

This looks fine to me, but I'm a bit confused as to how h1load gets setup with this test. Is it meant to be run by hand independently?

Sashan · 2025-12-08T07:49:22Z

This looks fine to me, but I'm a bit confused as to how h1load gets setup with this test. Is it meant to be run by hand independently?

it's run by bench_run_haproxy.sh this comes from run_test() function:

    RESULT=${RESULT_DIR}/h1load-dh-rsa-noreuse-${THREAD_COUNT}-${SSL_LIB}.out
    PORT=$(( ${PORT_RSA} + ${PROXY_CHAIN}  ))
    LD_LIBRARY_PATH=${OPENSSL_DIR}/lib ${H1LOAD} \
        -l \
        -P \
        -d ${TEST_TIME} \
        -c 500 \
        -t ${THREAD_COUNT} \
        -u \
        ${BASE_URL}${PORT} > ${RESULT} || exit 1

it's the H1LOAD variable which holds the path to h1load client. the client is linked with desired SSL library. I'm still verifying set up and figuring out the command line options to use. I've added also ability to use the siege [1] client just to cross check the results with other kind of tests.

the part up to collecting results is mostly done. I'm still working on gnuplot scripts to post-process data. I will include them to separate PR.

wtarreau · 2025-12-11T18:15:47Z

Sasha, before you spend too much time on gnuplot scripts, I'll share you some hints to ease your job (e.g. using -ll instead of -l to have raw numbers instead of human-friendly ones). I'm also finalizing a few small changes that ease selecting relevant lines if you want to compute averages. I'll ping you soon about this.

Sashan · 2025-12-12T10:21:43Z

Walter, thanks a lot. I will be also happy for tips on how to set the h1load arguments. to run benchmark tests.
What I'm looking for is to have a test which will run for certain period of time (let's say 10secs) trying to do as many requests/handshakes as possible using 1, 2, 4, 8, 16, 32, 64 threads. The more requests handled within the period of time the better. This is just very gross benchmark to get some basic metric. Once again thank you.

wtarreau · 2025-12-12T11:03:57Z

Agreed, that's what I want to show you because it's not quite hard. With -d you can set the duration of the test, with -s you can configure a slow ramp-up period (absolutely necessary to avoid measurement errors), and I'll also show you how to pick the relevant values to provide a meaningful measure. On the haproxy side, using taskset is an easy and convenient way to select the number of threads you want. I'll try to dedicate you some time next week to work on this. Today I'm busy chasing a few bugs.

jogme

First iteration - only cosmetics

jogme · 2026-01-07T12:36:36Z

bench-scripts/bench_config_haproxy.sh

+PORT_EC_REUSE=${BENCH_PORT_EC_REUSE:-10200}
+PORT_EC=${BENCH_PORT_EC:-10300}
 HAPROXY_VERSION='v3.2.0'
+CERT_SUBJ=${BENCH_CERT_SUBJ:-'/CN=localhost'}


duplicity; already present on line 20

jogme · 2026-01-07T12:36:49Z

bench-scripts/bench_config_haproxy.sh

+PORT_EC=${BENCH_PORT_EC:-10300}
 HAPROXY_VERSION='v3.2.0'
+CERT_SUBJ=${BENCH_CERT_SUBJ:-'/CN=localhost'}
+CERT_ALT_SUBJ=${BENCH_CERT_ALT_SUBJ:-'subjectAltName=DNS:localhost,IP:127.0.0.1'}


same as above; line 21

jogme · 2026-01-07T12:37:19Z

bench-scripts/bench_config_haproxy.sh

+CERT_SUBJ=${BENCH_CERT_SUBJ:-'/CN=localhost'}
+CERT_ALT_SUBJ=${BENCH_CERT_ALT_SUBJ:-'subjectAltName=DNS:localhost,IP:127.0.0.1'}
+PROXY_CHAIN=${BENCH_PROXY_CHAIN:-21}
+HOST=${BENCH_HOST:-'127.0.0.1'}


same as above; line 22 (and used on line 23)

jogme · 2026-01-07T12:41:39Z

bench-scripts/bench_config_haproxy.sh

+    typeset SSL_LIB=$1
+    typeset RSACERTKEY=''
+    typeset ECCERTKEY=''
+    typeset HAPROXY_CONF='etc/haproxy.conf'


the value is changed before usage on line 181, so this assignment seems redundant

jogme · 2026-01-19T08:04:12Z

bench-scripts/bench_config_haproxy.sh

+	        PROXYCERT=${RSACERTKEY}
+	    else
+	        PROXYCERT=${ECCERTKEY}
+	    fi


indentation is off here and a few places also below

yeah, tab vs. spaces, thanks for spotting that.

jogme · 2026-01-19T08:18:34Z

bench-scripts/bench_run_haproxy.sh

+    RESULT=${RESULT_DIR}/h1load-dh-rsa-reuse-${THREAD_COUNT}-${SSL_LIB}.out
+    PORT=$(( ${PORT_RSA_REUSE} + ${PROXY_CHAIN}  ))
+    LD_LIBRARY_PATH=${OPENSSL_DIR}/lib ${H1LOAD} \
+        -l \
+        -P \
+        -d ${TEST_TIME} \
+        -c 500 \
+        -t ${THREAD_COUNT} \
+        -u \
+        --tls-reuse \
+        ${BASE_URL}${PORT} > ${RESULT} || exit 1
+
+    RESULT=${RESULT_DIR}/h1load-dh-rsa-noreuse-${THREAD_COUNT}-${SSL_LIB}.out
+    PORT=$(( ${PORT_RSA} + ${PROXY_CHAIN}  ))
+    LD_LIBRARY_PATH=${OPENSSL_DIR}/lib ${H1LOAD} \
+        -l \
+        -P \
+        -d ${TEST_TIME} \
+        -c 500 \
+        -t ${THREAD_COUNT} \
+        -u \
+        ${BASE_URL}${PORT} > ${RESULT} || exit 1
+
+    if [[ -x ${SIEGE} ]] ; then
+        RESULT=${RESULT_DIR}/siege-dh-rsa-noreuse-${THREAD_COUNT}-${SSL_LIB}.out
+        LD_LIBRARY_PATH=${OPENSSL_DIR}/lib ${SIEGE} \
+            -b \
+            -c ${THREAD_COUNT} \
+            -t ${TEST_TIME}S \
+        "${BASE_URL}${SIEGE_PORT_RSA}/?s=1k" 2> ${RESULT} 1> /dev/null
+    fi
+
+    RESULT=${RESULT_DIR}/h1load-ec-dsa-reuse-${THREAD_COUNT}-${SSL_LIB}.out
+    PORT=$(( ${PORT_EC_REUSE} + ${PROXY_CHAIN}  ))
+    LD_LIBRARY_PATH=${OPENSSL_DIR}/lib ${H1LOAD} \
+        -l \
+        -P \
+        -d ${TEST_TIME} \
+        -c 500 \
+        -t ${THREAD_COUNT} \
+        -u \
+        --tls-reuse \
+        ${BASE_URL}${PORT} > ${RESULT} || exit 1
+
+    RESULT=${RESULT_DIR}/h1load-ec-dsa-noreuse-${THREAD_COUNT}-${SSL_LIB}.out
+    PORT=$(( ${PORT_EC} + ${PROXY_CHAIN}  ))
+    LD_LIBRARY_PATH=${OPENSSL_DIR}/lib ${H1LOAD} \
+        -l \
+        -P \
+        -d ${TEST_TIME} \
+        -c 500 \
+        -t ${THREAD_COUNT} \
+        -u \
+        ${BASE_URL}${PORT} > ${RESULT} || exit 1


I think this could be put into a for to eliminate duplicity

I'm going for what you suggest however I'm not entirely convinced it makes code easier to read/modify. whatever if you will like it can be kept or code can be reverted to what I have now in PR.

jogme · 2026-01-19T08:22:05Z

bench-scripts/common_util.sh

+	# note this is hack because we always assume
+	# openssl-master is installed in INSTALL root
+	#
+	$(LD_LIBRARY_PATH="${INSTALL_ROOT}/openssl-master/lib" "${OPENSSL}" \


nit: there can be a check added if the openssl-master dir is already installed

bench-scripts/common_util.sh

bench-scripts/bench_config_haproxy.sh

t-j-h · 2026-01-26T10:59:42Z

You comment that you aren't using siege as a client - but I still see a pile of references there - is this expected? Is it still actually used? If so the PR comment should be updated.

Sashan · 2026-01-27T06:44:18Z

You comment that you aren't using siege as a client - but I still see a pile of references there - is this expected? Is it still actually used? If so the PR comment should be updated.

I will fix the comment. I think it is useful to keep both tools working just for a cross check.

This change also simplifies current HA-proxy test set up. Testing no longer requires apache/nginx server as backend. Test also uses h1load [1] client now. The pull request also installs httpterm [2] http/1.1 server. It's used as a backend for tests perfromed by siege [3]. The HA-proxy configuration for testing matches the configuration used in 'State of SSL stacks' [4] write up. The h1load client currently runs with options as follows : h1load -l \ # long results, output expected by h1load shell script -P \ # report also percentiles for gathared data -d ${TEST_TIME} \ # test duration, TEST_TIME is 10secs -c 500 \ # 500 concurrent connections -t ${THREAD_COUNT} \ # gather data for 1, 2, 4, 8, 16, 32, 64 threads -u \ # use runtime instead of system time ${BASE_URL}${PORT} # url where to connect to [1] https://github.com/wtarreau/h1load [2] https://github.com/wtarreau/httpterm [3] https://github.com/JoeDog/siege [4] https://www.haproxy.com/blog/state-of-ssl-stacks

Sashan added this to Development Board Nov 27, 2025

Sashan moved this to Waiting Review in Development Board Nov 27, 2025

Sashan force-pushed the ha-proxy.h1load branch from e118d97 to 67ed709 Compare November 27, 2025 16:01

vavroch2010 moved this from Waiting Review to In Progress in Development Board Dec 1, 2025

vavroch2010 assigned Sashan Dec 2, 2025

nhorman reviewed Dec 7, 2025

View reviewed changes

Sashan force-pushed the ha-proxy.h1load branch 2 times, most recently from be2f9de to 7d10408 Compare December 17, 2025 14:33

Sashan moved this from In Progress to Waiting Review in Development Board Dec 17, 2025

vavroch2010 requested a review from jogme January 8, 2026 09:06

jogme requested changes Jan 19, 2026

View reviewed changes

vavroch2010 moved this from Waiting Review to In Progress in Development Board Jan 21, 2026

Sashan force-pushed the ha-proxy.h1load branch from 1c43370 to 7e377ad Compare January 27, 2026 07:09

switch HA-proxy tests to h1load client #69

Are you sure you want to change the base?

switch HA-proxy tests to h1load client #69

Uh oh!

Conversation

Sashan commented Nov 27, 2025

Uh oh!

nhorman left a comment

Choose a reason for hiding this comment

Uh oh!

Sashan commented Dec 8, 2025

Uh oh!

wtarreau commented Dec 11, 2025

Uh oh!

Sashan commented Dec 12, 2025

Uh oh!

wtarreau commented Dec 12, 2025

Uh oh!

jogme left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

t-j-h commented Jan 26, 2026

Uh oh!

Sashan commented Jan 27, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants