chore: reduce variance by dkharms · Pull Request #325 · ozontech/seq-db

dkharms · 2026-01-24T08:34:38Z

Description

Since Golang is a language with a garbage collection we end up with a non-deterministic memory usage (and therefore CPU usage). Here I will attempt to find GOGC/GOMEMLIMIT combinations for different benchmarks which reduce standard deviation between measurements from baseline and comparison seq-db instances.

So far, I think that for each scenario we will need different settings. But in overall GOGC=30 and GOMEMLIMIT=800MiB look promising. We have almost identical CPU and memory usage for two different instances built against the same commit.

I have read and followed all requirements in CONTRIBUTING.md;
I used LLM/AI assistance to make this pull request;

If you have used LLM/AI assistance please provide model name and full prompt:

Model: {{model-name}}
Prompt: {{prompt}}

github-actions · 2026-01-24T08:34:54Z

❌ PR Title Validation Failed
Please refer to CONTRIBUTING.md

dkharms · 2026-01-24T08:36:44Z

@seqbenchbot up 0-reduce-variance search-keyword-exact-match

seqbenchbot · 2026-01-24T08:36:48Z

Nice, @dkharms <(-^,^-)=b!

Your request was successfully served.
Identificator for your ongoing benchmark - 679fd404.

Here is a list of helpful links:

Take a look at Grafana dashboard;
Live-tailing logs are also available;

Have a great time!

github-actions · 2026-01-24T08:38:05Z

🔴 Performance Degradation

Some benchmarks have degraded compared to the previous run.
Click on Show table button to see full list of degraded benchmarks.

Show table

Name	Previous	Current	Ratio	Verdict
`Bitmask-4`	`32489f`	`4d1442`
	`0.00 B/op`	`3.00 B/op`	`NaN`	`🔴`
`GenerateDocs-4`	`32489f`	`4d1442`
	`958.10 ns/op`	`1199.00 ns/op`	`1.25`	`🔴`

codecov-commenter · 2026-01-24T08:39:09Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 71.58%. Comparing base (32489f0) to head (40062be).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #325      +/-   ##
==========================================
+ Coverage   71.55%   71.58%   +0.02%     
==========================================
  Files         204      204              
  Lines       14769    14769              
==========================================
+ Hits        10568    10572       +4     
+ Misses       3447     3442       -5     
- Partials      754      755       +1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

dkharms · 2026-01-24T08:44:26Z

@seqbenchbot down 679fd404

seqbenchbot · 2026-01-24T08:44:29Z

Nice, @dkharms <(-^,^-)=b!

Your request was successfully served.
The benchmark with identificator 679fd404 was stopped.

Have a great time!

dkharms · 2026-01-24T08:45:53Z

@seqbenchbot up 0-reduce-variance search-keyword-exact-match

seqbenchbot · 2026-01-24T08:45:56Z

Nice, @dkharms <(-^,^-)=b!

Your request was successfully served.
Identificator for your ongoing benchmark - 7227ca64.

Here is a list of helpful links:

Take a look at Grafana dashboard;
Live-tailing logs are also available;

Have a great time!

github-actions · 2026-01-24T08:48:55Z

🔴 Performance Degradation

Some benchmarks have degraded compared to the previous run.
Click on Show table button to see full list of degraded benchmarks.

Show table

Name	Previous	Current	Ratio	Verdict
`FindSequence_Deterministic/corner-cases-1-4`	`32489f`	`84d801`
	`56.95 ns/op`	`64.97 ns/op`	`1.14`	`🔴`
`FindSequence_Random/medium-4`	`32489f`	`84d801`
	`9785.69 MB/s`	`8002.51 MB/s`	`0.82`	`🔴`
	`104.60 ns/op`	`128.00 ns/op`	`1.22`	`🔴`
`FindSequence_Random/small-4`	`32489f`	`84d801`
	`4700.69 MB/s`	`3637.96 MB/s`	`0.77`	`🔴`
	`54.46 ns/op`	`70.37 ns/op`	`1.29`	`🔴`
`GenerateDocs-4`	`32489f`	`84d801`
	`958.10 ns/op`	`1142.00 ns/op`	`1.19`	`🔴`

dkharms · 2026-01-24T08:52:01Z

@seqbenchbot down 7227ca64

seqbenchbot · 2026-01-24T08:52:04Z

Nice, @dkharms <(-^,^-)=b!

Your request was successfully served.
The benchmark with identificator 7227ca64 was stopped.

Have a great time!

dkharms · 2026-01-24T08:53:56Z

@seqbenchbot up 0-reduce-variance search-keyword-exact-match

seqbenchbot · 2026-01-24T08:53:59Z

Nice, @dkharms <(-^,^-)=b!

Your request was successfully served.
Identificator for your ongoing benchmark - eaea18f7.

Here is a list of helpful links:

Take a look at Grafana dashboard;
Live-tailing logs are also available;

Have a great time!

dkharms · 2026-01-24T08:59:48Z

@seqbenchbot down eaea18f7

seqbenchbot · 2026-01-24T08:59:51Z

Nice, @dkharms <(-^,^-)=b!

Your request was successfully served.
The benchmark with identificator eaea18f7 was stopped.

Have a great time!

dkharms · 2026-01-24T09:01:14Z

@seqbenchbot up 0-reduce-variance search-keyword-exact-match

seqbenchbot · 2026-01-24T09:01:17Z

Nice, @dkharms <(-^,^-)=b!

Your request was successfully served.
Identificator for your ongoing benchmark - d899da02.

Here is a list of helpful links:

Take a look at Grafana dashboard;
Live-tailing logs are also available;

Have a great time!

github-actions · 2026-01-24T09:03:45Z

🔴 Performance Degradation

Some benchmarks have degraded compared to the previous run.
Click on Show table button to see full list of degraded benchmarks.

Show table

Name	Previous	Current	Ratio	Verdict
`Bitmask-4`	`32489f`	`320b2b`
	`0.00 B/op`	`3.00 B/op`	`NaN`	`🔴`
`GenerateDocs-4`	`32489f`	`320b2b`
	`958.10 ns/op`	`1141.00 ns/op`	`1.19`	`🔴`

github-actions · 2026-01-24T09:13:48Z

🔴 Performance Degradation

Some benchmarks have degraded compared to the previous run.
Click on Show table button to see full list of degraded benchmarks.

Show table

Name	Previous	Current	Ratio	Verdict
`FindSequence_Random/large-4`	`32489f`	`8b26ca`
	`13221.66 MB/s`	`11555.60 MB/s`	`0.87`	`🔴`
	`1239.00 ns/op`	`1418.00 ns/op`	`1.14`	`🔴`
`FindSequence_Random/medium-4`	`32489f`	`8b26ca`
	`9785.69 MB/s`	`6881.05 MB/s`	`0.70`	`🔴`
	`104.60 ns/op`	`148.80 ns/op`	`1.42`	`🔴`

dkharms · 2026-01-24T09:15:19Z

@seqbenchbot down d899da02

seqbenchbot · 2026-01-24T09:15:23Z

Nice, @dkharms <(-^,^-)=b!

Your request was successfully served.
The benchmark with identificator d899da02 was stopped.

Have a great time!

dkharms · 2026-01-24T09:16:46Z

@seqbenchbot up 0-reduce-variance search-keyword-exact-match

seqbenchbot · 2026-01-25T16:12:26Z

Nice, @dkharms <(-^,^-)=b!

Your request was successfully served.
Identificator for your ongoing benchmark - e196bb8f.

Here is a list of helpful links:

Take a look at Grafana dashboard;
Live-tailing logs are also available;

Have a great time!

github-actions · 2026-01-25T16:14:50Z

🔴 Performance Degradation

Some benchmarks have degraded compared to the previous run.
Click on Show table button to see full list of degraded benchmarks.

Show table

Name	Previous	Current	Ratio	Verdict
`FindSequence_Random/tiny-4`	`e2e0db`	`2d69e6`
	`3467.75 MB/s`	`2312.13 MB/s`	`0.67`	`🔴`
	`18.46 ns/op`	`27.68 ns/op`	`1.50`	`🔴`
`ProcessDocuments-4`	`e2e0db`	`2d69e6`
	`381.00 B/op`	`34099.00 B/op`	`89.50`	`🔴`
	`4.00 allocs/op`	`5.00 allocs/op`	`1.25`	`🔴`

dkharms · 2026-01-25T16:50:04Z

@seqbenchbot down e196bb8f

seqbenchbot · 2026-01-25T16:50:07Z

Nice, @dkharms <(-^,^-)=b!

Your request was successfully served.
The benchmark with identificator e196bb8f was stopped.

Have a great time!

dkharms · 2026-01-25T16:50:29Z

@seqbenchbot up 0-reduce-variance mixed

seqbenchbot · 2026-01-25T16:50:32Z

Nice, @dkharms <(-^,^-)=b!

Your request was successfully served.
Identificator for your ongoing benchmark - b6ce2949.

Here is a list of helpful links:

Take a look at Grafana dashboard;
Live-tailing logs are also available;

Have a great time!

dkharms · 2026-01-25T16:57:08Z

@seqbenchbot down b6ce2949

seqbenchbot · 2026-01-25T16:57:11Z

Nice, @dkharms <(-^,^-)=b!

Your request was successfully served.
The benchmark with identificator b6ce2949 was stopped.

Have a great time!

github-actions · 2026-01-25T17:01:33Z

🔴 Performance Degradation

Some benchmarks have degraded compared to the previous run.
Click on Show table button to see full list of degraded benchmarks.

Show table

Name	Previous	Current	Ratio	Verdict
`FindSequence_Random/large-4`	`e2e0db`	`cfbfc0`
	`12750.08 MB/s`	`11077.13 MB/s`	`0.87`	`🔴`
	`1285.00 ns/op`	`1479.00 ns/op`	`1.15`	`🔴`
`FindSequence_Random/small-4`	`e2e0db`	`cfbfc0`
	`5216.00 MB/s`	`4084.51 MB/s`	`0.78`	`🔴`
	`49.08 ns/op`	`62.68 ns/op`	`1.28`	`🔴`
`FindSequence_Random/tiny-4`	`e2e0db`	`cfbfc0`
	`3467.75 MB/s`	`2486.19 MB/s`	`0.72`	`🔴`
	`18.46 ns/op`	`25.74 ns/op`	`1.39`	`🔴`
`GenerateDocs-4`	`e2e0db`	`cfbfc0`
	`950.70 ns/op`	`1146.00 ns/op`	`1.21`	`🔴`
`ProcessDocuments-4`	`e2e0db`	`cfbfc0`
	`381.00 B/op`	`34233.00 B/op`	`89.85`	`🔴`
	`4.00 allocs/op`	`5.00 allocs/op`	`1.25`	`🔴`

dkharms · 2026-01-25T17:01:39Z

@seqbenchbot up 0-reduce-variance mixed

seqbenchbot · 2026-01-25T17:01:42Z

Nice, @dkharms <(-^,^-)=b!

Your request was successfully served.
Identificator for your ongoing benchmark - feb52121.

Here is a list of helpful links:

Take a look at Grafana dashboard;
Live-tailing logs are also available;

Have a great time!

dkharms · 2026-01-25T17:09:29Z

@seqbenchbot down feb52121

seqbenchbot · 2026-01-25T17:09:31Z

Nice, @dkharms <(-^,^-)=b!

Your request was successfully served.
The benchmark with identificator feb52121 was stopped.

Have a great time!

github-actions · 2026-01-25T17:09:50Z

🔴 Performance Degradation

Some benchmarks have degraded compared to the previous run.
Click on Show table button to see full list of degraded benchmarks.

Show table

Name	Previous	Current	Ratio	Verdict
`FindSequence_Random/small-4`	`e2e0db`	`0da61a`
	`5216.00 MB/s`	`3992.55 MB/s`	`0.77`	`🔴`
	`49.08 ns/op`	`64.12 ns/op`	`1.31`	`🔴`
`FindSequence_Random/tiny-4`	`e2e0db`	`0da61a`
	`3467.75 MB/s`	`2485.70 MB/s`	`0.72`	`🔴`
	`18.46 ns/op`	`25.75 ns/op`	`1.39`	`🔴`
`ProcessDocuments-4`	`e2e0db`	`0da61a`
	`381.00 B/op`	`32695.00 B/op`	`85.81`	`🔴`
	`4.00 allocs/op`	`5.00 allocs/op`	`1.25`	`🔴`

dkharms · 2026-01-25T17:10:14Z

@seqbenchbot up 0-reduce-variance mixed

seqbenchbot · 2026-01-25T17:10:16Z

Nice, @dkharms <(-^,^-)=b!

Your request was successfully served.
Identificator for your ongoing benchmark - 04d0f843.

Here is a list of helpful links:

Take a look at Grafana dashboard;
Live-tailing logs are also available;

Have a great time!

dkharms · 2026-01-25T17:26:36Z

@seqbenchbot down 04d0f843

seqbenchbot · 2026-01-25T17:26:39Z

Nice, @dkharms <(-^,^-)=b!

Your request was successfully served.
The benchmark with identificator 04d0f843 was stopped.

Have a great time!

dkharms · 2026-01-25T17:29:21Z

@seqbenchbot up 0-reduce-variance mixed

seqbenchbot · 2026-01-25T17:29:24Z

Oh-oh, @dkharms ('>o.o)'>!

Something went wrong and I couldn't process your request.
Please take a closer look at error message:

start container: started hook: wait until ready: context deadline exceeded

github-actions · 2026-01-25T17:34:14Z

🔴 Performance Degradation

Some benchmarks have degraded compared to the previous run.
Click on Show table button to see full list of degraded benchmarks.

Show table

Name	Previous	Current	Ratio	Verdict
`FindSequence_Random/medium-4`	`e2e0db`	`5bf453`
	`8612.17 MB/s`	`7293.70 MB/s`	`0.85`	`🔴`
	`118.90 ns/op`	`140.40 ns/op`	`1.18`	`🔴`
`FindSequence_Random/small-4`	`e2e0db`	`5bf453`
	`5216.00 MB/s`	`3419.53 MB/s`	`0.66`	`🔴`
	`49.08 ns/op`	`74.86 ns/op`	`1.53`	`🔴`
`FindSequence_Random/tiny-4`	`e2e0db`	`5bf453`
	`3467.75 MB/s`	`2340.47 MB/s`	`0.67`	`🔴`
	`18.46 ns/op`	`27.34 ns/op`	`1.48`	`🔴`
`Indexer-4`	`e2e0db`	`5bf453`
	`548625366.00 B/op`	`629186384.00 B/op`	`1.15`	`🔴`

dkharms · 2026-01-25T17:47:47Z

@seqbenchbot up 0-reduce-variance mixed

seqbenchbot · 2026-01-25T17:47:50Z

Nice, @dkharms <(-^,^-)=b!

Your request was successfully served.
Identificator for your ongoing benchmark - c0954df7.

Here is a list of helpful links:

Take a look at Grafana dashboard;
Live-tailing logs are also available;

Have a great time!

github-actions · 2026-01-25T17:51:35Z

🔴 Performance Degradation

Some benchmarks have degraded compared to the previous run.
Click on Show table button to see full list of degraded benchmarks.

Show table

Name	Previous	Current	Ratio	Verdict
`GenerateDocs-4`	`e2e0db`	`043249`
	`950.70 ns/op`	`1185.00 ns/op`	`1.25`	`🔴`
`Indexer-4`	`e2e0db`	`043249`
	`548625366.00 B/op`	`660853252.00 B/op`	`1.20`	`🔴`

dkharms · 2026-01-25T18:39:16Z

@seqbenchbot down c0954df7

seqbenchbot · 2026-01-25T18:39:18Z

Nice, @dkharms <(-^,^-)=b!

Your request was successfully served.
The benchmark with identificator c0954df7 was stopped.

Have a great time!

chore: GOGC=30

ab5790e

dkharms changed the title ~~chore: GOGC=30~~ chore: reduce variance Jan 24, 2026

chore: GOGC=10

4e96a56

chore: GOGC=20

b5a0d88

chore: GOGC=30

8fad21e

chore: GOMEMLIMIT=500MiB

7e8c4c0

dkharms force-pushed the 0-reduce-variance branch from 679c0af to 7e8c4c0 Compare January 24, 2026 09:15

chore: go1.26rc=2

2475a8d

chore: GOGC=100, baseline go1.24

cc83e27

chore: GOEXPERIMENT=nogreentea

40062be

dkharms force-pushed the 0-reduce-variance branch from f1dc071 to 40062be Compare January 25, 2026 17:47

Conversation

dkharms commented Jan 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Uh oh!

github-actions bot commented Jan 24, 2026

Uh oh!

dkharms commented Jan 24, 2026

Uh oh!

seqbenchbot commented Jan 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jan 24, 2026

🔴 Performance Degradation

Uh oh!

codecov-commenter commented Jan 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

dkharms commented Jan 24, 2026

Uh oh!

seqbenchbot commented Jan 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dkharms commented Jan 24, 2026

Uh oh!

seqbenchbot commented Jan 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jan 24, 2026

🔴 Performance Degradation

Uh oh!

dkharms commented Jan 24, 2026

Uh oh!

seqbenchbot commented Jan 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dkharms commented Jan 24, 2026

Uh oh!

seqbenchbot commented Jan 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dkharms commented Jan 24, 2026

Uh oh!

seqbenchbot commented Jan 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dkharms commented Jan 24, 2026

Uh oh!

seqbenchbot commented Jan 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jan 24, 2026

🔴 Performance Degradation

Uh oh!

github-actions bot commented Jan 24, 2026

🔴 Performance Degradation

Uh oh!

dkharms commented Jan 24, 2026

Uh oh!

seqbenchbot commented Jan 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dkharms commented Jan 24, 2026

Uh oh!

seqbenchbot commented Jan 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jan 25, 2026

🔴 Performance Degradation

Uh oh!

dkharms commented Jan 25, 2026

Uh oh!

seqbenchbot commented Jan 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dkharms commented Jan 25, 2026

Uh oh!

seqbenchbot commented Jan 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

dkharms commented Jan 24, 2026 •

edited

Loading

seqbenchbot commented Jan 24, 2026 •

edited

Loading

codecov-commenter commented Jan 24, 2026 •

edited

Loading

seqbenchbot commented Jan 24, 2026 •

edited

Loading

seqbenchbot commented Jan 24, 2026 •

edited

Loading

seqbenchbot commented Jan 24, 2026 •

edited

Loading

seqbenchbot commented Jan 24, 2026 •

edited

Loading

seqbenchbot commented Jan 24, 2026 •

edited

Loading

seqbenchbot commented Jan 24, 2026 •

edited

Loading

seqbenchbot commented Jan 24, 2026 •

edited

Loading

seqbenchbot commented Jan 25, 2026 •

edited

Loading

seqbenchbot commented Jan 25, 2026 •

edited

Loading

seqbenchbot commented Jan 25, 2026 •

edited

Loading

seqbenchbot commented Jan 25, 2026 •

edited

Loading

seqbenchbot commented Jan 25, 2026 •

edited

Loading

seqbenchbot commented Jan 25, 2026 •

edited

Loading

seqbenchbot commented Jan 25, 2026 •

edited

Loading

seqbenchbot commented Jan 25, 2026 •

edited

Loading

seqbenchbot commented Jan 25, 2026 •

edited

Loading

seqbenchbot commented Jan 25, 2026 •

edited

Loading

seqbenchbot commented Jan 25, 2026 •

edited

Loading