Skip to content

Use byte.Buffer for huge string appends #1

@ivanburak

Description

@ivanburak

Hi stanNthe5

Good code and this message isn't an issue. It's just info.

strings.Builder is good for small amount of string writings into one buffer.
If you'd like to write hundred-thousand times into a buffer it will be better to use bytes.Buffer.
You can look at the third line in Benchmarks, it looks good :)

strSlice_vs_strBuilder>go test -bench .
goos: windows
goarch: amd64
pkg: github.com/stanNthe5/stringbuf
cpu: Intel(R) Core(TM) i7-8565U CPU @ 1.80GHz
BenchmarkStringBuf_Append-8                16274             71824 ns/op          283066 B/op         16 allocs/op
BenchmarkStringsBuilder_Append-8            6300            181336 ns/op          906771 B/op         22 allocs/op
BenchmarkByteBuffer_Append-8               17455             69732 ns/op          409601 B/op          2 allocs/op
PASS
ok      github.com/stanNthe5/stringbuf  3.640s

The whole code is below (I changed the algorithm by saving a random substring to avoid possible optimization by Go and removed Prepend-s)
start, end := substringIdx() 
... 
   sample[start:end])
package stringbuf

import (
	"bytes"
	"math/rand/v2"
	"strings"
	"testing"
)

const sample = "abcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghijabcdefghij"
const times = 2000
const half = len(sample) / 2

// stringbuf
func BenchmarkStringBuf_Append(b *testing.B) {
	b.ReportAllocs()
	b.ResetTimer()
	for b.Loop() {
		var sb StringBuf
		start, end := substringIdx()
		for range times {
			sb.Append(sample[start:end])
		}
		_ = sb.String()
	}
}

// strings.Builder
func BenchmarkStringsBuilder_Append(b *testing.B) {
	b.ReportAllocs()
	b.ResetTimer()
	for b.Loop() {
		var sb strings.Builder
		start, end := substringIdx()
		for range times {
			sb.WriteString(sample[start:end])
		}
		_ = sb.String()
	}
}

// bytes.Buffer
func BenchmarkByteBuffer_Append(b *testing.B) {
	b.ReportAllocs()
	b.ResetTimer()
	for b.Loop() {
		var sb bytes.Buffer
		sb.Grow(half * times)
		start, end := substringIdx()
		for range times {
			sb.WriteString(sample[start:end])
		}
		_ = sb.String()
	}
}

func substringIdx() (start, end int) {
	start = rand.IntN(half)
	end = start + half
	return
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions