Skip to content

Instantly share code, notes, and snippets.

@dtjm

dtjm/join_test.go

Last active July 23, 2024 19:31
Show Gist options
  • Save dtjm/c6ebc86abe7515c988ec to your computer and use it in GitHub Desktop.
Save dtjm/c6ebc86abe7515c988ec to your computer and use it in GitHub Desktop.
Download ZIP
Benchmarking various ways of concatenating strings in Go
Raw
join_test.go
package join
import (
"fmt"
"strings"
"testing"
)
var (
testData = []string{"a", "b", "c", "d", "e"}
)
func BenchmarkJoin(b *testing.B) {
for i := 0; i < b.N; i++ {
s := strings.Join(testData, ":")
_ = s
}
}
func BenchmarkSprintf(b *testing.B) {
for i := 0; i < b.N; i++ {
s := fmt.Sprintf("%s:%s:%s:%s:%s", testData[0], testData[1], testData[2], testData[3], testData[4])
_ = s
}
}
func BenchmarkConcat(b *testing.B) {
for i := 0; i < b.N; i++ {
s := testData[0] + ":" + testData[1] + ":" + testData[2] + ":" + testData[3] + ":" + testData[4]
_ = s
}
}
@shuLhan
Copy link

shuLhan commented Sep 10, 2017

Another one with bytes.Buffer,

func BenchmarkBuffer(b *testing.B) {
	for i := 0; i < b.N; i++ {
		var b bytes.Buffer
		b.WriteString(testData[0])
		b.WriteByte(':')
		b.WriteString(testData[1])
		b.WriteByte(':')
		b.WriteString(testData[2])
		b.WriteByte(':')
		b.WriteString(testData[3])
		b.WriteByte(':')
		b.WriteString(testData[4])
		s := b.String()
		_ = s
	}

Benchmark results in my laptop,

Environment:

  • Intel(R) Core(TM) i7-4750HQ CPU @ 2.00GHz
  • Go 1.9
master ms 0 % go test -bench=. -benchmem
goos: linux
goarch: amd64
BenchmarkJoin-8         20000000               102 ns/op              32 B/op          2 allocs/op
BenchmarkSprintf-8       3000000               416 ns/op              96 B/op          6 allocs/op
BenchmarkConcat-8       20000000                67.1 ns/op             0 B/op          0 allocs/op
BenchmarkBuffer-8       10000000               152 ns/op             112 B/op          1 allocs/op
PASS
ok      _/home/ms/Unduhan/sandbox/go/stringsconcat      6.922s

@danmaina
Copy link

danmaina commented Jun 10, 2018 •
edited
Loading

Just to be clear is a lower value better?
And In this case is Concat the best?
Kindly expound more on the results you got here.

@cluffja
Copy link

cluffja commented Jun 16, 2018 •
edited
Loading

I would tend to agree with danmaina. I am assuming "ns/op" means "nanoseconds per operation" and "B/op" means "Bytes per operation". Not sure what the second column is referencing. Total operations perhaps? While it looks like Concat takes it by a long shot everything I've read says that Buffer should be the preferred method for large quantity use.

@xpzouying
Copy link

I update the benchmark testcase: string concatenation and bytes.Buffer.

  1. string concatenation:

string concatenation will allocate a new space when string change.

So the new concatenation testcase will allocate a new memory space.

func BenchmarkConcat(b *testing.B) {
	for i := 0; i < b.N; i++ {
		s := testData[0] + ":"
		s += testData[1] + ":"
		s += testData[2] + ":"
		s += testData[3] + ":"
		s += testData[4]
		_ = s
	}
}
  1. bytes.Buffer will pre-allocate memory, we could reuse the some memory using bytes.Buffer.Reset().
func BenchmarkBuffer(b *testing.B) {
	var buf bytes.Buffer

	for i := 0; i < b.N; i++ {
		buf.Reset()

		buf.WriteString(testData[0])
		buf.WriteByte(':')
		buf.WriteString(testData[1])
		buf.WriteByte(':')
		buf.WriteString(testData[2])
		buf.WriteByte(':')
		buf.WriteString(testData[3])
		buf.WriteByte(':')
		buf.WriteString(testData[4])
		s := buf.String()
		_ = s
	}
}

The new result will be:

  • go version: 1.10
  • env: macbook osx
goos: darwin
goarch: amd64
BenchmarkJoin-8         20000000                80.0 ns/op            32 B/op          2 allocs/op
BenchmarkSprintf-8       5000000               343 ns/op              96 B/op          6 allocs/op
BenchmarkConcat-8       10000000               171 ns/op              32 B/op          4 allocs/op
BenchmarkBuffer-8       30000000                53.1 ns/op             0 B/op          0 allocs/op
PASS

@solumos
Copy link

solumos commented Nov 1, 2018

I ran this with the updated testcases that @xpzouying created using go 1.11, with similar results:

  • go version: 1.11
  • env:
  Model Name:	MacBook Pro
  Model Identifier:	MacBookPro14,3
  Processor Name:	Intel Core i7
  Processor Speed:	2.8 GHz
  Number of Processors:	1
  Total Number of Cores:	4
  L2 Cache (per Core):	256 KB
  L3 Cache:	6 MB
  Memory:	16 GB

goos: darwin
goarch: amd64
BenchmarkJoin-8         20000000               100 ns/op              32 B/op          2 allocs/op
BenchmarkSprintf-8       5000000               381 ns/op              96 B/op          6 allocs/op
BenchmarkConcat-8       10000000               190 ns/op              32 B/op          4 allocs/op
BenchmarkBuffer-8       30000000                57.8 ns/op             0 B/op          0 allocs/op
PASS

@ww9
Copy link

ww9 commented Nov 27, 2018 •
edited
Loading

Benchmark of all methods discussed so far.

$ go version go1.11 windows/amd64

$ go test -benchmem -bench .

Function-CPU threads # b.N used ns per loop memory allocated per loop # memory allocations per loop
BenchmarkJoin-12 20000000 133 ns/op 32 B/op 2 allocs/op
BenchmarkSprintf-12 3000000 527 ns/op 96 B/op 6 allocs/op
BenchmarkConcat-12 10000000 239 ns/op 32 B/op 4 allocs/op
BenchmarkConcatOneLine-12 20000000 72.6 ns/op 0 B/op 0 allocs/op
BenchmarkBuffer-12 10000000 121 ns/op 112 B/op 1 allocs/op
BenchmarkBufferWithReset-12 20000000 64.3 ns/op 0 B/op 0 allocs/op

CPU threads

Note that Go implicitly used -cpu=12 in my machine due to having 6 cores, 12 threads. You can limit this by passing -cpu=# like so:

$ go test -cpu=8 -benchmem -bench . prints:

Function-CPU threads # b.N used ns per loop memory allocated per loop # memory allocations per loop
BenchmarkJoin-8 10000000 137 ns/op 32 B/op 2 allocs/op

main_test.go

package main

import (
	"bytes"
	"fmt"
	"strings"
	"testing"
)

var (
	testData = []string{"a", "b", "c", "d", "e"}
)

func BenchmarkJoin(b *testing.B) {
	for i := 0; i < b.N; i++ {
		s := strings.Join(testData, ":")
		_ = s
	}
}

func BenchmarkSprintf(b *testing.B) {
	for i := 0; i < b.N; i++ {
		s := fmt.Sprintf("%s:%s:%s:%s:%s", testData[0], testData[1], testData[2], testData[3], testData[4])
		_ = s
	}
}

func BenchmarkConcat(b *testing.B) {
	for i := 0; i < b.N; i++ {
		s := testData[0] + ":"
		s += testData[1] + ":"
		s += testData[2] + ":"
		s += testData[3] + ":"
		s += testData[4]
		_ = s
	}
}

func BenchmarkConcatOneLine(b *testing.B) {
	for i := 0; i < b.N; i++ {
		s := testData[0] + ":" +
			testData[1] + ":" +
			testData[2] + ":" +
			testData[3] + ":" +
			testData[4]
		_ = s
	}
}

func BenchmarkBuffer(b *testing.B) {
	for i := 0; i < b.N; i++ {
		var b bytes.Buffer
		b.WriteString(testData[0])
		b.WriteByte(':')
		b.WriteString(testData[1])
		b.WriteByte(':')
		b.WriteString(testData[2])
		b.WriteByte(':')
		b.WriteString(testData[3])
		b.WriteByte(':')
		b.WriteString(testData[4])
		s := b.String()
		_ = s
	}
}

func BenchmarkBufferWithReset(b *testing.B) {
	var buf bytes.Buffer

	for i := 0; i < b.N; i++ {
		buf.Reset()

		buf.WriteString(testData[0])
		buf.WriteByte(':')
		buf.WriteString(testData[1])
		buf.WriteByte(':')
		buf.WriteString(testData[2])
		buf.WriteByte(':')
		buf.WriteString(testData[3])
		buf.WriteByte(':')
		buf.WriteString(testData[4])
		s := buf.String()
		_ = s
	}
}

@ilya-korotya
Copy link

ilya-korotya commented Dec 24, 2018 •
edited
Loading

Added two new test for fmt.Fprintf and strings.Builder:

package benchmark

import (
	"bytes"
	"fmt"
	"strings"
	"testing"
)

var (
	testData = []string{"a", "b", "c", "d", "e"}
)

func BenchmarkJoin(b *testing.B) {
	for i := 0; i < b.N; i++ {
		s := strings.Join(testData, ":")
		_ = s
	}
}

func BenchmarkSprintf(b *testing.B) {
	for i := 0; i < b.N; i++ {
		s := fmt.Sprintf("%s:%s:%s:%s:%s", testData[0], testData[1], testData[2], testData[3], testData[4])
		_ = s
	}
}

func BenchmarkConcat(b *testing.B) {
	for i := 0; i < b.N; i++ {
		s := testData[0] + ":"
		s += testData[1] + ":"
		s += testData[2] + ":"
		s += testData[3] + ":"
		s += testData[4]
		_ = s
	}
}

func BenchmarkConcatOneLine(b *testing.B) {
	for i := 0; i < b.N; i++ {
		s := testData[0] + ":" +
			testData[1] + ":" +
			testData[2] + ":" +
			testData[3] + ":" +
			testData[4]
		_ = s
	}
}

func BenchmarkBuffer(b *testing.B) {
	for i := 0; i < b.N; i++ {
		var b bytes.Buffer
		b.WriteString(testData[0])
		b.WriteByte(':')
		b.WriteString(testData[1])
		b.WriteByte(':')
		b.WriteString(testData[2])
		b.WriteByte(':')
		b.WriteString(testData[3])
		b.WriteByte(':')
		b.WriteString(testData[4])
		s := b.String()
		_ = s
	}
}

func BenchmarkBufferWithReset(b *testing.B) {
	var buf bytes.Buffer

	for i := 0; i < b.N; i++ {
		buf.Reset()

		buf.WriteString(testData[0])
		buf.WriteByte(':')
		buf.WriteString(testData[1])
		buf.WriteByte(':')
		buf.WriteString(testData[2])
		buf.WriteByte(':')
		buf.WriteString(testData[3])
		buf.WriteByte(':')
		buf.WriteString(testData[4])
		s := buf.String()
		_ = s
	}
}

func BenchmarkBufferFprintf(b *testing.B) {
	buf := &bytes.Buffer{}

	for i := 0; i < b.N; i++ {
		buf.Reset()

		fmt.Fprintf(buf, "%s:%s:%s:%s:%s", testData[0], testData[1], testData[2], testData[3], testData[4])
		s := buf.String()
		_ = s
	}

}

func BenchmarkBufferStringBuilder(b *testing.B) {
	var buf strings.Builder

	for i := 0; i < b.N; i++ {
		buf.Reset()

		buf.WriteString(testData[0])
		buf.WriteByte(':')
		buf.WriteString(testData[1])
		buf.WriteByte(':')
		buf.WriteString(testData[2])
		buf.WriteByte(':')
		buf.WriteString(testData[3])
		buf.WriteByte(':')
		buf.WriteString(testData[4])
		s := buf.String()
		_ = s
	}
}

Environment:

  • Intel(R) Core(TM) i7-7600U CPU @ 2.80GHz
  • Go 1.10
BenchmarkJoin-4                         20000000                72.8 ns/op            32 B/op          2 allocs/op
BenchmarkSprintf-4                       5000000               311 ns/op              96 B/op          6 allocs/op
BenchmarkConcat-4                       10000000               158 ns/op              32 B/op          4 allocs/op
BenchmarkConcatOneLine-4                30000000                51.1 ns/op             0 B/op          0 allocs/op
BenchmarkBuffer-4                       20000000                97.0 ns/op           112 B/op          1 allocs/op
BenchmarkBufferWithReset-4              30000000                44.5 ns/op             0 B/op          0 allocs/op
BenchmarkBufferFprintf-4                 5000000               299 ns/op              80 B/op          5 allocs/op
BenchmarkBufferStrigBuilder-4           20000000               106 ns/op              24 B/op          2 allocs/op

@RezaOptic
Copy link

just run with new go version

package benchmark

import (
	"bytes"
	"fmt"
	"strings"
	"testing"
)

var (
	testData = []string{"a", "b", "c", "d", "e"}
)

func BenchmarkJoin(b *testing.B) {
	for i := 0; i < b.N; i++ {
		s := strings.Join(testData, ":")
		_ = s
	}
}

func BenchmarkSprintf(b *testing.B) {
	for i := 0; i < b.N; i++ {
		s := fmt.Sprintf("%s:%s:%s:%s:%s", testData[0], testData[1], testData[2], testData[3], testData[4])
		_ = s
	}
}

func BenchmarkConcat(b *testing.B) {
	for i := 0; i < b.N; i++ {
		s := testData[0] + ":"
		s += testData[1] + ":"
		s += testData[2] + ":"
		s += testData[3] + ":"
		s += testData[4]
		_ = s
	}
}

func BenchmarkConcatOneLine(b *testing.B) {
	for i := 0; i < b.N; i++ {
		s := testData[0] + ":" +
			testData[1] + ":" +
			testData[2] + ":" +
			testData[3] + ":" +
			testData[4]
		_ = s
	}
}

func BenchmarkBuffer(b *testing.B) {
	for i := 0; i < b.N; i++ {
		var b bytes.Buffer
		b.WriteString(testData[0])
		b.WriteByte(':')
		b.WriteString(testData[1])
		b.WriteByte(':')
		b.WriteString(testData[2])
		b.WriteByte(':')
		b.WriteString(testData[3])
		b.WriteByte(':')
		b.WriteString(testData[4])
		s := b.String()
		_ = s
	}
}

func BenchmarkBufferWithReset(b *testing.B) {
	var buf bytes.Buffer

	for i := 0; i < b.N; i++ {
		buf.Reset()

		buf.WriteString(testData[0])
		buf.WriteByte(':')
		buf.WriteString(testData[1])
		buf.WriteByte(':')
		buf.WriteString(testData[2])
		buf.WriteByte(':')
		buf.WriteString(testData[3])
		buf.WriteByte(':')
		buf.WriteString(testData[4])
		s := buf.String()
		_ = s
	}
}

func BenchmarkBufferFprintf(b *testing.B) {
	buf := &bytes.Buffer{}

	for i := 0; i < b.N; i++ {
		buf.Reset()

		fmt.Fprintf(buf, "%s:%s:%s:%s:%s", testData[0], testData[1], testData[2], testData[3], testData[4])
		s := buf.String()
		_ = s
	}

}

func BenchmarkBufferStringBuilder(b *testing.B) {
	var buf strings.Builder

	for i := 0; i < b.N; i++ {
		buf.Reset()

		buf.WriteString(testData[0])
		buf.WriteByte(':')
		buf.WriteString(testData[1])
		buf.WriteByte(':')
		buf.WriteString(testData[2])
		buf.WriteByte(':')
		buf.WriteString(testData[3])
		buf.WriteByte(':')
		buf.WriteString(testData[4])
		s := buf.String()
		_ = s
	}
}

Environment:

  • Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz

  • Go 1.14.2

go test -v -run=BENCH -bench=. -benchtime 5s -benchmem

goos: linux
goarch: amd64
BenchmarkJoin-8                  	74904537	        80.1 ns/op	      16 B/op	       1 allocs/op
BenchmarkSprintf-8               	15639747	       377 ns/op	      96 B/op	       6 allocs/op
BenchmarkConcat-8                	31753981	       188 ns/op	      32 B/op	       4 allocs/op
BenchmarkConcatOneLine-8         	88710248	        73.6 ns/op	       0 B/op	       0 allocs/op
BenchmarkBuffer-8                	61480548	        97.7 ns/op	      64 B/op	       1 allocs/op
BenchmarkBufferWithReset-8       	100000000	        60.2 ns/op	       0 B/op	       0 allocs/op
BenchmarkBufferFprintf-8         	16348393	       365 ns/op	      80 B/op	       5 allocs/op
BenchmarkBufferStringBuilder-8   	69169862	        85.1 ns/op	      24 B/op	       2 allocs/op

@GwynethLlewelyn
Copy link

Hm. Seems that concat done in one line might be the winner here: it's the easiest to read, does not allocate anything, and is only beaten by writing to a buffer, which, although being cool as a concept, looks much weirder (especially for someone who comes from other programming languages where string concatenation is simply done with an operator between strings...). I'm sure that there are many special cases where it's worth the extra typing effort to use buffers, but... I'm a big fan of keeping things simple and understandable. If the 'cost' of doing so is just a dozen extra nanoseconds... it's worth the trouble, IMHO.

@lovung
Copy link

lovung commented Sep 9, 2021

Use the code from @RezaOptic above

go version go1.17 darwin/amd64

Results

goos: darwin
goarch: amd64
pkg: ***
cpu: Intel(R) Core(TM) i7-4980HQ CPU @ 2.80GHz
BenchmarkJoin-8                  	18044035	        74.59 ns/op	      16 B/op	       1 allocs/op
BenchmarkSprintf-8               	 3249290	       393.9 ns/op	      96 B/op	       6 allocs/op
BenchmarkConcat-8                	 6670018	       257.7 ns/op	      32 B/op	       4 allocs/op
BenchmarkConcatOneLine-8         	20360762	        94.10 ns/op	       0 B/op	       0 allocs/op
BenchmarkBuffer-8                	16493529	       102.2 ns/op	      64 B/op	       1 allocs/op
BenchmarkBufferWithReset-8       	29284802	        51.32 ns/op	       0 B/op	       0 allocs/op
BenchmarkBufferFprintf-8         	 2917341	       392.3 ns/op	      80 B/op	       5 allocs/op
BenchmarkBufferStringBuilder-8   	15469124	        93.51 ns/op	      24 B/op	       2 allocs/op

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment