Skip to content

Instantly share code, notes, and snippets.

@itamarhaber

itamarhaber/edgex-go-redisdb.md

Created July 20, 2018 16:41
Show Gist options
  • Save itamarhaber/086dcb35692745ad0f1ff492676b47c7 to your computer and use it in GitHub Desktop.
Save itamarhaber/086dcb35692745ad0f1ff492676b47c7 to your computer and use it in GitHub Desktop.
Download ZIP
Edgexfoundry and Redis for Core Data PoC
Raw
edgex-go-redisdb.md

Edgexfoundry and Redis for Core Data PoC

Objectives

  • Implement the Redis client for edgex-go core data client tests/benchmarks
  • Compare between TCP, UDS and embedded Redis connections vs. Mongo's

Setting up

Server set up

  1. Install OS (tested on Ubuntu 16.04 LTS, MacOS 10)
  2. Install git, buildtools, curl, jemalloc, libzmq, pkg-config...
  3. Install Go (1.7 or above), don't forget to export your $GOPATH and $GOROOT
  4. Install Mongo
  5. Install Redis (v4.0 or above) & configure to use unix domain sockets

Dependencies set up

eredis

An embedded Redis experiment. Needed for running the benchmarks in embedded mode.

  1. Clone https://github.com/redislabs/eredis to $GOROOT/src/github.com/redislabs/eredis

    $ mkdir -p $GOPATH/src/github.com/redislabs
$ cd $GOPATH/src/github.com/redislabs
$ git clone --recursive https://github.com/redislabs/eredis
$ cd eredis

  2. You may need jemalloc installed, e.g. for Ubuntu:

    $ cd redis/deps/jemalloc
$ ./autogen.sh
$ make
$ make install_lib
$ sudo ldconfig
$ cd -

  3. Follow the instructions in the README for making and testing

    $ make
$ LD_LIBRARY_PATH=`pwd`/redis/src ./eredis_test

  4. Note for MacOS users: replace LD_LIBRARY_PATH=... with DYLD_LIBRARY_PATH=...

redigo

The "standard" Redis client for Go.

  1. go get github.com/gomodule/redigo/redis

  2. To use the embedded Redis client, you'll need to checkout the "eredis" branch from the RedisLabs/redigo fork:

    $ cd $GOPATH/src/github.com/gomodule/redigo
$ git remote add redislabs https://github.com/redislabs/redigo
$ git fetch redislabs
$ git checkout redislabs/eredis

mergo

A helper package for implementing UpdateReading/UpdateEvent.

  1. go get github.com/imdario/mergo

msgp

A faster-than-bson serializer (ref: https://github.com/alecthomas/go_serialization_benchmarks)

  1. go get -u -t github.com/tinylib/msgp

Set up edgex-go

  1. To run the Redis client benchmarks, you'll need to checkout the "redis_client" branch from the RedisLabs/edgex-go fork:

    $ mkdir $GOROOT/src/github.com/edgexfoundry
$ cd $GOROOT/src/github.com/edgexfoundry
$ git clone redislabs https://github.com/redislabs/edgex-go
$ cd edgex-go

  2. Follow the instructions in the README.md to install and build (you'll need to install glide and libzmq as well)

  3. Check out the redis_client branch

    $ git checkout redis_client

Running the tests/benchmarks

The Redis client's code (from the RedisLabs fork, "redis_client" branch) is at $GOPATH/src/github.com/edgexfoundry/edgex-go/core/data/clients/redis-client.go

Run the tests there with:

LD_LIBRARY_PATH=$GOPATH/src/github.com/redislabs/eredis/redis/src go test --tags redisRunning

Run the benchmarks there with:

LD_LIBRARY_PATH=$GOPATH/src/github.com/redislabs/eredis/redis/src go test -v --tags "mongoRunning redisRunning" -benchmem -run=^$ github.com/edgexfoundry/edgex-go/core/data/clients -bench "^Benchmark(MongoDB.*|RedisDB_.*)$"

Benchmark results

Deployment notes

  • The benchmark was performed on an AWS EC t2.xlarge instance, running Ubuntu 16
  • Redis and Mongo were installed as services, using the default settings
  • Redis was configured to listen at a TCP port and a socket
  • Code commits
    • github.com/redislabs/edgex-go.git origin/redis_client@2e866b8
    • github.com/redislabs/redigo.git origin/eredis@266f60b
    • github.com/redislabs/eredis.git origin/master@ac0af5e

Formatted

Benchmark MongoDB (micro/op) RedisDB_Embedded (micro/op) RedisDB_TCP (micro/op) RedisDB_UDS (micro/op)
AddEvent 352 193 201 189
AddReading 146 35 76 69
EventById 553 15 146 99
EventCount 94 3 45 31
Events 475,972 12,713 100,022 67,734
EventsForDevice 47,433 1,216 10,039 6,801
ReadingById 111 4 48 32
ReadingCount 103 3 45 31
Readings 3,658 1,681 1,466 1,429
ReadingsByDevice 725 162 230 216

Raw

ubuntu@ip-172-31-21-38:~/go/src/github.com/edgexfoundry/edgex-go/core/data/clients$ LD_LIBRARY_PATH=$GOPATH/src/github.com/redislabs/eredis/redis/src go test -v --tags "mongoRunning redisRunning" -benchmem -run=^$ github.com/edgexfoundry/edgex-go/core/data/clients -bench "^Benchmark(RedisDB_.*|MongoDB)$" | tee results.txt; ./benchtomd.awk results.txt
INFO: 2018/07/20 15:20:00 INFO: Connecting to mongo at: 0.0.0.0:27017
goos: linux
goarch: amd64
pkg: github.com/edgexfoundry/edgex-go/core/data/clients
BenchmarkMongoDB/AddReading-4 	   10000	    146033 ns/op	    3632 B/op	      70 allocs/op
BenchmarkMongoDB/Readings-4   	     500	   3658220 ns/op	  553779 B/op	   27201 allocs/op
BenchmarkMongoDB/ReadingCount-4         	   10000	    103906 ns/op	    1944 B/op	      44 allocs/op
BenchmarkMongoDB/ReadingById-4          	   10000	    111875 ns/op	    2648 B/op	      67 allocs/op
BenchmarkMongoDB/ReadingsByDevice-4     	    3000	    725460 ns/op	   65607 B/op	    2876 allocs/op
BenchmarkMongoDB/AddEvent-4             	    5000	    352712 ns/op	   12493 B/op	     169 allocs/op
BenchmarkMongoDB/Events-4               	       3	 475972675 ns/op	14389898 B/op	  458216 allocs/op
BenchmarkMongoDB/EventCount-4           	   20000	     94343 ns/op	    1832 B/op	      43 allocs/op
BenchmarkMongoDB/EventById-4            	    3000	    553164 ns/op	   16192 B/op	     498 allocs/op
BenchmarkMongoDB/EventsForDevice-4      	      30	  47433898 ns/op	 1454413 B/op	   45885 allocs/op
--- BENCH: BenchmarkMongoDB
	mongo-client_test.go:41: This benchmark needs to have a running mongo on localhost
INFO: 2018/07/20 15:20:20 INFO: Connecting to Redis at: 0.0.0.0:6379
BenchmarkRedisDB_TCP/AddReading-4       	   20000	     76292 ns/op	    1019 B/op	      42 allocs/op
BenchmarkRedisDB_TCP/Readings-4         	    1000	   1466055 ns/op	  459816 B/op	    9010 allocs/op
BenchmarkRedisDB_TCP/ReadingCount-4     	   30000	     45855 ns/op	      56 B/op	       3 allocs/op
BenchmarkRedisDB_TCP/ReadingById-4      	   30000	     48208 ns/op	     304 B/op	      10 allocs/op
BenchmarkRedisDB_TCP/ReadingsByDevice-4 	   10000	    230763 ns/op	   46608 B/op	     912 allocs/op
BenchmarkRedisDB_TCP/AddEvent-4         	   10000	    201766 ns/op	    8459 B/op	     257 allocs/op
BenchmarkRedisDB_TCP/Events-4           	      20	 100022091 ns/op	 1796015 B/op	   28010 allocs/op
BenchmarkRedisDB_TCP/EventCount-4       	   30000	     45243 ns/op	      56 B/op	       3 allocs/op
BenchmarkRedisDB_TCP/EventById-4        	   10000	    146776 ns/op	    1632 B/op	      29 allocs/op
BenchmarkRedisDB_TCP/EventsForDevice-4  	     100	  10039875 ns/op	  180560 B/op	    2812 allocs/op
--- BENCH: BenchmarkRedisDB_TCP
	redis-client_test.go:44: This benchmark needs to have a running Redis on localhost
INFO: 2018/07/20 15:20:39 INFO: Connecting to Redis at: /tmp/redis.sock:0
BenchmarkRedisDB_UDS/AddReading-4       	   20000	     69243 ns/op	    1019 B/op	      42 allocs/op
BenchmarkRedisDB_UDS/Readings-4         	    1000	   1429381 ns/op	  459818 B/op	    9010 allocs/op
BenchmarkRedisDB_UDS/ReadingCount-4     	   50000	     31006 ns/op	      56 B/op	       3 allocs/op
BenchmarkRedisDB_UDS/ReadingById-4      	   50000	     32760 ns/op	     304 B/op	      10 allocs/op
BenchmarkRedisDB_UDS/ReadingsByDevice-4 	   10000	    216540 ns/op	   46608 B/op	     912 allocs/op
BenchmarkRedisDB_UDS/AddEvent-4         	   10000	    189691 ns/op	    8460 B/op	     257 allocs/op
BenchmarkRedisDB_UDS/Events-4           	      20	  67734402 ns/op	 1796018 B/op	   28010 allocs/op
BenchmarkRedisDB_UDS/EventCount-4       	   50000	     31067 ns/op	      56 B/op	       3 allocs/op
BenchmarkRedisDB_UDS/EventById-4        	   20000	     99317 ns/op	    1632 B/op	      29 allocs/op
BenchmarkRedisDB_UDS/EventsForDevice-4  	     200	   6801701 ns/op	  180561 B/op	    2812 allocs/op
--- BENCH: BenchmarkRedisDB_UDS
	redis-client_test.go:57: This benchmark needs to have a running Redis listening /tmp/redis.sock
INFO: 2018/07/20 15:21:00 INFO: Connecting to Redis at: :0
BenchmarkRedisDB_Embedded/AddReading-4  	   50000	     35784 ns/op	    1807 B/op	      82 allocs/op
BenchmarkRedisDB_Embedded/Readings-4    	    1000	   1681639 ns/op	  656512 B/op	   10046 allocs/op
BenchmarkRedisDB_Embedded/ReadingCount-4         	  500000	      3435 ns/op	     128 B/op	       8 allocs/op
BenchmarkRedisDB_Embedded/ReadingById-4          	  300000	      4553 ns/op	     487 B/op	      15 allocs/op
BenchmarkRedisDB_Embedded/ReadingsByDevice-4     	   10000	    162353 ns/op	   67207 B/op	    1024 allocs/op
BenchmarkRedisDB_Embedded/AddEvent-4             	   10000	    193767 ns/op	   13140 B/op	     462 allocs/op
BenchmarkRedisDB_Embedded/Events-4               	     100	  12713301 ns/op	 2706796 B/op	   46043 allocs/op
BenchmarkRedisDB_Embedded/EventCount-4           	  500000	      3424 ns/op	     128 B/op	       8 allocs/op
BenchmarkRedisDB_Embedded/EventById-4            	  100000	     15376 ns/op	    2528 B/op	      51 allocs/op
BenchmarkRedisDB_Embedded/EventsForDevice-4      	    1000	   1216887 ns/op	  271912 B/op	    4624 allocs/op
--- BENCH: BenchmarkRedisDB_Embedded
	redis-client_test.go:69: This benchmark needs to have liberedis.so at LD_LIBRARY_PATH=$GOROOT/src/github.com/redislabs/eredis/redis/src
PASS
ok  	github.com/edgexfoundry/edgex-go/core/data/clients	77.661s

benchtomd.awk

#!/usr/bin/awk -f

/^Benchmark/ {
    name=substr($1, 10)
    split(name, s, "/")
    db=s[1]
    split(s[2], b, "-")
    test=b[1]

    dbs[db]=1
    tests[test]=1
    ops[db ":" test]=$3
}

END {
    ndbs = asorti(dbs)
    ntests = asorti(tests)
    printf("| Benchmark |")
    for (d in dbs) {
        printf(" %s (micro/op) |", dbs[d])
    }
    printf("\n")

    printf("|-|")
    for (d in dbs) {
        printf("-|")
    }
    printf("\n")

    for (t in tests) {
        printf("| %s |", tests[t])
        for (d in dbs) {
            printf(" %\047d |", ops[dbs[d] ":" tests[t]]/1000)
        }
        printf("\n")
    }
}

Notes

  1. @jduranf uses a variation on the standard benchmark to generate his numbers. A modified "memory-db_test.go" that tries to mimic his methodology is included in the fork.
  2. Readings: the benchmark uses a fixed set of 1K readings for all Readings benchmarks (excluding the AddReading benchmark)
  3. Events: the benchmark uses a fixed set of 1K events, each with 5 respective readings, for all Events benchmarks (excluding the AddReading benchmark)
  4. The framework's benchmarks, in general:
    1. Are less important when measuring the performance of a full-dump queries (Events, Readings). While these generate a decent load, that type of data access is unlikely/shouldn't to be experienced in production.
    2. Would benefit from adding ranges/filters tests for examining the performance of complex-er queries the are are deemed to be commonly used by the framework.

Next steps

  • Refactor according to PR 298/California
  • Get rid of bson.ObjectId in domain/models' struct definitions to allow independent serialization. This is currently a slow-downer and basically a built-in dependcy for any data layer implementation.
  • Include all dependencies of redislabs/edgex-go glide install
  • Export the Redis client properly, i.e. as edgex-go expects
  • Generalize and complete support for additional APIs (meta data, ...)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment