Effect of message size

10
1756711 records sent, 351342.2 records/sec (3.35 MB/sec), 3.8 ms avg latency, 188.0 max latency.
1939614 records sent, 387922.8 records/sec (3.70 MB/sec), 4.1 ms avg latency, 50.0 max latency.
1912514 records sent, 382502.8 records/sec (3.65 MB/sec), 3.2 ms avg latency, 52.0 max latency.
1808519 records sent, 361703.8 records/sec (3.45 MB/sec), 2.7 ms avg latency, 34.0 max latency.
1855175 records sent, 371035.0 records/sec (3.54 MB/sec), 2.8 ms avg latency, 40.0 max latency.
1895567 records sent, 379113.4 records/sec (3.62 MB/sec), 2.7 ms avg latency, 43.0 max latency.
1899486 records sent, 379897.2 records/sec (3.62 MB/sec), 2.7 ms avg latency, 40.0 max latency.
1838392 records sent, 367678.4 records/sec (3.51 MB/sec), 2.4 ms avg latency, 36.0 max latency.
1940247 records sent, 388049.4 records/sec (3.70 MB/sec), 2.5 ms avg latency, 43.0 max latency.
1873731 records sent, 374746.2 records/sec (3.57 MB/sec), 2.9 ms avg latency, 59.0 max latency.
1866566 records sent, 373313.2 records/sec (3.56 MB/sec), 2.6 ms avg latency, 56.0 max latency.
1870301 records sent, 374060.2 records/sec (3.57 MB/sec), 2.4 ms avg latency, 34.0 max latency.
1940909 records sent, 388181.8 records/sec (3.70 MB/sec), 2.8 ms avg latency, 56.0 max latency.
1915724 records sent, 383144.8 records/sec (3.65 MB/sec), 3.4 ms avg latency, 57.0 max latency.
1943571 records sent, 388714.2 records/sec (3.71 MB/sec), 3.0 ms avg latency, 45.0 max latency.
1853652 records sent, 370730.4 records/sec (3.54 MB/sec), 2.9 ms avg latency, 43.0 max latency.
1932239 records sent, 386447.8 records/sec (3.69 MB/sec), 2.8 ms avg latency, 44.0 max latency.
1943343 records sent, 388668.6 records/sec (3.71 MB/sec), 2.9 ms avg latency, 38.0 max latency.
1901148 records sent, 380229.6 records/sec (3.63 MB/sec), 2.6 ms avg latency, 41.0 max latency.
1778880 records sent, 355776.0 records/sec (3.39 MB/sec), 2.3 ms avg latency, 31.0 max latency.
1939644 records sent, 387928.8 records/sec (3.70 MB/sec), 2.4 ms avg latency, 36.0 max latency.
1907163 records sent, 381432.6 records/sec (3.64 MB/sec), 2.4 ms avg latency, 33.0 max latency.
1898420 records sent, 379684.0 records/sec (3.62 MB/sec), 2.4 ms avg latency, 37.0 max latency.
1939617 records sent, 387923.4 records/sec (3.70 MB/sec), 2.5 ms avg latency, 39.0 max latency.
1911538 records sent, 382307.6 records/sec (3.65 MB/sec), 2.4 ms avg latency, 33.0 max latency.
1907961 records sent, 381592.2 records/sec (3.64 MB/sec), 2.6 ms avg latency, 38.0 max latency.
1836284 records sent, 367256.8 records/sec (3.50 MB/sec), 2.6 ms avg latency, 35.0 max latency.
1899760 records sent, 379952.0 records/sec (3.62 MB/sec), 2.4 ms avg latency, 33.0 max latency.
1757356 records sent, 351471.2 records/sec (3.35 MB/sec), 2.3 ms avg latency, 35.0 max latency.
1835947 records sent, 367189.4 records/sec (3.50 MB/sec), 2.4 ms avg latency, 36.0 max latency.
1712425 records sent, 342485.0 records/sec (3.27 MB/sec), 2.4 ms avg latency, 40.0 max latency.
1792609 records sent, 358521.8 records/sec (3.42 MB/sec), 2.4 ms avg latency, 35.0 max latency.
1903583 records sent, 380716.6 records/sec (3.63 MB/sec), 2.4 ms avg latency, 36.0 max latency.
1869068 records sent, 373813.6 records/sec (3.56 MB/sec), 2.4 ms avg latency, 40.0 max latency.
1903910 records sent, 380782.0 records/sec (3.63 MB/sec), 2.4 ms avg latency, 36.0 max latency.
1934477 records sent, 386895.4 records/sec (3.69 MB/sec), 2.4 ms avg latency, 37.0 max latency.
1868607 records sent, 373721.4 records/sec (3.56 MB/sec), 2.5 ms avg latency, 36.0 max latency.
1914570 records sent, 382914.0 records/sec (3.65 MB/sec), 2.7 ms avg latency, 35.0 max latency.
1933073 records sent, 386614.6 records/sec (3.69 MB/sec), 2.6 ms avg latency, 35.0 max latency.
1892098 records sent, 378419.6 records/sec (3.61 MB/sec), 2.9 ms avg latency, 48.0 max latency.
1908384 records sent, 381676.8 records/sec (3.64 MB/sec), 2.9 ms avg latency, 49.0 max latency.
1779514 records sent, 355902.8 records/sec (3.39 MB/sec), 2.7 ms avg latency, 35.0 max latency.
1755487 records sent, 351097.4 records/sec (3.35 MB/sec), 2.7 ms avg latency, 38.0 max latency.
1843027 records sent, 368605.4 records/sec (3.52 MB/sec), 2.7 ms avg latency, 47.0 max latency.
1946220 records sent, 389244.0 records/sec (3.71 MB/sec), 2.8 ms avg latency, 37.0 max latency.
1918169 records sent, 383633.8 records/sec (3.66 MB/sec), 3.2 ms avg latency, 59.0 max latency.
1849696 records sent, 369939.2 records/sec (3.53 MB/sec), 2.6 ms avg latency, 36.0 max latency.
1910600 records sent, 382120.0 records/sec (3.64 MB/sec), 2.7 ms avg latency, 39.0 max latency.
1941565 records sent, 388313.0 records/sec (3.70 MB/sec), 2.9 ms avg latency, 37.0 max latency.
1871596 records sent, 374319.2 records/sec (3.57 MB/sec), 2.8 ms avg latency, 35.0 max latency.
1778332 records sent, 355666.4 records/sec (3.39 MB/sec), 2.9 ms avg latency, 42.0 max latency.
1895098 records sent, 379019.6 records/sec (3.61 MB/sec), 2.5 ms avg latency, 32.0 max latency.
1888680 records sent, 377736.0 records/sec (3.60 MB/sec), 2.4 ms avg latency, 33.0 max latency.
1936880 records sent, 387376.0 records/sec (3.69 MB/sec), 2.4 ms avg latency, 31.0 max latency.
1863632 records sent, 372726.4 records/sec (3.55 MB/sec), 2.3 ms avg latency, 28.0 max latency.
104857600 records sent, 375724.610418 records/sec (3.58 MB/sec), 2.69 ms avg latency, 188.00 ms max latency, 2 ms 50th, 4 ms 95th, 19 ms 99th, 35 ms 99.9th.

100
1578705 records sent, 315741.0 records/sec (30.11 MB/sec), 3.1 ms avg latency, 182.0 max latency.
1869450 records sent, 373890.0 records/sec (35.66 MB/sec), 2.9 ms avg latency, 51.0 max latency.
1790382 records sent, 358076.4 records/sec (34.15 MB/sec), 3.1 ms avg latency, 49.0 max latency.
1906098 records sent, 381219.6 records/sec (36.36 MB/sec), 2.7 ms avg latency, 41.0 max latency.
1818981 records sent, 363796.2 records/sec (34.69 MB/sec), 2.6 ms avg latency, 35.0 max latency.
10485760 records sent, 360670.037492 records/sec (34.40 MB/sec), 2.87 ms avg latency, 182.00 ms max latency, 2 ms 50th, 5 ms 95th, 20 ms 99th, 36 ms 99.9th.

1000
553291 records sent, 110636.1 records/sec (105.51 MB/sec), 443.5 ms avg latency, 1129.0 max latency.
1048576 records sent, 113864.263221 records/sec (108.59 MB/sec), 500.01 ms avg latency, 1497.00 ms max latency, 351 ms 50th, 1400 ms 95th, 1465 ms 99th, 1491 ms 99.9th.

10000
59458 records sent, 11891.6 records/sec (113.41 MB/sec), 454.7 ms avg latency, 945.0 max latency.
104857 records sent, 12070.565212 records/sec (115.11 MB/sec), 481.89 ms avg latency, 1038.00 ms max latency, 553 ms 50th, 961 ms 95th, 1024 ms 99th, 1033 ms 99.9th.

100000
5971 records sent, 1194.2 records/sec (113.89 MB/sec), 375.9 ms avg latency, 1095.0 max latency.
10485 records sent, 1195.416714 records/sec (114.00 MB/sec), 413.71 ms avg latency, 1303.00 ms max latency, 214 ms 50th, 1260 ms 95th, 1298 ms 99th, 1301 ms 99.9th.
[ec2-user@ip-10-50-250-4 kafka_2.12-2.1.0]$

Replication_factor_3

[ec2-user@ip-10-50-250-4 kafka_2.12-2.1.0]$ bin/kafka-topics.sh --zookeeper 10.50.206.17:2181,10.50.210.77:2181,10.50.203.130:2181 --create --topic test-rep-two --partitions 6 --replication-factor 3
Created topic "test-rep-two".
[ec2-user@ip-10-50-250-4 kafka_2.12-2.1.0]$ bin/kafka-run-class.sh org.apache.kafka.tools.ProducerPerformance --print-metrics --topic test-rep-two --num-records 6000000 --throughput 100000 --record-size 100 --producer-props bootstrap.servers=b-1.test.ceh1c4.c1.kafka.us-east-1.amazonaws.com:9092,b-3.test.ceh1c4.c1.kafka.us-east-1.amazonaws.com:9092,b-2.test.ceh1c4.c1.kafka.us-east-1.amazonaws.com:9092 acks=1 buffer.memory=67108864 batch.size=8196
499686 records sent, 99937.2 records/sec (9.53 MB/sec), 4.4 ms avg latency, 186.0 max latency.
499996 records sent, 99999.2 records/sec (9.54 MB/sec), 2.8 ms avg latency, 31.0 max latency.
500120 records sent, 100024.0 records/sec (9.54 MB/sec), 2.9 ms avg latency, 44.0 max latency.
499913 records sent, 99982.6 records/sec (9.54 MB/sec), 3.0 ms avg latency, 40.0 max latency.
500033 records sent, 100006.6 records/sec (9.54 MB/sec), 2.9 ms avg latency, 36.0 max latency.
500010 records sent, 100002.0 records/sec (9.54 MB/sec), 2.5 ms avg latency, 25.0 max latency.
500043 records sent, 100008.6 records/sec (9.54 MB/sec), 2.3 ms avg latency, 31.0 max latency.
499973 records sent, 99994.6 records/sec (9.54 MB/sec), 2.2 ms avg latency, 25.0 max latency.
499989 records sent, 99997.8 records/sec (9.54 MB/sec), 2.0 ms avg latency, 29.0 max latency.
500033 records sent, 100006.6 records/sec (9.54 MB/sec), 1.9 ms avg latency, 25.0 max latency.
500073 records sent, 100014.6 records/sec (9.54 MB/sec), 1.9 ms avg latency, 28.0 max latency.
499786 records sent, 99957.2 records/sec (9.53 MB/sec), 1.9 ms avg latency, 28.0 max latency.
6000000 records sent, 99985.002250 records/sec (9.54 MB/sec), 2.56 ms avg latency, 186.00 ms max latency, 2 ms 50th, 5 ms 95th, 13 ms 99th, 28 ms 99.9th.

Metric Name                                                                            Value
app-info:commit-id:{client-id=producer-1}                                            : 809be928f1ae004e
app-info:version:{client-id=producer-1}                                              : 2.1.0
kafka-metrics-count:count:{client-id=producer-1}                                     : 142.000
producer-metrics:batch-size-avg:{client-id=producer-1}                               : 1047.886
producer-metrics:batch-size-max:{client-id=producer-1}                               : 8137.000
producer-metrics:batch-split-rate:{client-id=producer-1}                             : 0.000
producer-metrics:batch-split-total:{client-id=producer-1}                            : 0.000
producer-metrics:buffer-available-bytes:{client-id=producer-1}                       : 67108864.000
producer-metrics:buffer-exhausted-rate:{client-id=producer-1}                        : 0.000
producer-metrics:buffer-exhausted-total:{client-id=producer-1}                       : 0.000
producer-metrics:buffer-total-bytes:{client-id=producer-1}                           : 67108864.000
producer-metrics:bufferpool-wait-ratio:{client-id=producer-1}                        : 0.000
producer-metrics:bufferpool-wait-time-total:{client-id=producer-1}                   : 0.000
producer-metrics:compression-rate-avg:{client-id=producer-1}                         : 1.000
producer-metrics:connection-close-rate:{client-id=producer-1}                        : 0.000
producer-metrics:connection-close-total:{client-id=producer-1}                       : 0.000
producer-metrics:connection-count:{client-id=producer-1}                             : 6.000
producer-metrics:connection-creation-rate:{client-id=producer-1}                     : 0.000
producer-metrics:connection-creation-total:{client-id=producer-1}                    : 6.000
producer-metrics:failed-authentication-rate:{client-id=producer-1}                   : 0.000
producer-metrics:failed-authentication-total:{client-id=producer-1}                  : 0.000
producer-metrics:incoming-byte-rate:{client-id=producer-1}                           : 520945.341
producer-metrics:incoming-byte-total:{client-id=producer-1}                          : 31194207.000
producer-metrics:io-ratio:{client-id=producer-1}                                     : 0.114
producer-metrics:io-time-ns-avg:{client-id=producer-1}                               : 13994.123
producer-metrics:io-wait-ratio:{client-id=producer-1}                                : 0.615
producer-metrics:io-wait-time-ns-avg:{client-id=producer-1}                          : 75650.225
producer-metrics:io-waittime-total:{client-id=producer-1}                            : 36594304870.000
producer-metrics:iotime-total:{client-id=producer-1}                                 : 6668735751.000
producer-metrics:metadata-age:{client-id=producer-1}                                 : 59.873
producer-metrics:network-io-rate:{client-id=producer-1}                              : 11110.900
producer-metrics:network-io-total:{client-id=producer-1}                             : 665354.000
producer-metrics:outgoing-byte-rate:{client-id=producer-1}                           : 11985331.569
producer-metrics:outgoing-byte-total:{client-id=producer-1}                          : 717705625.000
producer-metrics:produce-throttle-time-avg:{client-id=producer-1}                    : 0.000
producer-metrics:produce-throttle-time-max:{client-id=producer-1}                    : 0.000
producer-metrics:record-error-rate:{client-id=producer-1}                            : 0.000
producer-metrics:record-error-total:{client-id=producer-1}                           : 0.000
producer-metrics:record-queue-time-avg:{client-id=producer-1}                        : 0.218
producer-metrics:record-queue-time-max:{client-id=producer-1}                        : 35.000
producer-metrics:record-retry-rate:{client-id=producer-1}                            : 0.000
producer-metrics:record-retry-total:{client-id=producer-1}                           : 0.000
producer-metrics:record-send-rate:{client-id=producer-1}                             : 100262.353
producer-metrics:record-send-total:{client-id=producer-1}                            : 6000000.000
producer-metrics:record-size-avg:{client-id=producer-1}                              : 186.000
producer-metrics:record-size-max:{client-id=producer-1}                              : 186.000
producer-metrics:records-per-request-avg:{client-id=producer-1}                      : 18.036
producer-metrics:request-latency-avg:{client-id=producer-1}                          : 1.809
producer-metrics:request-latency-max:{client-id=producer-1}                          : 43.000
producer-metrics:request-rate:{client-id=producer-1}                                 : 5555.450
producer-metrics:request-size-avg:{client-id=producer-1}                             : 2157.365
producer-metrics:request-size-max:{client-id=producer-1}                             : 16344.000
producer-metrics:request-total:{client-id=producer-1}                                : 332677.000
producer-metrics:requests-in-flight:{client-id=producer-1}                           : 0.000
producer-metrics:response-rate:{client-id=producer-1}                                : 5555.728
producer-metrics:response-total:{client-id=producer-1}                               : 332677.000
producer-metrics:select-rate:{client-id=producer-1}                                  : 8128.219
producer-metrics:select-total:{client-id=producer-1}                                 : 463565.000
producer-metrics:successful-authentication-rate:{client-id=producer-1}               : 0.000
producer-metrics:successful-authentication-total:{client-id=producer-1}              : 0.000
producer-metrics:waiting-threads:{client-id=producer-1}                              : 0.000
producer-node-metrics:incoming-byte-rate:{client-id=producer-1, node-id=node--1}     : 14.980
producer-node-metrics:incoming-byte-rate:{client-id=producer-1, node-id=node--2}     : 4.610
producer-node-metrics:incoming-byte-rate:{client-id=producer-1, node-id=node--3}     : 4.610
producer-node-metrics:incoming-byte-rate:{client-id=producer-1, node-id=node-1}      : 170555.726
producer-node-metrics:incoming-byte-rate:{client-id=producer-1, node-id=node-2}      : 182027.778
producer-node-metrics:incoming-byte-rate:{client-id=producer-1, node-id=node-3}      : 168698.942
producer-node-metrics:incoming-byte-total:{client-id=producer-1, node-id=node--1}    : 897.000
producer-node-metrics:incoming-byte-total:{client-id=producer-1, node-id=node--2}    : 276.000
producer-node-metrics:incoming-byte-total:{client-id=producer-1, node-id=node--3}    : 276.000
producer-node-metrics:incoming-byte-total:{client-id=producer-1, node-id=node-1}     : 10207078.000
producer-node-metrics:incoming-byte-total:{client-id=producer-1, node-id=node-2}     : 10890904.000
producer-node-metrics:incoming-byte-total:{client-id=producer-1, node-id=node-3}     : 10094776.000
producer-node-metrics:outgoing-byte-rate:{client-id=producer-1, node-id=node--1}     : 1.119
producer-node-metrics:outgoing-byte-rate:{client-id=producer-1, node-id=node--2}     : 0.401
producer-node-metrics:outgoing-byte-rate:{client-id=producer-1, node-id=node--3}     : 0.401
producer-node-metrics:outgoing-byte-rate:{client-id=producer-1, node-id=node-1}      : 3991061.976
producer-node-metrics:outgoing-byte-rate:{client-id=producer-1, node-id=node-2}      : 4014300.125
producer-node-metrics:outgoing-byte-rate:{client-id=producer-1, node-id=node-3}      : 3987310.953
producer-node-metrics:outgoing-byte-total:{client-id=producer-1, node-id=node--1}    : 67.000
producer-node-metrics:outgoing-byte-total:{client-id=producer-1, node-id=node--2}    : 24.000
producer-node-metrics:outgoing-byte-total:{client-id=producer-1, node-id=node--3}    : 24.000
producer-node-metrics:outgoing-byte-total:{client-id=producer-1, node-id=node-1}     : 238849095.000
producer-node-metrics:outgoing-byte-total:{client-id=producer-1, node-id=node-2}     : 240235791.000
producer-node-metrics:outgoing-byte-total:{client-id=producer-1, node-id=node-3}     : 238620624.000
producer-node-metrics:request-latency-avg:{client-id=producer-1, node-id=node--1}    : 0.000
producer-node-metrics:request-latency-avg:{client-id=producer-1, node-id=node--2}    : 0.000
producer-node-metrics:request-latency-avg:{client-id=producer-1, node-id=node--3}    : 0.000
producer-node-metrics:request-latency-avg:{client-id=producer-1, node-id=node-1}     : 1.850
producer-node-metrics:request-latency-avg:{client-id=producer-1, node-id=node-2}     : 1.668
producer-node-metrics:request-latency-avg:{client-id=producer-1, node-id=node-3}     : 1.920
producer-node-metrics:request-latency-max:{client-id=producer-1, node-id=node--1}    : -Infinity
producer-node-metrics:request-latency-max:{client-id=producer-1, node-id=node--2}    : -Infinity
producer-node-metrics:request-latency-max:{client-id=producer-1, node-id=node--3}    : -Infinity
producer-node-metrics:request-latency-max:{client-id=producer-1, node-id=node-1}     : 43.000
producer-node-metrics:request-latency-max:{client-id=producer-1, node-id=node-2}     : 36.000
producer-node-metrics:request-latency-max:{client-id=producer-1, node-id=node-3}     : 39.000
producer-node-metrics:request-rate:{client-id=producer-1, node-id=node--1}           : 0.033
producer-node-metrics:request-rate:{client-id=producer-1, node-id=node--2}           : 0.017
producer-node-metrics:request-rate:{client-id=producer-1, node-id=node--3}           : 0.017
producer-node-metrics:request-rate:{client-id=producer-1, node-id=node-1}            : 1817.368
producer-node-metrics:request-rate:{client-id=producer-1, node-id=node-2}            : 1944.875
producer-node-metrics:request-rate:{client-id=producer-1, node-id=node-3}            : 1796.544
producer-node-metrics:request-size-avg:{client-id=producer-1, node-id=node--1}       : 33.500
producer-node-metrics:request-size-avg:{client-id=producer-1, node-id=node--2}       : 24.000
producer-node-metrics:request-size-avg:{client-id=producer-1, node-id=node--3}       : 24.000
producer-node-metrics:request-size-avg:{client-id=producer-1, node-id=node-1}        : 2196.031
producer-node-metrics:request-size-avg:{client-id=producer-1, node-id=node-2}        : 2064.005
producer-node-metrics:request-size-avg:{client-id=producer-1, node-id=node-3}        : 2219.396
producer-node-metrics:request-size-max:{client-id=producer-1, node-id=node--1}       : 43.000
producer-node-metrics:request-size-max:{client-id=producer-1, node-id=node--2}       : 24.000
producer-node-metrics:request-size-max:{client-id=producer-1, node-id=node--3}       : 24.000
producer-node-metrics:request-size-max:{client-id=producer-1, node-id=node-1}        : 16344.000
producer-node-metrics:request-size-max:{client-id=producer-1, node-id=node-2}        : 16344.000
producer-node-metrics:request-size-max:{client-id=producer-1, node-id=node-3}        : 16344.000
producer-node-metrics:request-total:{client-id=producer-1, node-id=node--1}          : 2.000
producer-node-metrics:request-total:{client-id=producer-1, node-id=node--2}          : 1.000
producer-node-metrics:request-total:{client-id=producer-1, node-id=node--3}          : 1.000
producer-node-metrics:request-total:{client-id=producer-1, node-id=node-1}           : 108764.000
producer-node-metrics:request-total:{client-id=producer-1, node-id=node-2}           : 116393.000
producer-node-metrics:request-total:{client-id=producer-1, node-id=node-3}           : 107516.000
producer-node-metrics:response-rate:{client-id=producer-1, node-id=node--1}          : 0.033
producer-node-metrics:response-rate:{client-id=producer-1, node-id=node--2}          : 0.017
producer-node-metrics:response-rate:{client-id=producer-1, node-id=node--3}          : 0.017
producer-node-metrics:response-rate:{client-id=producer-1, node-id=node-1}           : 1817.398
producer-node-metrics:response-rate:{client-id=producer-1, node-id=node-2}           : 1945.363
producer-node-metrics:response-rate:{client-id=producer-1, node-id=node-3}           : 1796.755
producer-node-metrics:response-total:{client-id=producer-1, node-id=node--1}         : 2.000
producer-node-metrics:response-total:{client-id=producer-1, node-id=node--2}         : 1.000
producer-node-metrics:response-total:{client-id=producer-1, node-id=node--3}         : 1.000
producer-node-metrics:response-total:{client-id=producer-1, node-id=node-1}          : 108764.000
producer-node-metrics:response-total:{client-id=producer-1, node-id=node-2}          : 116393.000
producer-node-metrics:response-total:{client-id=producer-1, node-id=node-3}          : 107516.000
producer-topic-metrics:byte-rate:{client-id=producer-1, topic=test-rep-two}          : 11604552.288
producer-topic-metrics:byte-total:{client-id=producer-1, topic=test-rep-two}         : 694439618.000
producer-topic-metrics:compression-rate:{client-id=producer-1, topic=test-rep-two}   : 1.000
producer-topic-metrics:record-error-rate:{client-id=producer-1, topic=test-rep-two}  : 0.000
producer-topic-metrics:record-error-total:{client-id=producer-1, topic=test-rep-two} : 0.000
producer-topic-metrics:record-retry-rate:{client-id=producer-1, topic=test-rep-two}  : 0.000
producer-topic-metrics:record-retry-total:{client-id=producer-1, topic=test-rep-two} : 0.000
producer-topic-metrics:record-send-rate:{client-id=producer-1, topic=test-rep-two}   : 100264.029
producer-topic-metrics:record-send-total:{client-id=producer-1, topic=test-rep-two}  : 6000000.000

sync_rep_1

[ec2-user@ip-10-50-250-4 kafka_2.12-2.1.0]$ bin/kafka-producer-perf-test.sh \
>   --topic test-rep-one \
>   --num-records 50000000 \
>   --record-size 100 \
>   --throughput -1 \
>   --producer-props acks=1 \
>   bootstrap.servers=b-1.test.ceh1c4.c1.kafka.us-east-1.amazonaws.com:9092,b-3.test.ceh1c4.c1.kafka.us-east-1.amazonaws.com:9092,b-2.test.ceh1c4.c1.kafka.us-east-1.amazonaws.com:9092 \
>   buffer.memory=67108864 \
>   batch.size=8196
1729565 records sent, 345913.0 records/sec (32.99 MB/sec), 1.7 ms avg latency, 185.0 max latency.
1896232 records sent, 379246.4 records/sec (36.17 MB/sec), 1.4 ms avg latency, 25.0 max latency.
1880300 records sent, 376060.0 records/sec (35.86 MB/sec), 1.4 ms avg latency, 26.0 max latency.
1793605 records sent, 358721.0 records/sec (34.21 MB/sec), 1.3 ms avg latency, 25.0 max latency.
1888272 records sent, 377654.4 records/sec (36.02 MB/sec), 1.4 ms avg latency, 27.0 max latency.
1791640 records sent, 358328.0 records/sec (34.17 MB/sec), 1.3 ms avg latency, 20.0 max latency.
1889377 records sent, 377875.4 records/sec (36.04 MB/sec), 1.3 ms avg latency, 21.0 max latency.
1888423 records sent, 377684.6 records/sec (36.02 MB/sec), 1.3 ms avg latency, 21.0 max latency.
1767456 records sent, 353491.2 records/sec (33.71 MB/sec), 1.2 ms avg latency, 21.0 max latency.
1685451 records sent, 337090.2 records/sec (32.15 MB/sec), 1.2 ms avg latency, 21.0 max latency.
1889489 records sent, 377897.8 records/sec (36.04 MB/sec), 1.3 ms avg latency, 23.0 max latency.
1885835 records sent, 377167.0 records/sec (35.97 MB/sec), 1.3 ms avg latency, 24.0 max latency.
1892982 records sent, 378596.4 records/sec (36.11 MB/sec), 2.1 ms avg latency, 109.0 max latency.
1882984 records sent, 376596.8 records/sec (35.92 MB/sec), 1.4 ms avg latency, 20.0 max latency.
1642416 records sent, 328483.2 records/sec (31.33 MB/sec), 1.2 ms avg latency, 23.0 max latency.
1870688 records sent, 374137.6 records/sec (35.68 MB/sec), 1.3 ms avg latency, 21.0 max latency.
1888282 records sent, 377656.4 records/sec (36.02 MB/sec), 1.3 ms avg latency, 22.0 max latency.
1803216 records sent, 360643.2 records/sec (34.39 MB/sec), 1.4 ms avg latency, 27.0 max latency.
1893813 records sent, 378762.6 records/sec (36.12 MB/sec), 1.3 ms avg latency, 21.0 max latency.
1882072 records sent, 376414.4 records/sec (35.90 MB/sec), 1.3 ms avg latency, 23.0 max latency.
1879131 records sent, 375826.2 records/sec (35.84 MB/sec), 1.3 ms avg latency, 22.0 max latency.
1575082 records sent, 315016.4 records/sec (30.04 MB/sec), 1.2 ms avg latency, 21.0 max latency.
1823870 records sent, 364774.0 records/sec (34.79 MB/sec), 1.3 ms avg latency, 23.0 max latency.
1619056 records sent, 323811.2 records/sec (30.88 MB/sec), 1.1 ms avg latency, 22.0 max latency.
1885929 records sent, 377185.8 records/sec (35.97 MB/sec), 1.3 ms avg latency, 24.0 max latency.
1883026 records sent, 376605.2 records/sec (35.92 MB/sec), 1.4 ms avg latency, 24.0 max latency.
1848844 records sent, 369768.8 records/sec (35.26 MB/sec), 1.4 ms avg latency, 25.0 max latency.
50000000 records sent, 365032.779944 records/sec (34.81 MB/sec), 1.36 ms avg latency, 185.00 ms max latency, 1 ms 50th, 2 ms 95th, 4 ms 99th, 20 ms 99.9th.

test-rep-one

[ec2-user@ip-10-50-250-4 kafka_2.12-2.1.0]$ bin/kafka-run-class.sh org.apache.kafka.tools.ProducerPerformance --print-metrics --topic test-rep-one --num-records 6000000 --throughput 100000 --record-size 100 --producer-props bootstrap.servers=b-1.test.ceh1c4.c1.kafka.us-east-1.amazonaws.com:9092,b-3.test.ceh1c4.c1.kafka.us-east-1.amazonaws.com:9092,b-2.test.ceh1c4.c1.kafka.us-east-1.amazonaws.com:9092 buffer.memory=67108864 batch.size=8196


499844 records sent, 99948.8 records/sec (9.53 MB/sec), 2.9 ms avg latency, 206.0 max latency.
500306 records sent, 100061.2 records/sec (9.54 MB/sec), 1.0 ms avg latency, 20.0 max latency.
499890 records sent, 99978.0 records/sec (9.53 MB/sec), 0.9 ms avg latency, 15.0 max latency.
500062 records sent, 100012.4 records/sec (9.54 MB/sec), 1.0 ms avg latency, 16.0 max latency.
500045 records sent, 100009.0 records/sec (9.54 MB/sec), 1.0 ms avg latency, 10.0 max latency.
499988 records sent, 99997.6 records/sec (9.54 MB/sec), 1.0 ms avg latency, 17.0 max latency.
500067 records sent, 99993.4 records/sec (9.54 MB/sec), 0.9 ms avg latency, 10.0 max latency.
500040 records sent, 100008.0 records/sec (9.54 MB/sec), 0.9 ms avg latency, 17.0 max latency.
500017 records sent, 100003.4 records/sec (9.54 MB/sec), 0.9 ms avg latency, 15.0 max latency.
499955 records sent, 99991.0 records/sec (9.54 MB/sec), 0.9 ms avg latency, 16.0 max latency.
500088 records sent, 100017.6 records/sec (9.54 MB/sec), 0.9 ms avg latency, 18.0 max latency.
6000000 records sent, 99996.666778 records/sec (9.54 MB/sec), 1.12 ms avg latency, 206.00 ms max latency, 1 ms 50th, 2 ms 95th, 4 ms 99th, 32 ms 99.9th.

Metric Name                                                                            Value
app-info:commit-id:{client-id=producer-1}                                            : 809be928f1ae004e
app-info:version:{client-id=producer-1}                                              : 2.1.0
kafka-metrics-count:count:{client-id=producer-1}                                     : 142.000
producer-metrics:batch-size-avg:{client-id=producer-1}                               : 803.569
producer-metrics:batch-size-max:{client-id=producer-1}                               : 8137.000
producer-metrics:batch-split-rate:{client-id=producer-1}                             : 0.000
producer-metrics:batch-split-total:{client-id=producer-1}                            : 0.000
producer-metrics:buffer-available-bytes:{client-id=producer-1}                       : 67108864.000
producer-metrics:buffer-exhausted-rate:{client-id=producer-1}                        : 0.000
producer-metrics:buffer-exhausted-total:{client-id=producer-1}                       : 0.000
producer-metrics:buffer-total-bytes:{client-id=producer-1}                           : 67108864.000
producer-metrics:bufferpool-wait-ratio:{client-id=producer-1}                        : 0.000
producer-metrics:bufferpool-wait-time-total:{client-id=producer-1}                   : 0.000
producer-metrics:compression-rate-avg:{client-id=producer-1}                         : 1.000
producer-metrics:connection-close-rate:{client-id=producer-1}                        : 0.000
producer-metrics:connection-close-total:{client-id=producer-1}                       : 0.000
producer-metrics:connection-count:{client-id=producer-1}                             : 6.000
producer-metrics:connection-creation-rate:{client-id=producer-1}                     : 0.100
producer-metrics:connection-creation-total:{client-id=producer-1}                    : 6.000
producer-metrics:failed-authentication-rate:{client-id=producer-1}                   : 0.000
producer-metrics:failed-authentication-total:{client-id=producer-1}                  : 0.000
producer-metrics:incoming-byte-rate:{client-id=producer-1}                           : 692150.147
producer-metrics:incoming-byte-total:{client-id=producer-1}                          : 41437645.000
producer-metrics:io-ratio:{client-id=producer-1}                                     : 0.135
producer-metrics:io-time-ns-avg:{client-id=producer-1}                               : 17219.919
producer-metrics:io-wait-ratio:{client-id=producer-1}                                : 0.563
producer-metrics:io-wait-time-ns-avg:{client-id=producer-1}                          : 72027.694
producer-metrics:io-waittime-total:{client-id=producer-1}                            : 33259022567.000
producer-metrics:iotime-total:{client-id=producer-1}                                 : 7945539489.000
producer-metrics:metadata-age:{client-id=producer-1}                                 : 59.861
producer-metrics:network-io-rate:{client-id=producer-1}                              : 14750.735
producer-metrics:network-io-total:{client-id=producer-1}                             : 883156.000
producer-metrics:outgoing-byte-rate:{client-id=producer-1}                           : 12336898.331
producer-metrics:outgoing-byte-total:{client-id=producer-1}                          : 738622440.000
producer-metrics:produce-throttle-time-avg:{client-id=producer-1}                    : 0.000
producer-metrics:produce-throttle-time-max:{client-id=producer-1}                    : 0.000
producer-metrics:record-error-rate:{client-id=producer-1}                            : 0.000
producer-metrics:record-error-total:{client-id=producer-1}                           : 0.000
producer-metrics:record-queue-time-avg:{client-id=producer-1}                        : 0.124
producer-metrics:record-queue-time-max:{client-id=producer-1}                        : 24.000
producer-metrics:record-retry-rate:{client-id=producer-1}                            : 0.000
producer-metrics:record-retry-total:{client-id=producer-1}                           : 0.000
producer-metrics:record-send-rate:{client-id=producer-1}                             : 100282.462
producer-metrics:record-send-total:{client-id=producer-1}                            : 6000000.000
producer-metrics:record-size-avg:{client-id=producer-1}                              : 186.000
producer-metrics:record-size-max:{client-id=producer-1}                              : 186.000
producer-metrics:records-per-request-avg:{client-id=producer-1}                      : 13.588
producer-metrics:request-latency-avg:{client-id=producer-1}                          : 0.785
producer-metrics:request-latency-max:{client-id=producer-1}                          : 34.000
producer-metrics:request-rate:{client-id=producer-1}                                 : 7375.367
producer-metrics:request-size-avg:{client-id=producer-1}                             : 1672.688
producer-metrics:request-size-max:{client-id=producer-1}                             : 16344.000
producer-metrics:request-total:{client-id=producer-1}                                : 441578.000
producer-metrics:requests-in-flight:{client-id=producer-1}                           : 0.000
producer-metrics:response-rate:{client-id=producer-1}                                : 7375.860
producer-metrics:response-total:{client-id=producer-1}                               : 441578.000
producer-metrics:select-rate:{client-id=producer-1}                                  : 7820.015
producer-metrics:select-total:{client-id=producer-1}                                 : 457874.000
producer-metrics:successful-authentication-rate:{client-id=producer-1}               : 0.000
producer-metrics:successful-authentication-total:{client-id=producer-1}              : 0.000
producer-metrics:waiting-threads:{client-id=producer-1}                              : 0.000
producer-node-metrics:incoming-byte-rate:{client-id=producer-1, node-id=node--1}     : 4.611
producer-node-metrics:incoming-byte-rate:{client-id=producer-1, node-id=node--2}     : 4.610
producer-node-metrics:incoming-byte-rate:{client-id=producer-1, node-id=node--3}     : 13.380
producer-node-metrics:incoming-byte-rate:{client-id=producer-1, node-id=node-1}      : 208924.401
producer-node-metrics:incoming-byte-rate:{client-id=producer-1, node-id=node-2}      : 286130.791
producer-node-metrics:incoming-byte-rate:{client-id=producer-1, node-id=node-3}      : 197493.348
producer-node-metrics:incoming-byte-total:{client-id=producer-1, node-id=node--1}    : 276.000
producer-node-metrics:incoming-byte-total:{client-id=producer-1, node-id=node--2}    : 276.000
producer-node-metrics:incoming-byte-total:{client-id=producer-1, node-id=node--3}    : 801.000
producer-node-metrics:incoming-byte-total:{client-id=producer-1, node-id=node-1}     : 12499738.000
producer-node-metrics:incoming-byte-total:{client-id=producer-1, node-id=node-2}     : 17120922.000
producer-node-metrics:incoming-byte-total:{client-id=producer-1, node-id=node-3}     : 11815632.000
producer-node-metrics:outgoing-byte-rate:{client-id=producer-1, node-id=node--1}     : 0.401
producer-node-metrics:outgoing-byte-rate:{client-id=producer-1, node-id=node--2}     : 0.401
producer-node-metrics:outgoing-byte-rate:{client-id=producer-1, node-id=node--3}     : 1.119
producer-node-metrics:outgoing-byte-rate:{client-id=producer-1, node-id=node-1}      : 4069985.109
producer-node-metrics:outgoing-byte-rate:{client-id=producer-1, node-id=node-2}      : 4227574.454
producer-node-metrics:outgoing-byte-rate:{client-id=producer-1, node-id=node-3}      : 4046688.744
producer-node-metrics:outgoing-byte-total:{client-id=producer-1, node-id=node--1}    : 24.000
producer-node-metrics:outgoing-byte-total:{client-id=producer-1, node-id=node--2}    : 24.000
producer-node-metrics:outgoing-byte-total:{client-id=producer-1, node-id=node--3}    : 67.000
producer-node-metrics:outgoing-byte-total:{client-id=producer-1, node-id=node-1}     : 243527559.000
producer-node-metrics:outgoing-byte-total:{client-id=producer-1, node-id=node-2}     : 252961145.000
producer-node-metrics:outgoing-byte-total:{client-id=producer-1, node-id=node-3}     : 242133621.000
producer-node-metrics:request-latency-avg:{client-id=producer-1, node-id=node--1}    : 0.000
producer-node-metrics:request-latency-avg:{client-id=producer-1, node-id=node--2}    : 0.000
producer-node-metrics:request-latency-avg:{client-id=producer-1, node-id=node--3}    : 0.000
producer-node-metrics:request-latency-avg:{client-id=producer-1, node-id=node-1}     : 0.876
producer-node-metrics:request-latency-avg:{client-id=producer-1, node-id=node-2}     : 0.584
producer-node-metrics:request-latency-avg:{client-id=producer-1, node-id=node-3}     : 0.980
producer-node-metrics:request-latency-max:{client-id=producer-1, node-id=node--1}    : -Infinity
producer-node-metrics:request-latency-max:{client-id=producer-1, node-id=node--2}    : -Infinity
producer-node-metrics:request-latency-max:{client-id=producer-1, node-id=node--3}    : -Infinity
producer-node-metrics:request-latency-max:{client-id=producer-1, node-id=node-1}     : 34.000
producer-node-metrics:request-latency-max:{client-id=producer-1, node-id=node-2}     : 34.000
producer-node-metrics:request-latency-max:{client-id=producer-1, node-id=node-3}     : 34.000
producer-node-metrics:request-rate:{client-id=producer-1, node-id=node--1}           : 0.017
producer-node-metrics:request-rate:{client-id=producer-1, node-id=node--2}           : 0.017
producer-node-metrics:request-rate:{client-id=producer-1, node-id=node--3}           : 0.033
producer-node-metrics:request-rate:{client-id=producer-1, node-id=node-1}            : 2226.068
producer-node-metrics:request-rate:{client-id=producer-1, node-id=node-2}            : 3051.456
producer-node-metrics:request-rate:{client-id=producer-1, node-id=node-3}            : 2102.163
producer-node-metrics:request-size-avg:{client-id=producer-1, node-id=node--1}       : 24.000
producer-node-metrics:request-size-avg:{client-id=producer-1, node-id=node--2}       : 24.000
producer-node-metrics:request-size-avg:{client-id=producer-1, node-id=node--3}       : 33.500
producer-node-metrics:request-size-avg:{client-id=producer-1, node-id=node-1}        : 1828.299
producer-node-metrics:request-size-avg:{client-id=producer-1, node-id=node-2}        : 1385.405
producer-node-metrics:request-size-avg:{client-id=producer-1, node-id=node-3}        : 1924.980
producer-node-metrics:request-size-max:{client-id=producer-1, node-id=node--1}       : 24.000
producer-node-metrics:request-size-max:{client-id=producer-1, node-id=node--2}       : 24.000
producer-node-metrics:request-size-max:{client-id=producer-1, node-id=node--3}       : 43.000
producer-node-metrics:request-size-max:{client-id=producer-1, node-id=node-1}        : 16344.000
producer-node-metrics:request-size-max:{client-id=producer-1, node-id=node-2}        : 16344.000
producer-node-metrics:request-size-max:{client-id=producer-1, node-id=node-3}        : 16344.000
producer-node-metrics:request-total:{client-id=producer-1, node-id=node--1}          : 1.000
producer-node-metrics:request-total:{client-id=producer-1, node-id=node--2}          : 1.000
producer-node-metrics:request-total:{client-id=producer-1, node-id=node--3}          : 2.000
producer-node-metrics:request-total:{client-id=producer-1, node-id=node-1}           : 133199.000
producer-node-metrics:request-total:{client-id=producer-1, node-id=node-2}           : 182590.000
producer-node-metrics:request-total:{client-id=producer-1, node-id=node-3}           : 125785.000
producer-node-metrics:response-rate:{client-id=producer-1, node-id=node--1}          : 0.017
producer-node-metrics:response-rate:{client-id=producer-1, node-id=node--2}          : 0.017
producer-node-metrics:response-rate:{client-id=producer-1, node-id=node--3}          : 0.033
producer-node-metrics:response-rate:{client-id=producer-1, node-id=node-1}           : 2226.328
producer-node-metrics:response-rate:{client-id=producer-1, node-id=node-2}           : 3051.507
producer-node-metrics:response-rate:{client-id=producer-1, node-id=node-3}           : 2102.444
producer-node-metrics:response-total:{client-id=producer-1, node-id=node--1}         : 1.000
producer-node-metrics:response-total:{client-id=producer-1, node-id=node--2}         : 1.000
producer-node-metrics:response-total:{client-id=producer-1, node-id=node--3}         : 2.000
producer-node-metrics:response-total:{client-id=producer-1, node-id=node-1}          : 133199.000
producer-node-metrics:response-total:{client-id=producer-1, node-id=node-2}          : 182590.000
producer-node-metrics:response-total:{client-id=producer-1, node-id=node-3}          : 125785.000
producer-topic-metrics:byte-rate:{client-id=producer-1, topic=test-rep-one}          : 11829041.267
producer-topic-metrics:byte-total:{client-id=producer-1, topic=test-rep-one}         : 707731539.000
producer-topic-metrics:compression-rate:{client-id=producer-1, topic=test-rep-one}   : 1.000
producer-topic-metrics:record-error-rate:{client-id=producer-1, topic=test-rep-one}  : 0.000
producer-topic-metrics:record-error-total:{client-id=producer-1, topic=test-rep-one} : 0.000
producer-topic-metrics:record-retry-rate:{client-id=producer-1, topic=test-rep-one}  : 0.000
producer-topic-metrics:record-retry-total:{client-id=producer-1, topic=test-rep-one} : 0.000
producer-topic-metrics:record-send-rate:{client-id=producer-1, topic=test-rep-one}   : 100282.462
producer-topic-metrics:record-send-total:{client-id=producer-1, topic=test-rep-one}  : 6000000.000