Skip to content

Instantly share code, notes, and snippets.

@conniec

conniec/application logs with same application id and increased `state.cleanup.delay.ms`

Last active July 17, 2017 21:30
Show Gist options
  • Save conniec/baf0c011be8f29a4e09af0ceb136e33e to your computer and use it in GitHub Desktop.
Save conniec/baf0c011be8f29a4e09af0ceb136e33e to your computer and use it in GitHub Desktop.
Download ZIP
kafka-streams description
Raw
application logs with same application id and increased `state.cleanup.delay.ms`
14:28:45.733 [kafka-streams-test-service-akka.actor.default-dispatcher-4] INFO akka.event.slf4j.Slf4jLogger - Slf4jLogger started
14:28:45.971 [main] INFO o.apache.kafka.streams.StreamsConfig - StreamsConfig values:
application.id = map-function-scala-example
application.server =
bootstrap.servers = [localhost:9092]
buffered.records.per.partition = 1000
cache.max.bytes.buffering = 10485760
client.id =
commit.interval.ms = 30000
connections.max.idle.ms = 540000
key.serde = class org.apache.kafka.common.serialization.Serdes$StringSerde
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
num.standby.replicas = 0
num.stream.threads = 1
partition.grouper = class org.apache.kafka.streams.processor.DefaultPartitionGrouper
poll.ms = 100
receive.buffer.bytes = 32768
reconnect.backoff.ms = 50
replication.factor = 1
request.timeout.ms = 40000
retry.backoff.ms = 100
rocksdb.config.setter = null
security.protocol = PLAINTEXT
send.buffer.bytes = 131072
state.cleanup.delay.ms = 60000
state.dir = /tmp/kafka-streams
timestamp.extractor = class org.apache.kafka.streams.processor.FailOnInvalidTimestamp
value.serde = class org.apache.kafka.common.serialization.Serdes$StringSerde
windowstore.changelog.additional.retention.ms = 86400000
zookeeper.connect =
14:28:46.113 [main] INFO o.a.k.c.consumer.ConsumerConfig - ConsumerConfig values:
auto.commit.interval.ms = 5000
auto.offset.reset = earliest
bootstrap.servers = [localhost:9092]
check.crcs = true
client.id = map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-global-restore-consumer
connections.max.idle.ms = 540000
enable.auto.commit = false
exclude.internal.topics = true
fetch.max.bytes = 52428800
fetch.max.wait.ms = 500
fetch.min.bytes = 1
group.id =
heartbeat.interval.ms = 3000
interceptor.classes = null
internal.leave.group.on.close = false
key.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer
max.partition.fetch.bytes = 1048576
max.poll.interval.ms = 2147483647
max.poll.records = 1000
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
partition.assignment.strategy = [class org.apache.kafka.clients.consumer.RangeAssignor]
receive.buffer.bytes = 65536
reconnect.backoff.ms = 50
request.timeout.ms = 305000
retry.backoff.ms = 100
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
send.buffer.bytes = 131072
session.timeout.ms = 10000
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = null
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
value.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer
14:28:46.114 [main] DEBUG o.a.k.clients.consumer.KafkaConsumer - Starting the Kafka consumer
14:28:46.135 [main] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 1 to Cluster(id = null, nodes = [localhost:9092 (id: -1 rack: null)], partitions = [])
14:28:46.171 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-closed:
14:28:46.173 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-created:
14:28:46.174 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent-received:
14:28:46.174 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent:
14:28:46.176 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-received:
14:28:46.176 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name select-time:
14:28:46.177 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name io-time:
14:28:46.219 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name heartbeat-latency
14:28:46.220 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name join-latency
14:28:46.220 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name sync-latency
14:28:46.223 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name commit-latency
14:28:46.241 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-fetched
14:28:46.242 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name records-fetched
14:28:46.242 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name fetch-latency
14:28:46.243 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name records-lag
14:28:46.243 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name fetch-throttle-time
14:28:46.245 [main] INFO o.a.kafka.common.utils.AppInfoParser - Kafka version : 0.10.2.1-cp2
14:28:46.245 [main] INFO o.a.kafka.common.utils.AppInfoParser - Kafka commitId : 5092731dfc53f849
14:28:46.245 [main] DEBUG o.a.k.clients.consumer.KafkaConsumer - Kafka consumer created
14:28:46.267 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name thread.map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1.commit-latency
14:28:46.268 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name thread.map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1.poll-latency
14:28:46.270 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name thread.map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1.process-latency
14:28:46.271 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name thread.map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1.punctuate-latency
14:28:46.271 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name thread.map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1.task-created
14:28:46.271 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name thread.map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1.task-closed
14:28:46.271 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name thread.map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1.skipped-records
14:28:46.272 [main] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] Creating producer client
14:28:46.284 [main] INFO o.a.k.c.producer.ProducerConfig - ProducerConfig values:
acks = 1
batch.size = 16384
block.on.buffer.full = false
bootstrap.servers = [localhost:9092]
buffer.memory = 33554432
client.id = map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1-producer
compression.type = none
connections.max.idle.ms = 540000
interceptor.classes = null
key.serializer = class org.apache.kafka.common.serialization.ByteArraySerializer
linger.ms = 100
max.block.ms = 60000
max.in.flight.requests.per.connection = 5
max.request.size = 1048576
metadata.fetch.timeout.ms = 60000
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.sample.window.ms = 30000
partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner
receive.buffer.bytes = 32768
reconnect.backoff.ms = 50
request.timeout.ms = 30000
retries = 10
retry.backoff.ms = 100
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
send.buffer.bytes = 131072
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = null
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
timeout.ms = 30000
value.serializer = class org.apache.kafka.common.serialization.ByteArraySerializer
14:28:46.292 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bufferpool-wait-time
14:28:46.297 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name buffer-exhausted-records
14:28:46.297 [main] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 1 to Cluster(id = null, nodes = [localhost:9092 (id: -1 rack: null)], partitions = [])
14:28:46.298 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-closed:
14:28:46.298 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-created:
14:28:46.298 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent-received:
14:28:46.298 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent:
14:28:46.298 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-received:
14:28:46.298 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name select-time:
14:28:46.298 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name io-time:
14:28:46.305 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name batch-size
14:28:46.305 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name compression-rate
14:28:46.305 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name queue-time
14:28:46.306 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name request-time
14:28:46.306 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name produce-throttle-time
14:28:46.306 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name records-per-request
14:28:46.306 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name record-retries
14:28:46.307 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name errors
14:28:46.307 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name record-size-max
14:28:46.311 [main] INFO o.a.kafka.common.utils.AppInfoParser - Kafka version : 0.10.2.1-cp2
14:28:46.311 [kafka-producer-network-thread | map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1-producer] DEBUG o.a.k.c.producer.internals.Sender - Starting Kafka producer I/O thread.
14:28:46.311 [main] INFO o.a.kafka.common.utils.AppInfoParser - Kafka commitId : 5092731dfc53f849
14:28:46.311 [main] DEBUG o.a.k.clients.producer.KafkaProducer - Kafka producer started
14:28:46.311 [main] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] Creating consumer client
14:28:46.314 [main] INFO o.a.k.c.consumer.ConsumerConfig - ConsumerConfig values:
auto.commit.interval.ms = 5000
auto.offset.reset = earliest
bootstrap.servers = [localhost:9092]
check.crcs = true
client.id = map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1-consumer
connections.max.idle.ms = 540000
enable.auto.commit = false
exclude.internal.topics = true
fetch.max.bytes = 52428800
fetch.max.wait.ms = 500
fetch.min.bytes = 1
group.id = map-function-scala-example
heartbeat.interval.ms = 3000
interceptor.classes = null
internal.leave.group.on.close = false
key.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer
max.partition.fetch.bytes = 1048576
max.poll.interval.ms = 2147483647
max.poll.records = 1000
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
partition.assignment.strategy = [org.apache.kafka.streams.processor.internals.StreamPartitionAssignor]
receive.buffer.bytes = 65536
reconnect.backoff.ms = 50
request.timeout.ms = 305000
retry.backoff.ms = 100
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
send.buffer.bytes = 131072
session.timeout.ms = 10000
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = null
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
value.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer
14:28:46.314 [main] DEBUG o.a.k.clients.consumer.KafkaConsumer - Starting the Kafka consumer
14:28:46.315 [main] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 1 to Cluster(id = null, nodes = [localhost:9092 (id: -1 rack: null)], partitions = [])
14:28:46.315 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-closed:
14:28:46.315 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-created:
14:28:46.315 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent-received:
14:28:46.315 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent:
14:28:46.316 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-received:
14:28:46.316 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name select-time:
14:28:46.316 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name io-time:
14:28:46.323 [main] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 1 to Cluster(id = null, nodes = [localhost:9092 (id: -1 rack: null)], partitions = [])
14:28:46.323 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-closed:
14:28:46.323 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-created:
14:28:46.323 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent-received:
14:28:46.323 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent:
14:28:46.324 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-received:
14:28:46.324 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name select-time:
14:28:46.324 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name io-time:
14:28:46.325 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name heartbeat-latency
14:28:46.325 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name join-latency
14:28:46.325 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name sync-latency
14:28:46.325 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name commit-latency
14:28:46.326 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-fetched
14:28:46.326 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name records-fetched
14:28:46.326 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name fetch-latency
14:28:46.326 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name records-lag
14:28:46.326 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name fetch-throttle-time
14:28:46.326 [main] INFO o.a.kafka.common.utils.AppInfoParser - Kafka version : 0.10.2.1-cp2
14:28:46.327 [main] INFO o.a.kafka.common.utils.AppInfoParser - Kafka commitId : 5092731dfc53f849
14:28:46.327 [main] DEBUG o.a.k.clients.consumer.KafkaConsumer - Kafka consumer created
14:28:46.327 [main] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] Creating restore consumer client
14:28:46.327 [main] INFO o.a.k.c.consumer.ConsumerConfig - ConsumerConfig values:
auto.commit.interval.ms = 5000
auto.offset.reset = earliest
bootstrap.servers = [localhost:9092]
check.crcs = true
client.id = map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1-restore-consumer
connections.max.idle.ms = 540000
enable.auto.commit = false
exclude.internal.topics = true
fetch.max.bytes = 52428800
fetch.max.wait.ms = 500
fetch.min.bytes = 1
group.id =
heartbeat.interval.ms = 3000
interceptor.classes = null
internal.leave.group.on.close = false
key.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer
max.partition.fetch.bytes = 1048576
max.poll.interval.ms = 2147483647
max.poll.records = 1000
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
partition.assignment.strategy = [class org.apache.kafka.clients.consumer.RangeAssignor]
receive.buffer.bytes = 65536
reconnect.backoff.ms = 50
request.timeout.ms = 305000
retry.backoff.ms = 100
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
send.buffer.bytes = 131072
session.timeout.ms = 10000
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = null
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
value.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer
14:28:46.327 [main] DEBUG o.a.k.clients.consumer.KafkaConsumer - Starting the Kafka consumer
14:28:46.327 [main] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 1 to Cluster(id = null, nodes = [localhost:9092 (id: -1 rack: null)], partitions = [])
14:28:46.327 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-closed:
14:28:46.327 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-created:
14:28:46.328 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent-received:
14:28:46.328 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent:
14:28:46.328 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-received:
14:28:46.328 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name select-time:
14:28:46.328 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name io-time:
14:28:46.329 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name heartbeat-latency
14:28:46.329 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name join-latency
14:28:46.329 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name sync-latency
14:28:46.329 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name commit-latency
14:28:46.330 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-fetched
14:28:46.330 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name records-fetched
14:28:46.330 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name fetch-latency
14:28:46.331 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name records-lag
14:28:46.331 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name fetch-throttle-time
14:28:46.331 [main] INFO o.a.kafka.common.utils.AppInfoParser - Kafka version : 0.10.2.1-cp2
14:28:46.331 [main] INFO o.a.kafka.common.utils.AppInfoParser - Kafka commitId : 5092731dfc53f849
14:28:46.331 [main] DEBUG o.a.k.clients.consumer.KafkaConsumer - Kafka consumer created
14:28:46.331 [main] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] State transition from NOT_RUNNING to RUNNING.
14:28:46.335 [main] DEBUG o.apache.kafka.streams.KafkaStreams - stream-client [map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191] Starting Kafka Stream process.
14:28:46.336 [main] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 1 to Cluster(id = null, nodes = [localhost:9092 (id: -1 rack: null)], partitions = [])
14:28:46.336 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-closed:
14:28:46.336 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-created:
14:28:46.336 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent-received:
14:28:46.336 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent:
14:28:46.336 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-received:
14:28:46.337 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name select-time:
14:28:46.337 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name io-time:
14:28:46.337 [main] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 1 to Cluster(id = null, nodes = [localhost:9092 (id: -1 rack: null)], partitions = [])
14:28:46.337 [main] DEBUG o.apache.kafka.clients.NetworkClient - Initiating connection to node -1 at localhost:9092.
14:28:46.404 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--1.bytes-sent
14:28:46.405 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--1.bytes-received
14:28:46.405 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--1.latency
14:28:46.408 [main] DEBUG o.a.kafka.common.network.Selector - Created socket with SO_RCVBUF = 326640, SO_SNDBUF = 146988, SO_TIMEOUT = 0 to node -1
14:28:46.408 [main] DEBUG o.apache.kafka.clients.NetworkClient - Completed connection to node -1. Fetching API versions.
14:28:46.409 [main] DEBUG o.apache.kafka.clients.NetworkClient - Initiating API versions fetch from node -1.
14:28:46.495 [main] DEBUG o.apache.kafka.clients.NetworkClient - Recorded API versions for node -1: (Produce(0): 0 to 2 [usable: 2], Fetch(1): 0 to 3 [usable: 3], Offsets(2): 0 to 1 [usable: 1], Metadata(3): 0 to 2 [usable: 2], LeaderAndIsr(4): 0 [usable: 0], StopReplica(5): 0 [usable: 0], UpdateMetadata(6): 0 to 3 [usable: 3], ControlledShutdown(7): 1 [usable: 1], OffsetCommit(8): 0 to 2 [usable: 2], OffsetFetch(9): 0 to 2 [usable: 2], GroupCoordinator(10): 0 [usable: 0], JoinGroup(11): 0 to 1 [usable: 1], Heartbeat(12): 0 [usable: 0], LeaveGroup(13): 0 [usable: 0], SyncGroup(14): 0 [usable: 0], DescribeGroups(15): 0 [usable: 0], ListGroups(16): 0 [usable: 0], SaslHandshake(17): 0 [usable: 0], ApiVersions(18): 0 [usable: 0], CreateTopics(19): 0 to 1 [usable: 1], DeleteTopics(20): 0 [usable: 0])
14:28:46.609 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name connections-closed:
14:28:46.609 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name connections-created:
14:28:46.609 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name bytes-sent-received:
14:28:46.610 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name bytes-sent:
14:28:46.610 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name bytes-received:
14:28:46.610 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name select-time:
14:28:46.610 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name io-time:
14:28:46.611 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name node--1.bytes-sent
14:28:46.611 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name node--1.bytes-received
14:28:46.611 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name node--1.latency
14:28:46.611 [main] INFO o.apache.kafka.streams.KafkaStreams - stream-client [map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191] State transition from CREATED to RUNNING.
14:28:46.625 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name put
14:28:46.626 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name test-store-put
14:28:46.626 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name put-if-absent
14:28:46.626 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name test-store-put-if-absent
14:28:46.627 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name get
14:28:46.627 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name test-store-get
14:28:46.627 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name delete
14:28:46.627 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name test-store-delete
14:28:46.628 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name put-all
14:28:46.628 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name test-store-put-all
14:28:46.628 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name all
14:28:46.628 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name test-store-all
14:28:46.628 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name range
14:28:46.629 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name test-store-range
14:28:46.629 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name flush
14:28:46.629 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name test-store-flush
14:28:46.629 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name restore
14:28:46.629 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name test-store-restore
14:28:46.766 [GlobalStreamThread] INFO o.a.k.s.p.i.GlobalStateManagerImpl - restoring state for global store test-store
14:28:46.768 [GlobalStreamThread] DEBUG o.apache.kafka.clients.NetworkClient - Initiating connection to node -1 at localhost:9092.
14:28:46.769 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--1.bytes-sent
14:28:46.770 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--1.bytes-received
14:28:46.770 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--1.latency
14:28:46.770 [GlobalStreamThread] DEBUG o.a.kafka.common.network.Selector - Created socket with SO_RCVBUF = 342972, SO_SNDBUF = 146988, SO_TIMEOUT = 0 to node -1
14:28:46.770 [GlobalStreamThread] DEBUG o.apache.kafka.clients.NetworkClient - Completed connection to node -1. Fetching API versions.
14:28:46.770 [GlobalStreamThread] DEBUG o.apache.kafka.clients.NetworkClient - Initiating API versions fetch from node -1.
14:28:46.772 [GlobalStreamThread] DEBUG o.apache.kafka.clients.NetworkClient - Recorded API versions for node -1: (Produce(0): 0 to 2 [usable: 2], Fetch(1): 0 to 3 [usable: 3], Offsets(2): 0 to 1 [usable: 1], Metadata(3): 0 to 2 [usable: 2], LeaderAndIsr(4): 0 [usable: 0], StopReplica(5): 0 [usable: 0], UpdateMetadata(6): 0 to 3 [usable: 3], ControlledShutdown(7): 1 [usable: 1], OffsetCommit(8): 0 to 2 [usable: 2], OffsetFetch(9): 0 to 2 [usable: 2], GroupCoordinator(10): 0 [usable: 0], JoinGroup(11): 0 to 1 [usable: 1], Heartbeat(12): 0 [usable: 0], LeaveGroup(13): 0 [usable: 0], SyncGroup(14): 0 [usable: 0], DescribeGroups(15): 0 [usable: 0], ListGroups(16): 0 [usable: 0], SaslHandshake(17): 0 [usable: 0], ApiVersions(18): 0 [usable: 0], CreateTopics(19): 0 to 1 [usable: 1], DeleteTopics(20): 0 [usable: 0])
14:28:46.773 [GlobalStreamThread] DEBUG o.apache.kafka.clients.NetworkClient - Sending metadata request (type=MetadataRequest, topics=) to node -1
14:28:46.778 [GlobalStreamThread] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 2 to Cluster(id = _Epo8QiSSuGbI1zi8iEXLA, nodes = [10.20.69.86:9092 (id: 0 rack: null)], partitions = [])
14:28:46.784 [GlobalStreamThread] DEBUG o.a.k.clients.consumer.KafkaConsumer - Subscribed to partition(s): StreamConfig2-0
14:28:46.785 [GlobalStreamThread] DEBUG o.a.k.c.consumer.internals.Fetcher - Partition StreamConfig2-0 is unknown for fetching offset, wait for metadata refresh
14:28:46.785 [GlobalStreamThread] DEBUG o.apache.kafka.clients.NetworkClient - Initialize connection to node 0 for sending metadata request
14:28:46.785 [GlobalStreamThread] DEBUG o.apache.kafka.clients.NetworkClient - Initiating connection to node 0 at 10.20.69.86:9092.
14:28:46.786 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node-0.bytes-sent
14:28:46.786 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node-0.bytes-received
14:28:46.786 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node-0.latency
14:28:46.786 [GlobalStreamThread] DEBUG o.a.kafka.common.network.Selector - Created socket with SO_RCVBUF = 342972, SO_SNDBUF = 146988, SO_TIMEOUT = 0 to node 0
14:28:46.786 [GlobalStreamThread] DEBUG o.apache.kafka.clients.NetworkClient - Completed connection to node 0. Fetching API versions.
14:28:46.786 [GlobalStreamThread] DEBUG o.apache.kafka.clients.NetworkClient - Initiating API versions fetch from node 0.
14:28:46.788 [GlobalStreamThread] DEBUG o.apache.kafka.clients.NetworkClient - Recorded API versions for node 0: (Produce(0): 0 to 2 [usable: 2], Fetch(1): 0 to 3 [usable: 3], Offsets(2): 0 to 1 [usable: 1], Metadata(3): 0 to 2 [usable: 2], LeaderAndIsr(4): 0 [usable: 0], StopReplica(5): 0 [usable: 0], UpdateMetadata(6): 0 to 3 [usable: 3], ControlledShutdown(7): 1 [usable: 1], OffsetCommit(8): 0 to 2 [usable: 2], OffsetFetch(9): 0 to 2 [usable: 2], GroupCoordinator(10): 0 [usable: 0], JoinGroup(11): 0 to 1 [usable: 1], Heartbeat(12): 0 [usable: 0], LeaveGroup(13): 0 [usable: 0], SyncGroup(14): 0 [usable: 0], DescribeGroups(15): 0 [usable: 0], ListGroups(16): 0 [usable: 0], SaslHandshake(17): 0 [usable: 0], ApiVersions(18): 0 [usable: 0], CreateTopics(19): 0 to 1 [usable: 1], DeleteTopics(20): 0 [usable: 0])
14:28:46.789 [GlobalStreamThread] DEBUG o.apache.kafka.clients.NetworkClient - Sending metadata request (type=MetadataRequest, topics=StreamConfig2) to node 0
14:28:46.790 [GlobalStreamThread] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 3 to Cluster(id = _Epo8QiSSuGbI1zi8iEXLA, nodes = [10.20.69.86:9092 (id: 0 rack: null)], partitions = [Partition(topic = StreamConfig2, partition = 0, leader = 0, replicas = [0], isr = [0])])
14:28:46.797 [GlobalStreamThread] DEBUG o.a.k.c.consumer.internals.Fetcher - Handling ListOffsetResponse response for StreamConfig2-0. Fetched offset 2, timestamp -1
14:28:46.797 [GlobalStreamThread] DEBUG o.a.k.clients.consumer.KafkaConsumer - Subscribed to partition(s): StreamConfig2-0
14:28:46.797 [GlobalStreamThread] DEBUG o.a.k.clients.consumer.KafkaConsumer - Seeking to offset 2 for partition StreamConfig2-0
14:28:46.797 [GlobalStreamThread] DEBUG o.a.k.clients.consumer.KafkaConsumer - Unsubscribed all topics or patterns and assigned partitions
14:28:46.800 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name -1_-1-test-store-hitRatio
14:28:46.802 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name process
14:28:46.802 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name task.-1_-1.KSTREAM-SOURCE-0000000000-process
14:28:46.802 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name punctuate
14:28:46.802 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name task.-1_-1.KSTREAM-SOURCE-0000000000-punctuate
14:28:46.803 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name create
14:28:46.803 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name task.-1_-1.KSTREAM-SOURCE-0000000000-create
14:28:46.803 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name destroy
14:28:46.803 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name task.-1_-1.KSTREAM-SOURCE-0000000000-destroy
14:28:46.804 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name forward
14:28:46.804 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name task.-1_-1.KSTREAM-SOURCE-0000000000-forward
14:28:46.806 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name task.-1_-1.KTABLE-SOURCE-0000000001-process
14:28:46.806 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name task.-1_-1.KTABLE-SOURCE-0000000001-punctuate
14:28:46.807 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name task.-1_-1.KTABLE-SOURCE-0000000001-create
14:28:46.807 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name task.-1_-1.KTABLE-SOURCE-0000000001-destroy
14:28:46.807 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name task.-1_-1.KTABLE-SOURCE-0000000001-forward
14:28:46.810 [GlobalStreamThread] DEBUG o.a.k.clients.consumer.KafkaConsumer - Subscribed to partition(s): StreamConfig2-0
14:28:46.810 [GlobalStreamThread] DEBUG o.a.k.clients.consumer.KafkaConsumer - Seeking to offset 2 for partition StreamConfig2-0
14:28:46.811 [StreamThread-1] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] Starting
14:28:46.811 [StreamThread-1] DEBUG o.a.k.clients.consumer.KafkaConsumer - Subscribed to pattern: StreamConfig2
14:28:46.811 [GlobalStreamThread] DEBUG o.a.k.c.consumer.internals.Fetcher - Sending fetch for partitions [StreamConfig2-0] to broker 10.20.69.86:9092 (id: 0 rack: null)
14:28:46.812 [StreamThread-1] DEBUG o.a.k.c.c.i.AbstractCoordinator - Sending GroupCoordinator request for group map-function-scala-example to broker localhost:9092 (id: -1 rack: null)
14:28:46.812 [StreamThread-1] DEBUG o.apache.kafka.clients.NetworkClient - Initiating connection to node -1 at localhost:9092.
14:28:46.812 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--1.bytes-sent
14:28:46.813 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--1.bytes-received
14:28:46.813 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--1.latency
14:28:46.813 [StreamThread-1] DEBUG o.a.kafka.common.network.Selector - Created socket with SO_RCVBUF = 342972, SO_SNDBUF = 146988, SO_TIMEOUT = 0 to node -1
14:28:46.814 [StreamThread-1] DEBUG o.apache.kafka.clients.NetworkClient - Completed connection to node -1. Fetching API versions.
14:28:46.814 [main] INFO o.apache.kafka.streams.KafkaStreams - stream-client [map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191] Started Kafka Stream process
14:28:46.814 [StreamThread-1] DEBUG o.apache.kafka.clients.NetworkClient - Initiating API versions fetch from node -1.
14:28:46.817 [StreamThread-1] DEBUG o.apache.kafka.clients.NetworkClient - Recorded API versions for node -1: (Produce(0): 0 to 2 [usable: 2], Fetch(1): 0 to 3 [usable: 3], Offsets(2): 0 to 1 [usable: 1], Metadata(3): 0 to 2 [usable: 2], LeaderAndIsr(4): 0 [usable: 0], StopReplica(5): 0 [usable: 0], UpdateMetadata(6): 0 to 3 [usable: 3], ControlledShutdown(7): 1 [usable: 1], OffsetCommit(8): 0 to 2 [usable: 2], OffsetFetch(9): 0 to 2 [usable: 2], GroupCoordinator(10): 0 [usable: 0], JoinGroup(11): 0 to 1 [usable: 1], Heartbeat(12): 0 [usable: 0], LeaveGroup(13): 0 [usable: 0], SyncGroup(14): 0 [usable: 0], DescribeGroups(15): 0 [usable: 0], ListGroups(16): 0 [usable: 0], SaslHandshake(17): 0 [usable: 0], ApiVersions(18): 0 [usable: 0], CreateTopics(19): 0 to 1 [usable: 1], DeleteTopics(20): 0 [usable: 0])
14:28:46.818 [StreamThread-1] DEBUG o.apache.kafka.clients.NetworkClient - Sending metadata request (type=MetadataRequest, topics=<ALL>) to node -1
14:28:46.824 [StreamThread-1] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 2 to Cluster(id = _Epo8QiSSuGbI1zi8iEXLA, nodes = [10.20.69.86:9092 (id: 0 rack: null)], partitions = [Partition(topic = StreamConfig2, partition = 0, leader = 0, replicas = [0], isr = [0])])
14:28:46.830 [StreamThread-1] DEBUG o.a.k.c.c.i.AbstractCoordinator - Received GroupCoordinator response ClientResponse(receivedTimeMs=1500326926829, latencyMs=17, disconnected=false, requestHeader={api_key=10,api_version=0,correlation_id=0,client_id=map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1-consumer}, responseBody={error_code=0,coordinator={node_id=0,host=10.20.69.86,port=9092}}) for group map-function-scala-example
14:28:46.830 [StreamThread-1] INFO o.a.k.c.c.i.AbstractCoordinator - Discovered coordinator 10.20.69.86:9092 (id: 2147483647 rack: null) for group map-function-scala-example.
14:28:46.830 [StreamThread-1] DEBUG o.apache.kafka.clients.NetworkClient - Initiating connection to node 2147483647 at 10.20.69.86:9092.
14:28:46.833 [kafka-coordinator-heartbeat-thread | map-function-scala-example] DEBUG o.a.k.c.c.i.AbstractCoordinator - Heartbeat thread for group map-function-scala-example started
14:28:46.833 [StreamThread-1] INFO o.a.k.c.c.i.ConsumerCoordinator - Revoking previously assigned partitions [] for group map-function-scala-example
14:28:46.833 [StreamThread-1] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] at state RUNNING: partitions [] revoked at the beginning of consumer rebalance.
14:28:46.833 [StreamThread-1] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] State transition from RUNNING to PARTITIONS_REVOKED.
14:28:46.833 [StreamThread-1] INFO o.apache.kafka.streams.KafkaStreams - stream-client [map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191] State transition from RUNNING to REBALANCING.
14:28:46.833 [StreamThread-1] DEBUG o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] suspendTasksAndState: suspending all active tasks [] and standby tasks []
14:28:46.837 [StreamThread-1] DEBUG o.a.k.clients.consumer.KafkaConsumer - Unsubscribed all topics or patterns and assigned partitions
Looking at config
14:28:46.838 [StreamThread-1] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] Updating suspended tasks to contain active tasks []
14:28:46.838 [StreamThread-1] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] Removing all active tasks []
14:28:46.838 [StreamThread-1] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] Removing all standby tasks []
14:28:46.838 [StreamThread-1] INFO o.a.k.c.c.i.AbstractCoordinator - (Re-)joining group map-function-scala-example
iterator created
HAS NEXT source-topic sink-topic
14:28:46.848 [StreamThread-1] DEBUG o.a.k.s.p.i.StreamPartitionAssignor - stream-thread [StreamThread-1] found [StreamConfig2] topics possibly matching regex
14:28:46.848 [StreamThread-1] DEBUG o.a.k.s.processor.TopologyBuilder - stream-thread [StreamThread-1] updating builder with SubscriptionUpdates{updatedTopicSubscriptions=[StreamConfig2]} topic(s) with possible matching regex subscription(s)
14:28:46.852 [StreamThread-1] DEBUG o.a.k.c.c.i.AbstractCoordinator - Sending JoinGroup ((type: JoinGroupRequest, groupId=map-function-scala-example, sessionTimeout=10000, rebalanceTimeout=2147483647, memberId=, protocolType=consumer, groupProtocols=org.apache.kafka.common.requests.JoinGroupRequest$ProtocolMetadata@51b1300d)) to coordinator 10.20.69.86:9092 (id: 2147483647 rack: null)
14:28:46.852 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node-2147483647.bytes-sent
CREATING source-topic sink-topic
14:28:46.853 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node-2147483647.bytes-received
14:28:46.853 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node-2147483647.latency
14:28:46.853 [StreamThread-1] DEBUG o.a.kafka.common.network.Selector - Created socket with SO_RCVBUF = 342972, SO_SNDBUF = 146988, SO_TIMEOUT = 0 to node 2147483647
14:28:46.853 [StreamThread-1] DEBUG o.apache.kafka.clients.NetworkClient - Completed connection to node 2147483647. Fetching API versions.
14:28:46.853 [StreamThread-1] DEBUG o.apache.kafka.clients.NetworkClient - Initiating API versions fetch from node 2147483647.
14:28:46.855 [StreamThread-1] DEBUG o.apache.kafka.clients.NetworkClient - Recorded API versions for node 2147483647: (Produce(0): 0 to 2 [usable: 2], Fetch(1): 0 to 3 [usable: 3], Offsets(2): 0 to 1 [usable: 1], Metadata(3): 0 to 2 [usable: 2], LeaderAndIsr(4): 0 [usable: 0], StopReplica(5): 0 [usable: 0], UpdateMetadata(6): 0 to 3 [usable: 3], ControlledShutdown(7): 1 [usable: 1], OffsetCommit(8): 0 to 2 [usable: 2], OffsetFetch(9): 0 to 2 [usable: 2], GroupCoordinator(10): 0 [usable: 0], JoinGroup(11): 0 to 1 [usable: 1], Heartbeat(12): 0 [usable: 0], LeaveGroup(13): 0 [usable: 0], SyncGroup(14): 0 [usable: 0], DescribeGroups(15): 0 [usable: 0], ListGroups(16): 0 [usable: 0], SaslHandshake(17): 0 [usable: 0], ApiVersions(18): 0 [usable: 0], CreateTopics(19): 0 to 1 [usable: 1], DeleteTopics(20): 0 [usable: 0])
14:28:46.870 [StreamThread-1] DEBUG o.a.k.c.c.i.AbstractCoordinator - Received successful JoinGroup response for group map-function-scala-example: {error_code=0,generation_id=31,group_protocol=stream,leader_id=map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1-consumer-34336ba2-46de-498a-b55d-262b15e08020,member_id=map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1-consumer-34336ba2-46de-498a-b55d-262b15e08020,members=[{member_id=map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1-consumer-34336ba2-46de-498a-b55d-262b15e08020,member_metadata=java.nio.HeapByteBuffer[pos=0 lim=65 cap=65]}]}
14:28:46.872 [StreamThread-1] DEBUG o.a.k.c.c.i.ConsumerCoordinator - Performing assignment for group map-function-scala-example using strategy stream with subscriptions {map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1-consumer-34336ba2-46de-498a-b55d-262b15e08020=Subscription(topics=[StreamConfig2])}
14:28:46.872 [main] INFO o.apache.kafka.streams.StreamsConfig - StreamsConfig values:
application.id = map-function-scala-example
application.server =
bootstrap.servers = [localhost:9092]
buffered.records.per.partition = 1000
cache.max.bytes.buffering = 10485760
client.id =
commit.interval.ms = 30000
connections.max.idle.ms = 540000
key.serde = class org.apache.kafka.common.serialization.Serdes$ByteArraySerde
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
num.standby.replicas = 0
num.stream.threads = 1
partition.grouper = class org.apache.kafka.streams.processor.DefaultPartitionGrouper
poll.ms = 100
receive.buffer.bytes = 32768
reconnect.backoff.ms = 50
replication.factor = 1
request.timeout.ms = 40000
retry.backoff.ms = 100
rocksdb.config.setter = null
security.protocol = PLAINTEXT
send.buffer.bytes = 131072
state.cleanup.delay.ms = 600000
state.dir = /tmp/kafka-streams
timestamp.extractor = class org.apache.kafka.streams.processor.FailOnInvalidTimestamp
value.serde = class org.apache.kafka.common.serialization.Serdes$StringSerde
windowstore.changelog.additional.retention.ms = 86400000
zookeeper.connect =
14:28:46.872 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name thread.map-function-scala-example-72622e43-3a62-476d-8232-915f15356b54-StreamThread-2.commit-latency
14:28:46.873 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name thread.map-function-scala-example-72622e43-3a62-476d-8232-915f15356b54-StreamThread-2.poll-latency
14:28:46.873 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name thread.map-function-scala-example-72622e43-3a62-476d-8232-915f15356b54-StreamThread-2.process-latency
14:28:46.873 [StreamThread-1] INFO o.a.k.s.p.i.StreamPartitionAssignor - stream-thread [StreamThread-1] Constructed client metadata {f215d043-5170-41a5-8bf1-99d7c5873191=ClientMetadata{hostInfo=null, consumers=[map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1-consumer-34336ba2-46de-498a-b55d-262b15e08020], state=[activeTasks: ([]) assignedTasks: ([]) prevActiveTasks: ([]) prevAssignedTasks: ([0_0]) capacity: 1.0 cost: 0.0]}} from the member subscriptions.
14:28:46.873 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name thread.map-function-scala-example-72622e43-3a62-476d-8232-915f15356b54-StreamThread-2.punctuate-latency
14:28:46.873 [StreamThread-1] DEBUG o.a.k.s.p.i.StreamPartitionAssignor - stream-thread [StreamThread-1] Starting to validate internal topics in partition assignor.
14:28:46.873 [StreamThread-1] INFO o.a.k.s.p.i.StreamPartitionAssignor - stream-thread [StreamThread-1] Completed validating internal topics in partition assignor
14:28:46.873 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name thread.map-function-scala-example-72622e43-3a62-476d-8232-915f15356b54-StreamThread-2.task-created
14:28:46.873 [StreamThread-1] DEBUG o.a.k.s.p.i.StreamPartitionAssignor - stream-thread [StreamThread-1] Created repartition topics [] from the parsed topology.
14:28:46.873 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name thread.map-function-scala-example-72622e43-3a62-476d-8232-915f15356b54-StreamThread-2.task-closed
14:28:46.873 [StreamThread-1] DEBUG o.a.k.s.p.i.StreamPartitionAssignor - stream-thread [StreamThread-1] Starting to validate internal topics in partition assignor.
14:28:46.874 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name thread.map-function-scala-example-72622e43-3a62-476d-8232-915f15356b54-StreamThread-2.skipped-records
14:28:46.874 [StreamThread-1] INFO o.a.k.s.p.i.StreamPartitionAssignor - stream-thread [StreamThread-1] Completed validating internal topics in partition assignor
14:28:46.874 [main] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-2] Creating producer client
14:28:46.874 [StreamThread-1] DEBUG o.a.k.s.p.i.StreamPartitionAssignor - stream-thread [StreamThread-1] Created state changelog topics {} from the parsed topology.
14:28:46.874 [main] INFO o.a.k.c.producer.ProducerConfig - ProducerConfig values:
acks = 1
batch.size = 16384
block.on.buffer.full = false
bootstrap.servers = [localhost:9092]
buffer.memory = 33554432
client.id = map-function-scala-example-72622e43-3a62-476d-8232-915f15356b54-StreamThread-2-producer
compression.type = none
connections.max.idle.ms = 540000
interceptor.classes = null
key.serializer = class org.apache.kafka.common.serialization.ByteArraySerializer
linger.ms = 100
max.block.ms = 60000
max.in.flight.requests.per.connection = 5
max.request.size = 1048576
metadata.fetch.timeout.ms = 60000
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.sample.window.ms = 30000
partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner
receive.buffer.bytes = 32768
reconnect.backoff.ms = 50
request.timeout.ms = 30000
retries = 10
retry.backoff.ms = 100
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
send.buffer.bytes = 131072
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = null
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
timeout.ms = 30000
value.serializer = class org.apache.kafka.common.serialization.ByteArraySerializer
14:28:46.874 [StreamThread-1] DEBUG o.a.k.s.p.i.StreamPartitionAssignor - stream-thread [StreamThread-1] Assigning tasks [] to clients {f215d043-5170-41a5-8bf1-99d7c5873191=[activeTasks: ([]) assignedTasks: ([]) prevActiveTasks: ([]) prevAssignedTasks: ([0_0]) capacity: 1.0 cost: 0.0]} with number of replicas 0
14:28:46.875 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bufferpool-wait-time
14:28:46.875 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name buffer-exhausted-records
14:28:46.876 [main] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 1 to Cluster(id = null, nodes = [localhost:9092 (id: -1 rack: null)], partitions = [])
14:28:46.876 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-closed:
14:28:46.876 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-created:
14:28:46.876 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent-received:
14:28:46.876 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent:
14:28:46.877 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-received:
14:28:46.877 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name select-time:
14:28:46.877 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name io-time:
14:28:46.877 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name batch-size
14:28:46.877 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name compression-rate
14:28:46.877 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name queue-time
14:28:46.877 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name request-time
14:28:46.877 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name produce-throttle-time
14:28:46.878 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name records-per-request
14:28:46.878 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name record-retries
14:28:46.878 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name errors
14:28:46.878 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name record-size-max
14:28:46.878 [kafka-producer-network-thread | map-function-scala-example-72622e43-3a62-476d-8232-915f15356b54-StreamThread-2-producer] DEBUG o.a.k.c.producer.internals.Sender - Starting Kafka producer I/O thread.
14:28:46.879 [main] INFO o.a.kafka.common.utils.AppInfoParser - Kafka version : 0.10.2.1-cp2
14:28:46.879 [main] INFO o.a.kafka.common.utils.AppInfoParser - Kafka commitId : 5092731dfc53f849
14:28:46.879 [main] DEBUG o.a.k.clients.producer.KafkaProducer - Kafka producer started
14:28:46.879 [main] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-2] Creating consumer client
14:28:46.879 [StreamThread-1] INFO o.a.k.s.p.i.StreamPartitionAssignor - stream-thread [StreamThread-1] Assigned tasks to clients as {f215d043-5170-41a5-8bf1-99d7c5873191=[activeTasks: ([]) assignedTasks: ([]) prevActiveTasks: ([]) prevAssignedTasks: ([0_0]) capacity: 1.0 cost: 0.0]}.
14:28:46.879 [main] INFO o.a.k.c.consumer.ConsumerConfig - ConsumerConfig values:
auto.commit.interval.ms = 5000
auto.offset.reset = earliest
bootstrap.servers = [localhost:9092]
check.crcs = true
client.id = map-function-scala-example-72622e43-3a62-476d-8232-915f15356b54-StreamThread-2-consumer
connections.max.idle.ms = 540000
enable.auto.commit = false
exclude.internal.topics = true
fetch.max.bytes = 52428800
fetch.max.wait.ms = 500
fetch.min.bytes = 1
group.id = map-function-scala-example
heartbeat.interval.ms = 3000
interceptor.classes = null
internal.leave.group.on.close = false
key.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer
max.partition.fetch.bytes = 1048576
max.poll.interval.ms = 2147483647
max.poll.records = 1000
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
partition.assignment.strategy = [org.apache.kafka.streams.processor.internals.StreamPartitionAssignor]
receive.buffer.bytes = 65536
reconnect.backoff.ms = 50
request.timeout.ms = 305000
retry.backoff.ms = 100
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
send.buffer.bytes = 131072
session.timeout.ms = 10000
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = null
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
value.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer
14:28:46.880 [main] DEBUG o.a.k.clients.consumer.KafkaConsumer - Starting the Kafka consumer
14:28:46.880 [main] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 1 to Cluster(id = null, nodes = [localhost:9092 (id: -1 rack: null)], partitions = [])
14:28:46.880 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-closed:
14:28:46.881 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-created:
14:28:46.881 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent-received:
14:28:46.881 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent:
14:28:46.881 [StreamThread-1] WARN o.a.k.c.c.i.ConsumerCoordinator - The following subscribed topics are not assigned to any members in the group map-function-scala-example : [StreamConfig2]
14:28:46.881 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-received:
14:28:46.881 [StreamThread-1] DEBUG o.a.k.c.c.i.ConsumerCoordinator - Finished assignment for group map-function-scala-example: {map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1-consumer-34336ba2-46de-498a-b55d-262b15e08020=Assignment(partitions=[])}
14:28:46.882 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name select-time:
14:28:46.882 [StreamThread-1] DEBUG o.a.k.c.c.i.AbstractCoordinator - Sending leader SyncGroup for group map-function-scala-example to coordinator 10.20.69.86:9092 (id: 2147483647 rack: null): (type=SyncGroupRequest, groupId=map-function-scala-example, generationId=31, memberId=map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1-consumer-34336ba2-46de-498a-b55d-262b15e08020, groupAssignment=map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1-consumer-34336ba2-46de-498a-b55d-262b15e08020)
14:28:46.882 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name io-time:
14:28:46.882 [main] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 1 to Cluster(id = null, nodes = [localhost:9092 (id: -1 rack: null)], partitions = [])
14:28:46.883 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-closed:
14:28:46.883 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-created:
14:28:46.883 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent-received:
14:28:46.883 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent:
14:28:46.883 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-received:
14:28:46.883 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name select-time:
14:28:46.883 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name io-time:
14:28:46.884 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name heartbeat-latency
14:28:46.884 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name join-latency
14:28:46.884 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name sync-latency
14:28:46.884 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name commit-latency
14:28:46.884 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-fetched
14:28:46.885 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name records-fetched
14:28:46.885 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name fetch-latency
14:28:46.885 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name records-lag
14:28:46.885 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name fetch-throttle-time
14:28:46.885 [main] INFO o.a.kafka.common.utils.AppInfoParser - Kafka version : 0.10.2.1-cp2
14:28:46.885 [main] INFO o.a.kafka.common.utils.AppInfoParser - Kafka commitId : 5092731dfc53f849
14:28:46.886 [main] DEBUG o.a.k.clients.consumer.KafkaConsumer - Kafka consumer created
14:28:46.886 [main] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-2] Creating restore consumer client
14:28:46.886 [main] INFO o.a.k.c.consumer.ConsumerConfig - ConsumerConfig values:
auto.commit.interval.ms = 5000
auto.offset.reset = earliest
bootstrap.servers = [localhost:9092]
check.crcs = true
client.id = map-function-scala-example-72622e43-3a62-476d-8232-915f15356b54-StreamThread-2-restore-consumer
connections.max.idle.ms = 540000
enable.auto.commit = false
exclude.internal.topics = true
fetch.max.bytes = 52428800
fetch.max.wait.ms = 500
fetch.min.bytes = 1
group.id =
heartbeat.interval.ms = 3000
interceptor.classes = null
internal.leave.group.on.close = false
key.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer
max.partition.fetch.bytes = 1048576
max.poll.interval.ms = 2147483647
max.poll.records = 1000
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
partition.assignment.strategy = [class org.apache.kafka.clients.consumer.RangeAssignor]
receive.buffer.bytes = 65536
reconnect.backoff.ms = 50
request.timeout.ms = 305000
retry.backoff.ms = 100
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
send.buffer.bytes = 131072
session.timeout.ms = 10000
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = null
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
value.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer
14:28:46.886 [main] DEBUG o.a.k.clients.consumer.KafkaConsumer - Starting the Kafka consumer
14:28:46.887 [main] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 1 to Cluster(id = null, nodes = [localhost:9092 (id: -1 rack: null)], partitions = [])
14:28:46.887 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-closed:
14:28:46.887 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-created:
14:28:46.887 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent-received:
14:28:46.887 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent:
14:28:46.887 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-received:
14:28:46.887 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name select-time:
14:28:46.888 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name io-time:
14:28:46.888 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name heartbeat-latency
14:28:46.888 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name join-latency
14:28:46.888 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name sync-latency
14:28:46.889 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name commit-latency
14:28:46.889 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-fetched
14:28:46.889 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name records-fetched
14:28:46.889 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name fetch-latency
14:28:46.889 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name records-lag
14:28:46.889 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name fetch-throttle-time
14:28:46.890 [main] INFO o.a.kafka.common.utils.AppInfoParser - Kafka version : 0.10.2.1-cp2
14:28:46.890 [main] INFO o.a.kafka.common.utils.AppInfoParser - Kafka commitId : 5092731dfc53f849
14:28:46.890 [main] DEBUG o.a.k.clients.consumer.KafkaConsumer - Kafka consumer created
14:28:46.890 [main] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-2] State transition from NOT_RUNNING to RUNNING.
14:28:46.890 [main] DEBUG o.apache.kafka.streams.KafkaStreams - stream-client [map-function-scala-example-72622e43-3a62-476d-8232-915f15356b54] Starting Kafka Stream process.
14:28:46.890 [main] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 1 to Cluster(id = null, nodes = [localhost:9092 (id: -1 rack: null)], partitions = [])
14:28:46.891 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-closed:
14:28:46.891 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-created:
14:28:46.891 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent-received:
14:28:46.891 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent:
14:28:46.891 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-received:
14:28:46.891 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name select-time:
14:28:46.891 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name io-time:
14:28:46.892 [main] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 1 to Cluster(id = null, nodes = [localhost:9092 (id: -1 rack: null)], partitions = [])
14:28:46.892 [main] DEBUG o.apache.kafka.clients.NetworkClient - Initiating connection to node -1 at localhost:9092.
14:28:46.908 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--1.bytes-sent
14:28:46.908 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--1.bytes-received
14:28:46.908 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--1.latency
14:28:46.909 [main] DEBUG o.a.kafka.common.network.Selector - Created socket with SO_RCVBUF = 326640, SO_SNDBUF = 146988, SO_TIMEOUT = 0 to node -1
14:28:46.909 [main] DEBUG o.apache.kafka.clients.NetworkClient - Completed connection to node -1. Fetching API versions.
14:28:46.909 [main] DEBUG o.apache.kafka.clients.NetworkClient - Initiating API versions fetch from node -1.
14:28:46.914 [main] DEBUG o.apache.kafka.clients.NetworkClient - Recorded API versions for node -1: (Produce(0): 0 to 2 [usable: 2], Fetch(1): 0 to 3 [usable: 3], Offsets(2): 0 to 1 [usable: 1], Metadata(3): 0 to 2 [usable: 2], LeaderAndIsr(4): 0 [usable: 0], StopReplica(5): 0 [usable: 0], UpdateMetadata(6): 0 to 3 [usable: 3], ControlledShutdown(7): 1 [usable: 1], OffsetCommit(8): 0 to 2 [usable: 2], OffsetFetch(9): 0 to 2 [usable: 2], GroupCoordinator(10): 0 [usable: 0], JoinGroup(11): 0 to 1 [usable: 1], Heartbeat(12): 0 [usable: 0], LeaveGroup(13): 0 [usable: 0], SyncGroup(14): 0 [usable: 0], DescribeGroups(15): 0 [usable: 0], ListGroups(16): 0 [usable: 0], SaslHandshake(17): 0 [usable: 0], ApiVersions(18): 0 [usable: 0], CreateTopics(19): 0 to 1 [usable: 1], DeleteTopics(20): 0 [usable: 0])
14:28:46.924 [StreamThread-1] INFO o.a.k.c.c.i.AbstractCoordinator - Successfully joined group map-function-scala-example with generation 31
14:28:46.926 [StreamThread-1] INFO o.a.k.c.c.i.ConsumerCoordinator - Setting newly assigned partitions [] for group map-function-scala-example
14:28:46.926 [StreamThread-1] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] at state PARTITIONS_REVOKED: new partitions [] assigned at the end of consumer rebalance.
14:28:46.926 [StreamThread-1] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] State transition from PARTITIONS_REVOKED to ASSIGNING_PARTITIONS.
14:28:46.926 [StreamThread-1] INFO o.apache.kafka.streams.KafkaStreams - stream-client [map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191] State transition from REBALANCING to REBALANCING.
14:28:46.927 [StreamThread-1] DEBUG o.a.k.clients.consumer.KafkaConsumer - Unsubscribed all topics or patterns and assigned partitions
14:28:46.927 [StreamThread-1] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] State transition from ASSIGNING_PARTITIONS to RUNNING.
14:28:46.927 [StreamThread-1] INFO o.apache.kafka.streams.KafkaStreams - stream-client [map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191] State transition from REBALANCING to RUNNING.
14:28:47.019 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name connections-closed:
14:28:47.019 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name connections-created:
14:28:47.019 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name bytes-sent-received:
14:28:47.019 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name bytes-sent:
14:28:47.019 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name bytes-received:
14:28:47.019 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name select-time:
14:28:47.019 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name io-time:
14:28:47.019 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name node--1.bytes-sent
14:28:47.020 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name node--1.bytes-received
14:28:47.020 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name node--1.latency
14:28:47.020 [main] INFO o.apache.kafka.streams.KafkaStreams - stream-client [map-function-scala-example-72622e43-3a62-476d-8232-915f15356b54] State transition from CREATED to RUNNING.
14:28:47.020 [main] INFO o.apache.kafka.streams.KafkaStreams - stream-client [map-function-scala-example-72622e43-3a62-476d-8232-915f15356b54] Started Kafka Stream process
Starting stream...
14:28:47.020 [StreamThread-2] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-2] Starting
HAS NEXT source-topic2 sink-topic2
CREATING source-topic2 sink-topic2
14:28:47.020 [StreamThread-2] DEBUG o.a.k.clients.consumer.KafkaConsumer - Subscribed to pattern: source-topic
14:28:47.020 [StreamThread-2] DEBUG o.a.k.c.c.i.AbstractCoordinator - Sending GroupCoordinator request for group map-function-scala-example to broker localhost:9092 (id: -1 rack: null)
14:28:47.021 [StreamThread-2] DEBUG o.apache.kafka.clients.NetworkClient - Initiating connection to node -1 at localhost:9092.
14:28:47.021 [StreamThread-2] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--1.bytes-sent
14:28:47.022 [main] INFO o.apache.kafka.streams.StreamsConfig - StreamsConfig values:
application.id = map-function-scala-example
application.server =
bootstrap.servers = [localhost:9092]
buffered.records.per.partition = 1000
cache.max.bytes.buffering = 10485760
client.id =
commit.interval.ms = 30000
connections.max.idle.ms = 540000
key.serde = class org.apache.kafka.common.serialization.Serdes$ByteArraySerde
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
num.standby.replicas = 0
num.stream.threads = 1
partition.grouper = class org.apache.kafka.streams.processor.DefaultPartitionGrouper
poll.ms = 100
receive.buffer.bytes = 32768
reconnect.backoff.ms = 50
replication.factor = 1
request.timeout.ms = 40000
retry.backoff.ms = 100
rocksdb.config.setter = null
security.protocol = PLAINTEXT
send.buffer.bytes = 131072
state.cleanup.delay.ms = 600000
state.dir = /tmp/kafka-streams
timestamp.extractor = class org.apache.kafka.streams.processor.FailOnInvalidTimestamp
value.serde = class org.apache.kafka.common.serialization.Serdes$StringSerde
windowstore.changelog.additional.retention.ms = 86400000
zookeeper.connect =
14:28:47.022 [StreamThread-2] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--1.bytes-received
14:28:47.022 [StreamThread-2] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--1.latency
14:28:47.022 [StreamThread-2] DEBUG o.a.kafka.common.network.Selector - Created socket with SO_RCVBUF = 342972, SO_SNDBUF = 146988, SO_TIMEOUT = 0 to node -1
14:28:47.022 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name thread.map-function-scala-example-1af7fd3f-aaef-4b8a-ac1b-904af3cfe4bb-StreamThread-3.commit-latency
14:28:47.022 [StreamThread-2] DEBUG o.apache.kafka.clients.NetworkClient - Completed connection to node -1. Fetching API versions.
14:28:47.022 [StreamThread-2] DEBUG o.apache.kafka.clients.NetworkClient - Initiating API versions fetch from node -1.
14:28:47.023 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name thread.map-function-scala-example-1af7fd3f-aaef-4b8a-ac1b-904af3cfe4bb-StreamThread-3.poll-latency
14:28:47.023 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name thread.map-function-scala-example-1af7fd3f-aaef-4b8a-ac1b-904af3cfe4bb-StreamThread-3.process-latency
14:28:47.027 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name thread.map-function-scala-example-1af7fd3f-aaef-4b8a-ac1b-904af3cfe4bb-StreamThread-3.punctuate-latency
14:28:47.028 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name thread.map-function-scala-example-1af7fd3f-aaef-4b8a-ac1b-904af3cfe4bb-StreamThread-3.task-created
14:28:47.028 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name thread.map-function-scala-example-1af7fd3f-aaef-4b8a-ac1b-904af3cfe4bb-StreamThread-3.task-closed
14:28:47.028 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name thread.map-function-scala-example-1af7fd3f-aaef-4b8a-ac1b-904af3cfe4bb-StreamThread-3.skipped-records
14:28:47.029 [main] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-3] Creating producer client
14:28:47.034 [main] INFO o.a.k.c.producer.ProducerConfig - ProducerConfig values:
acks = 1
batch.size = 16384
block.on.buffer.full = false
bootstrap.servers = [localhost:9092]
buffer.memory = 33554432
client.id = map-function-scala-example-1af7fd3f-aaef-4b8a-ac1b-904af3cfe4bb-StreamThread-3-producer
compression.type = none
connections.max.idle.ms = 540000
interceptor.classes = null
key.serializer = class org.apache.kafka.common.serialization.ByteArraySerializer
linger.ms = 100
max.block.ms = 60000
max.in.flight.requests.per.connection = 5
max.request.size = 1048576
metadata.fetch.timeout.ms = 60000
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.sample.window.ms = 30000
partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner
receive.buffer.bytes = 32768
reconnect.backoff.ms = 50
request.timeout.ms = 30000
retries = 10
retry.backoff.ms = 100
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
send.buffer.bytes = 131072
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = null
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
timeout.ms = 30000
value.serializer = class org.apache.kafka.common.serialization.ByteArraySerializer
14:28:47.034 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bufferpool-wait-time
14:28:47.034 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name buffer-exhausted-records
14:28:47.036 [main] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 1 to Cluster(id = null, nodes = [localhost:9092 (id: -1 rack: null)], partitions = [])
14:28:47.037 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-closed:
14:28:47.037 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-created:
14:28:47.037 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent-received:
14:28:47.037 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent:
14:28:47.037 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-received:
14:28:47.038 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name select-time:
14:28:47.038 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name io-time:
14:28:47.041 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name batch-size
14:28:47.041 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name compression-rate
14:28:47.041 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name queue-time
14:28:47.041 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name request-time
14:28:47.041 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name produce-throttle-time
14:28:47.044 [StreamThread-2] DEBUG o.apache.kafka.clients.NetworkClient - Recorded API versions for node -1: (Produce(0): 0 to 2 [usable: 2], Fetch(1): 0 to 3 [usable: 3], Offsets(2): 0 to 1 [usable: 1], Metadata(3): 0 to 2 [usable: 2], LeaderAndIsr(4): 0 [usable: 0], StopReplica(5): 0 [usable: 0], UpdateMetadata(6): 0 to 3 [usable: 3], ControlledShutdown(7): 1 [usable: 1], OffsetCommit(8): 0 to 2 [usable: 2], OffsetFetch(9): 0 to 2 [usable: 2], GroupCoordinator(10): 0 [usable: 0], JoinGroup(11): 0 to 1 [usable: 1], Heartbeat(12): 0 [usable: 0], LeaveGroup(13): 0 [usable: 0], SyncGroup(14): 0 [usable: 0], DescribeGroups(15): 0 [usable: 0], ListGroups(16): 0 [usable: 0], SaslHandshake(17): 0 [usable: 0], ApiVersions(18): 0 [usable: 0], CreateTopics(19): 0 to 1 [usable: 1], DeleteTopics(20): 0 [usable: 0])
14:28:47.041 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name records-per-request
14:28:47.044 [StreamThread-2] DEBUG o.apache.kafka.clients.NetworkClient - Sending metadata request (type=MetadataRequest, topics=<ALL>) to node -1
14:28:47.044 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name record-retries
14:28:47.044 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name errors
14:28:47.044 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name record-size-max
14:28:47.045 [main] INFO o.a.kafka.common.utils.AppInfoParser - Kafka version : 0.10.2.1-cp2
14:28:47.045 [main] INFO o.a.kafka.common.utils.AppInfoParser - Kafka commitId : 5092731dfc53f849
14:28:47.045 [kafka-producer-network-thread | map-function-scala-example-1af7fd3f-aaef-4b8a-ac1b-904af3cfe4bb-StreamThread-3-producer] DEBUG o.a.k.c.producer.internals.Sender - Starting Kafka producer I/O thread.
14:28:47.045 [main] DEBUG o.a.k.clients.producer.KafkaProducer - Kafka producer started
14:28:47.046 [main] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-3] Creating consumer client
14:28:47.046 [main] INFO o.a.k.c.consumer.ConsumerConfig - ConsumerConfig values:
auto.commit.interval.ms = 5000
auto.offset.reset = earliest
bootstrap.servers = [localhost:9092]
check.crcs = true
client.id = map-function-scala-example-1af7fd3f-aaef-4b8a-ac1b-904af3cfe4bb-StreamThread-3-consumer
connections.max.idle.ms = 540000
enable.auto.commit = false
exclude.internal.topics = true
fetch.max.bytes = 52428800
fetch.max.wait.ms = 500
fetch.min.bytes = 1
group.id = map-function-scala-example
heartbeat.interval.ms = 3000
interceptor.classes = null
internal.leave.group.on.close = false
key.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer
max.partition.fetch.bytes = 1048576
max.poll.interval.ms = 2147483647
max.poll.records = 1000
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
partition.assignment.strategy = [org.apache.kafka.streams.processor.internals.StreamPartitionAssignor]
receive.buffer.bytes = 65536
reconnect.backoff.ms = 50
request.timeout.ms = 305000
retry.backoff.ms = 100
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
send.buffer.bytes = 131072
session.timeout.ms = 10000
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = null
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
value.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer
14:28:47.046 [main] DEBUG o.a.k.clients.consumer.KafkaConsumer - Starting the Kafka consumer
14:28:47.047 [main] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 1 to Cluster(id = null, nodes = [localhost:9092 (id: -1 rack: null)], partitions = [])
14:28:47.047 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-closed:
14:28:47.048 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-created:
14:28:47.048 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent-received:
14:28:47.048 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent:
14:28:47.049 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-received:
14:28:47.049 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name select-time:
14:28:47.050 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name io-time:
14:28:47.050 [StreamThread-2] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 2 to Cluster(id = _Epo8QiSSuGbI1zi8iEXLA, nodes = [10.20.69.86:9092 (id: 0 rack: null)], partitions = [])
14:28:47.050 [main] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 1 to Cluster(id = null, nodes = [localhost:9092 (id: -1 rack: null)], partitions = [])
14:28:47.051 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-closed:
14:28:47.051 [StreamThread-2] DEBUG o.a.k.c.c.i.AbstractCoordinator - Received GroupCoordinator response ClientResponse(receivedTimeMs=1500326927050, latencyMs=29, disconnected=false, requestHeader={api_key=10,api_version=0,correlation_id=0,client_id=map-function-scala-example-72622e43-3a62-476d-8232-915f15356b54-StreamThread-2-consumer}, responseBody={error_code=0,coordinator={node_id=0,host=10.20.69.86,port=9092}}) for group map-function-scala-example
14:28:47.051 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-created:
14:28:47.051 [StreamThread-2] INFO o.a.k.c.c.i.AbstractCoordinator - Discovered coordinator 10.20.69.86:9092 (id: 2147483647 rack: null) for group map-function-scala-example.
14:28:47.051 [StreamThread-2] DEBUG o.apache.kafka.clients.NetworkClient - Initiating connection to node 2147483647 at 10.20.69.86:9092.
14:28:47.051 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent-received:
14:28:47.051 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent:
14:28:47.052 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-received:
14:28:47.052 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name select-time:
14:28:47.052 [StreamThread-2] INFO o.a.k.c.c.i.ConsumerCoordinator - Revoking previously assigned partitions [] for group map-function-scala-example
14:28:47.052 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name io-time:
14:28:47.052 [StreamThread-2] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-2] at state RUNNING: partitions [] revoked at the beginning of consumer rebalance.
14:28:47.052 [kafka-coordinator-heartbeat-thread | map-function-scala-example] DEBUG o.a.k.c.c.i.AbstractCoordinator - Heartbeat thread for group map-function-scala-example started
14:28:47.053 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name heartbeat-latency
14:28:47.053 [StreamThread-2] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-2] State transition from RUNNING to PARTITIONS_REVOKED.
14:28:47.054 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name join-latency
14:28:47.054 [StreamThread-2] INFO o.apache.kafka.streams.KafkaStreams - stream-client [map-function-scala-example-72622e43-3a62-476d-8232-915f15356b54] State transition from RUNNING to REBALANCING.
14:28:47.054 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name sync-latency
14:28:47.054 [StreamThread-2] DEBUG o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-2] suspendTasksAndState: suspending all active tasks [] and standby tasks []
14:28:47.055 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name commit-latency
14:28:47.055 [StreamThread-2] DEBUG o.a.k.clients.consumer.KafkaConsumer - Unsubscribed all topics or patterns and assigned partitions
14:28:47.055 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-fetched
14:28:47.055 [StreamThread-2] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-2] Updating suspended tasks to contain active tasks []
14:28:47.055 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name records-fetched
14:28:47.056 [StreamThread-2] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-2] Removing all active tasks []
14:28:47.056 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name fetch-latency
14:28:47.056 [StreamThread-2] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-2] Removing all standby tasks []
14:28:47.056 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name records-lag
14:28:47.056 [StreamThread-2] INFO o.a.k.c.c.i.AbstractCoordinator - (Re-)joining group map-function-scala-example
14:28:47.056 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name fetch-throttle-time
14:28:47.057 [main] INFO o.a.kafka.common.utils.AppInfoParser - Kafka version : 0.10.2.1-cp2
14:28:47.057 [StreamThread-2] DEBUG o.a.k.s.p.i.StreamPartitionAssignor - stream-thread [StreamThread-2] found [] topics possibly matching regex
14:28:47.057 [main] INFO o.a.kafka.common.utils.AppInfoParser - Kafka commitId : 5092731dfc53f849
14:28:47.057 [StreamThread-2] DEBUG o.a.k.s.processor.TopologyBuilder - stream-thread [StreamThread-2] updating builder with SubscriptionUpdates{updatedTopicSubscriptions=[]} topic(s) with possible matching regex subscription(s)
14:28:47.057 [main] DEBUG o.a.k.clients.consumer.KafkaConsumer - Kafka consumer created
14:28:47.058 [main] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-3] Creating restore consumer client
14:28:47.058 [StreamThread-2] DEBUG o.a.k.c.c.i.AbstractCoordinator - Sending JoinGroup ((type: JoinGroupRequest, groupId=map-function-scala-example, sessionTimeout=10000, rebalanceTimeout=2147483647, memberId=, protocolType=consumer, groupProtocols=org.apache.kafka.common.requests.JoinGroupRequest$ProtocolMetadata@30009b20)) to coordinator 10.20.69.86:9092 (id: 2147483647 rack: null)
14:28:47.058 [StreamThread-2] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node-2147483647.bytes-sent
14:28:47.058 [main] INFO o.a.k.c.consumer.ConsumerConfig - ConsumerConfig values:
auto.commit.interval.ms = 5000
auto.offset.reset = earliest
bootstrap.servers = [localhost:9092]
check.crcs = true
client.id = map-function-scala-example-1af7fd3f-aaef-4b8a-ac1b-904af3cfe4bb-StreamThread-3-restore-consumer
connections.max.idle.ms = 540000
enable.auto.commit = false
exclude.internal.topics = true
fetch.max.bytes = 52428800
fetch.max.wait.ms = 500
fetch.min.bytes = 1
group.id =
heartbeat.interval.ms = 3000
interceptor.classes = null
internal.leave.group.on.close = false
key.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer
max.partition.fetch.bytes = 1048576
max.poll.interval.ms = 2147483647
max.poll.records = 1000
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
partition.assignment.strategy = [class org.apache.kafka.clients.consumer.RangeAssignor]
receive.buffer.bytes = 65536
reconnect.backoff.ms = 50
request.timeout.ms = 305000
retry.backoff.ms = 100
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
send.buffer.bytes = 131072
session.timeout.ms = 10000
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = null
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
value.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer
14:28:47.058 [main] DEBUG o.a.k.clients.consumer.KafkaConsumer - Starting the Kafka consumer
14:28:47.058 [StreamThread-2] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node-2147483647.bytes-received
14:28:47.058 [StreamThread-2] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node-2147483647.latency
14:28:47.058 [main] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 1 to Cluster(id = null, nodes = [localhost:9092 (id: -1 rack: null)], partitions = [])
14:28:47.058 [StreamThread-2] DEBUG o.a.kafka.common.network.Selector - Created socket with SO_RCVBUF = 342972, SO_SNDBUF = 146988, SO_TIMEOUT = 0 to node 2147483647
14:28:47.059 [StreamThread-2] DEBUG o.apache.kafka.clients.NetworkClient - Completed connection to node 2147483647. Fetching API versions.
14:28:47.059 [StreamThread-2] DEBUG o.apache.kafka.clients.NetworkClient - Initiating API versions fetch from node 2147483647.
14:28:47.059 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-closed:
14:28:47.059 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-created:
14:28:47.059 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent-received:
14:28:47.059 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent:
14:28:47.060 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-received:
14:28:47.060 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name select-time:
14:28:47.060 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name io-time:
14:28:47.060 [StreamThread-2] DEBUG o.apache.kafka.clients.NetworkClient - Recorded API versions for node 2147483647: (Produce(0): 0 to 2 [usable: 2], Fetch(1): 0 to 3 [usable: 3], Offsets(2): 0 to 1 [usable: 1], Metadata(3): 0 to 2 [usable: 2], LeaderAndIsr(4): 0 [usable: 0], StopReplica(5): 0 [usable: 0], UpdateMetadata(6): 0 to 3 [usable: 3], ControlledShutdown(7): 1 [usable: 1], OffsetCommit(8): 0 to 2 [usable: 2], OffsetFetch(9): 0 to 2 [usable: 2], GroupCoordinator(10): 0 [usable: 0], JoinGroup(11): 0 to 1 [usable: 1], Heartbeat(12): 0 [usable: 0], LeaveGroup(13): 0 [usable: 0], SyncGroup(14): 0 [usable: 0], DescribeGroups(15): 0 [usable: 0], ListGroups(16): 0 [usable: 0], SaslHandshake(17): 0 [usable: 0], ApiVersions(18): 0 [usable: 0], CreateTopics(19): 0 to 1 [usable: 1], DeleteTopics(20): 0 [usable: 0])
14:28:47.060 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name heartbeat-latency
14:28:47.060 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name join-latency
14:28:47.060 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name sync-latency
14:28:47.061 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name commit-latency
14:28:47.061 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-fetched
14:28:47.061 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name records-fetched
14:28:47.061 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name fetch-latency
14:28:47.062 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name records-lag
14:28:47.062 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name fetch-throttle-time
14:28:47.062 [main] INFO o.a.kafka.common.utils.AppInfoParser - Kafka version : 0.10.2.1-cp2
14:28:47.062 [main] INFO o.a.kafka.common.utils.AppInfoParser - Kafka commitId : 5092731dfc53f849
14:28:47.062 [main] DEBUG o.a.k.clients.consumer.KafkaConsumer - Kafka consumer created
14:28:47.062 [main] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-3] State transition from NOT_RUNNING to RUNNING.
14:28:47.062 [main] DEBUG o.apache.kafka.streams.KafkaStreams - stream-client [map-function-scala-example-1af7fd3f-aaef-4b8a-ac1b-904af3cfe4bb] Starting Kafka Stream process.
14:28:47.063 [main] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 1 to Cluster(id = null, nodes = [localhost:9092 (id: -1 rack: null)], partitions = [])
14:28:47.063 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-closed:
14:28:47.063 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-created:
14:28:47.064 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent-received:
14:28:47.064 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent:
14:28:47.064 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-received:
14:28:47.064 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name select-time:
14:28:47.065 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name io-time:
14:28:47.065 [main] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 1 to Cluster(id = null, nodes = [localhost:9092 (id: -1 rack: null)], partitions = [])
14:28:47.065 [main] DEBUG o.apache.kafka.clients.NetworkClient - Initiating connection to node -1 at localhost:9092.
14:28:47.065 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--1.bytes-sent
14:28:47.066 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--1.bytes-received
14:28:47.066 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--1.latency
14:28:47.066 [main] DEBUG o.a.kafka.common.network.Selector - Created socket with SO_RCVBUF = 326640, SO_SNDBUF = 146988, SO_TIMEOUT = 0 to node -1
14:28:47.066 [main] DEBUG o.apache.kafka.clients.NetworkClient - Completed connection to node -1. Fetching API versions.
14:28:47.066 [main] DEBUG o.apache.kafka.clients.NetworkClient - Initiating API versions fetch from node -1.
14:28:47.068 [main] DEBUG o.apache.kafka.clients.NetworkClient - Recorded API versions for node -1: (Produce(0): 0 to 2 [usable: 2], Fetch(1): 0 to 3 [usable: 3], Offsets(2): 0 to 1 [usable: 1], Metadata(3): 0 to 2 [usable: 2], LeaderAndIsr(4): 0 [usable: 0], StopReplica(5): 0 [usable: 0], UpdateMetadata(6): 0 to 3 [usable: 3], ControlledShutdown(7): 1 [usable: 1], OffsetCommit(8): 0 to 2 [usable: 2], OffsetFetch(9): 0 to 2 [usable: 2], GroupCoordinator(10): 0 [usable: 0], JoinGroup(11): 0 to 1 [usable: 1], Heartbeat(12): 0 [usable: 0], LeaveGroup(13): 0 [usable: 0], SyncGroup(14): 0 [usable: 0], DescribeGroups(15): 0 [usable: 0], ListGroups(16): 0 [usable: 0], SaslHandshake(17): 0 [usable: 0], ApiVersions(18): 0 [usable: 0], CreateTopics(19): 0 to 1 [usable: 1], DeleteTopics(20): 0 [usable: 0])
14:28:47.172 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name connections-closed:
14:28:47.173 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name connections-created:
14:28:47.173 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name bytes-sent-received:
14:28:47.173 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name bytes-sent:
14:28:47.173 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name bytes-received:
14:28:47.173 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name select-time:
14:28:47.173 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name io-time:
14:28:47.173 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name node--1.bytes-sent
14:28:47.174 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name node--1.bytes-received
14:28:47.174 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name node--1.latency
14:28:47.174 [main] INFO o.apache.kafka.streams.KafkaStreams - stream-client [map-function-scala-example-1af7fd3f-aaef-4b8a-ac1b-904af3cfe4bb] State transition from CREATED to RUNNING.
14:28:47.174 [main] INFO o.apache.kafka.streams.KafkaStreams - stream-client [map-function-scala-example-1af7fd3f-aaef-4b8a-ac1b-904af3cfe4bb] Started Kafka Stream process
Starting stream...
14:28:47.174 [StreamThread-3] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-3] Starting
14:28:47.174 [StreamThread-3] DEBUG o.a.k.clients.consumer.KafkaConsumer - Subscribed to pattern: source-topic2
HAS NEXT test-name json-config
14:28:47.174 [StreamThread-3] DEBUG o.a.k.c.c.i.AbstractCoordinator - Sending GroupCoordinator request for group map-function-scala-example to broker localhost:9092 (id: -1 rack: null)
CREATING test-name json-config
14:28:47.174 [StreamThread-3] DEBUG o.apache.kafka.clients.NetworkClient - Initiating connection to node -1 at localhost:9092.
14:28:47.175 [StreamThread-3] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--1.bytes-sent
14:28:47.175 [main] INFO o.apache.kafka.streams.StreamsConfig - StreamsConfig values:
application.id = map-function-scala-example
application.server =
bootstrap.servers = [localhost:9092]
buffered.records.per.partition = 1000
cache.max.bytes.buffering = 10485760
client.id =
commit.interval.ms = 30000
connections.max.idle.ms = 540000
key.serde = class org.apache.kafka.common.serialization.Serdes$ByteArraySerde
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
num.standby.replicas = 0
num.stream.threads = 1
partition.grouper = class org.apache.kafka.streams.processor.DefaultPartitionGrouper
poll.ms = 100
receive.buffer.bytes = 32768
reconnect.backoff.ms = 50
replication.factor = 1
request.timeout.ms = 40000
retry.backoff.ms = 100
rocksdb.config.setter = null
security.protocol = PLAINTEXT
send.buffer.bytes = 131072
state.cleanup.delay.ms = 600000
state.dir = /tmp/kafka-streams
timestamp.extractor = class org.apache.kafka.streams.processor.FailOnInvalidTimestamp
value.serde = class org.apache.kafka.common.serialization.Serdes$StringSerde
windowstore.changelog.additional.retention.ms = 86400000
zookeeper.connect =
14:28:47.175 [StreamThread-3] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--1.bytes-received
14:28:47.175 [StreamThread-3] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--1.latency
14:28:47.175 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name thread.map-function-scala-example-4694886d-66a8-4f8b-9c31-0447e77b2e76-StreamThread-4.commit-latency
14:28:47.176 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name thread.map-function-scala-example-4694886d-66a8-4f8b-9c31-0447e77b2e76-StreamThread-4.poll-latency
14:28:47.176 [StreamThread-3] DEBUG o.a.kafka.common.network.Selector - Created socket with SO_RCVBUF = 342972, SO_SNDBUF = 146988, SO_TIMEOUT = 0 to node -1
14:28:47.176 [StreamThread-3] DEBUG o.apache.kafka.clients.NetworkClient - Completed connection to node -1. Fetching API versions.
14:28:47.176 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name thread.map-function-scala-example-4694886d-66a8-4f8b-9c31-0447e77b2e76-StreamThread-4.process-latency
14:28:47.176 [StreamThread-3] DEBUG o.apache.kafka.clients.NetworkClient - Initiating API versions fetch from node -1.
14:28:47.176 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name thread.map-function-scala-example-4694886d-66a8-4f8b-9c31-0447e77b2e76-StreamThread-4.punctuate-latency
14:28:47.176 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name thread.map-function-scala-example-4694886d-66a8-4f8b-9c31-0447e77b2e76-StreamThread-4.task-created
14:28:47.176 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name thread.map-function-scala-example-4694886d-66a8-4f8b-9c31-0447e77b2e76-StreamThread-4.task-closed
14:28:47.177 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name thread.map-function-scala-example-4694886d-66a8-4f8b-9c31-0447e77b2e76-StreamThread-4.skipped-records
14:28:47.177 [main] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-4] Creating producer client
14:28:47.177 [main] INFO o.a.k.c.producer.ProducerConfig - ProducerConfig values:
acks = 1
batch.size = 16384
block.on.buffer.full = false
bootstrap.servers = [localhost:9092]
buffer.memory = 33554432
client.id = map-function-scala-example-4694886d-66a8-4f8b-9c31-0447e77b2e76-StreamThread-4-producer
compression.type = none
connections.max.idle.ms = 540000
interceptor.classes = null
key.serializer = class org.apache.kafka.common.serialization.ByteArraySerializer
linger.ms = 100
max.block.ms = 60000
max.in.flight.requests.per.connection = 5
max.request.size = 1048576
metadata.fetch.timeout.ms = 60000
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.sample.window.ms = 30000
partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner
receive.buffer.bytes = 32768
reconnect.backoff.ms = 50
request.timeout.ms = 30000
retries = 10
retry.backoff.ms = 100
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
send.buffer.bytes = 131072
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = null
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
timeout.ms = 30000
value.serializer = class org.apache.kafka.common.serialization.ByteArraySerializer
14:28:47.177 [StreamThread-3] DEBUG o.apache.kafka.clients.NetworkClient - Recorded API versions for node -1: (Produce(0): 0 to 2 [usable: 2], Fetch(1): 0 to 3 [usable: 3], Offsets(2): 0 to 1 [usable: 1], Metadata(3): 0 to 2 [usable: 2], LeaderAndIsr(4): 0 [usable: 0], StopReplica(5): 0 [usable: 0], UpdateMetadata(6): 0 to 3 [usable: 3], ControlledShutdown(7): 1 [usable: 1], OffsetCommit(8): 0 to 2 [usable: 2], OffsetFetch(9): 0 to 2 [usable: 2], GroupCoordinator(10): 0 [usable: 0], JoinGroup(11): 0 to 1 [usable: 1], Heartbeat(12): 0 [usable: 0], LeaveGroup(13): 0 [usable: 0], SyncGroup(14): 0 [usable: 0], DescribeGroups(15): 0 [usable: 0], ListGroups(16): 0 [usable: 0], SaslHandshake(17): 0 [usable: 0], ApiVersions(18): 0 [usable: 0], CreateTopics(19): 0 to 1 [usable: 1], DeleteTopics(20): 0 [usable: 0])
14:28:47.178 [StreamThread-3] DEBUG o.apache.kafka.clients.NetworkClient - Sending metadata request (type=MetadataRequest, topics=<ALL>) to node -1
14:28:47.178 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bufferpool-wait-time
14:28:47.178 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name buffer-exhausted-records
14:28:47.178 [main] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 1 to Cluster(id = null, nodes = [localhost:9092 (id: -1 rack: null)], partitions = [])
14:28:47.178 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-closed:
14:28:47.178 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-created:
14:28:47.179 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent-received:
14:28:47.179 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent:
14:28:47.179 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-received:
14:28:47.179 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name select-time:
14:28:47.179 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name io-time:
14:28:47.179 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name batch-size
14:28:47.179 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name compression-rate
14:28:47.180 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name queue-time
14:28:47.180 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name request-time
14:28:47.180 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name produce-throttle-time
14:28:47.180 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name records-per-request
14:28:47.180 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name record-retries
14:28:47.180 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name errors
14:28:47.180 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name record-size-max
14:28:47.182 [StreamThread-3] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 2 to Cluster(id = _Epo8QiSSuGbI1zi8iEXLA, nodes = [10.20.69.86:9092 (id: 0 rack: null)], partitions = [])
14:28:47.183 [StreamThread-3] DEBUG o.a.k.c.c.i.AbstractCoordinator - Received GroupCoordinator response ClientResponse(receivedTimeMs=1500326927183, latencyMs=9, disconnected=false, requestHeader={api_key=10,api_version=0,correlation_id=0,client_id=map-function-scala-example-1af7fd3f-aaef-4b8a-ac1b-904af3cfe4bb-StreamThread-3-consumer}, responseBody={error_code=0,coordinator={node_id=0,host=10.20.69.86,port=9092}}) for group map-function-scala-example
14:28:47.183 [StreamThread-3] INFO o.a.k.c.c.i.AbstractCoordinator - Discovered coordinator 10.20.69.86:9092 (id: 2147483647 rack: null) for group map-function-scala-example.
14:28:47.183 [StreamThread-3] DEBUG o.apache.kafka.clients.NetworkClient - Initiating connection to node 2147483647 at 10.20.69.86:9092.
14:28:47.183 [main] INFO o.a.kafka.common.utils.AppInfoParser - Kafka version : 0.10.2.1-cp2
14:28:47.184 [main] INFO o.a.kafka.common.utils.AppInfoParser - Kafka commitId : 5092731dfc53f849
14:28:47.184 [main] DEBUG o.a.k.clients.producer.KafkaProducer - Kafka producer started
14:28:47.184 [main] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-4] Creating consumer client
14:28:47.184 [main] INFO o.a.k.c.consumer.ConsumerConfig - ConsumerConfig values:
auto.commit.interval.ms = 5000
auto.offset.reset = earliest
bootstrap.servers = [localhost:9092]
check.crcs = true
client.id = map-function-scala-example-4694886d-66a8-4f8b-9c31-0447e77b2e76-StreamThread-4-consumer
connections.max.idle.ms = 540000
enable.auto.commit = false
exclude.internal.topics = true
fetch.max.bytes = 52428800
fetch.max.wait.ms = 500
fetch.min.bytes = 1
group.id = map-function-scala-example
heartbeat.interval.ms = 3000
interceptor.classes = null
internal.leave.group.on.close = false
key.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer
max.partition.fetch.bytes = 1048576
max.poll.interval.ms = 2147483647
max.poll.records = 1000
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
partition.assignment.strategy = [org.apache.kafka.streams.processor.internals.StreamPartitionAssignor]
receive.buffer.bytes = 65536
reconnect.backoff.ms = 50
request.timeout.ms = 305000
retry.backoff.ms = 100
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
send.buffer.bytes = 131072
session.timeout.ms = 10000
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = null
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
value.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer
14:28:47.184 [main] DEBUG o.a.k.clients.consumer.KafkaConsumer - Starting the Kafka consumer
14:28:47.185 [main] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 1 to Cluster(id = null, nodes = [localhost:9092 (id: -1 rack: null)], partitions = [])
14:28:47.185 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-closed:
14:28:47.185 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-created:
14:28:47.185 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent-received:
14:28:47.185 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent:
14:28:47.186 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-received:
14:28:47.186 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name select-time:
14:28:47.186 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name io-time:
14:28:47.186 [kafka-producer-network-thread | map-function-scala-example-4694886d-66a8-4f8b-9c31-0447e77b2e76-StreamThread-4-producer] DEBUG o.a.k.c.producer.internals.Sender - Starting Kafka producer I/O thread.
14:28:47.186 [main] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 1 to Cluster(id = null, nodes = [localhost:9092 (id: -1 rack: null)], partitions = [])
14:28:47.187 [StreamThread-3] INFO o.a.k.c.c.i.ConsumerCoordinator - Revoking previously assigned partitions [] for group map-function-scala-example
14:28:47.187 [kafka-coordinator-heartbeat-thread | map-function-scala-example] DEBUG o.a.k.c.c.i.AbstractCoordinator - Heartbeat thread for group map-function-scala-example started
14:28:47.187 [StreamThread-3] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-3] at state RUNNING: partitions [] revoked at the beginning of consumer rebalance.
14:28:47.187 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-closed:
14:28:47.187 [StreamThread-3] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-3] State transition from RUNNING to PARTITIONS_REVOKED.
14:28:47.187 [StreamThread-3] INFO o.apache.kafka.streams.KafkaStreams - stream-client [map-function-scala-example-1af7fd3f-aaef-4b8a-ac1b-904af3cfe4bb] State transition from RUNNING to REBALANCING.
14:28:47.187 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-created:
14:28:47.187 [StreamThread-3] DEBUG o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-3] suspendTasksAndState: suspending all active tasks [] and standby tasks []
14:28:47.187 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent-received:
14:28:47.187 [StreamThread-3] DEBUG o.a.k.clients.consumer.KafkaConsumer - Unsubscribed all topics or patterns and assigned partitions
14:28:47.187 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent:
14:28:47.187 [StreamThread-3] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-3] Updating suspended tasks to contain active tasks []
14:28:47.187 [StreamThread-3] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-3] Removing all active tasks []
14:28:47.188 [StreamThread-3] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-3] Removing all standby tasks []
14:28:47.188 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-received:
14:28:47.188 [StreamThread-3] INFO o.a.k.c.c.i.AbstractCoordinator - (Re-)joining group map-function-scala-example
14:28:47.188 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name select-time:
14:28:47.188 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name io-time:
14:28:47.188 [StreamThread-3] DEBUG o.a.k.s.p.i.StreamPartitionAssignor - stream-thread [StreamThread-3] found [] topics possibly matching regex
14:28:47.188 [StreamThread-3] DEBUG o.a.k.s.processor.TopologyBuilder - stream-thread [StreamThread-3] updating builder with SubscriptionUpdates{updatedTopicSubscriptions=[]} topic(s) with possible matching regex subscription(s)
14:28:47.188 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name heartbeat-latency
14:28:47.188 [StreamThread-3] DEBUG o.a.k.c.c.i.AbstractCoordinator - Sending JoinGroup ((type: JoinGroupRequest, groupId=map-function-scala-example, sessionTimeout=10000, rebalanceTimeout=2147483647, memberId=, protocolType=consumer, groupProtocols=org.apache.kafka.common.requests.JoinGroupRequest$ProtocolMetadata@27f95d9d)) to coordinator 10.20.69.86:9092 (id: 2147483647 rack: null)
14:28:47.188 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name join-latency
14:28:47.188 [StreamThread-3] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node-2147483647.bytes-sent
14:28:47.188 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name sync-latency
14:28:47.189 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name commit-latency
14:28:47.189 [StreamThread-3] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node-2147483647.bytes-received
14:28:47.189 [StreamThread-3] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node-2147483647.latency
14:28:47.189 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-fetched
14:28:47.189 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name records-fetched
14:28:47.189 [StreamThread-3] DEBUG o.a.kafka.common.network.Selector - Created socket with SO_RCVBUF = 342972, SO_SNDBUF = 146988, SO_TIMEOUT = 0 to node 2147483647
14:28:47.189 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name fetch-latency
14:28:47.189 [StreamThread-3] DEBUG o.apache.kafka.clients.NetworkClient - Completed connection to node 2147483647. Fetching API versions.
14:28:47.189 [StreamThread-3] DEBUG o.apache.kafka.clients.NetworkClient - Initiating API versions fetch from node 2147483647.
14:28:47.189 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name records-lag
14:28:47.189 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name fetch-throttle-time
14:28:47.190 [main] INFO o.a.kafka.common.utils.AppInfoParser - Kafka version : 0.10.2.1-cp2
14:28:47.190 [main] INFO o.a.kafka.common.utils.AppInfoParser - Kafka commitId : 5092731dfc53f849
14:28:47.190 [main] DEBUG o.a.k.clients.consumer.KafkaConsumer - Kafka consumer created
14:28:47.190 [main] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-4] Creating restore consumer client
14:28:47.190 [main] INFO o.a.k.c.consumer.ConsumerConfig - ConsumerConfig values:
auto.commit.interval.ms = 5000
auto.offset.reset = earliest
bootstrap.servers = [localhost:9092]
check.crcs = true
client.id = map-function-scala-example-4694886d-66a8-4f8b-9c31-0447e77b2e76-StreamThread-4-restore-consumer
connections.max.idle.ms = 540000
enable.auto.commit = false
exclude.internal.topics = true
fetch.max.bytes = 52428800
fetch.max.wait.ms = 500
fetch.min.bytes = 1
group.id =
heartbeat.interval.ms = 3000
interceptor.classes = null
internal.leave.group.on.close = false
key.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer
max.partition.fetch.bytes = 1048576
max.poll.interval.ms = 2147483647
max.poll.records = 1000
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
partition.assignment.strategy = [class org.apache.kafka.clients.consumer.RangeAssignor]
receive.buffer.bytes = 65536
reconnect.backoff.ms = 50
request.timeout.ms = 305000
retry.backoff.ms = 100
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.mechanism = GSSAPI
security.protocol = PLAINTEXT
send.buffer.bytes = 131072
session.timeout.ms = 10000
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = null
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
value.deserializer = class org.apache.kafka.common.serialization.ByteArrayDeserializer
14:28:47.190 [main] DEBUG o.a.k.clients.consumer.KafkaConsumer - Starting the Kafka consumer
14:28:47.191 [main] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 1 to Cluster(id = null, nodes = [localhost:9092 (id: -1 rack: null)], partitions = [])
14:28:47.191 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-closed:
14:28:47.191 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-created:
14:28:47.191 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent-received:
14:28:47.191 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent:
14:28:47.191 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-received:
14:28:47.192 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name select-time:
14:28:47.192 [StreamThread-3] DEBUG o.apache.kafka.clients.NetworkClient - Recorded API versions for node 2147483647: (Produce(0): 0 to 2 [usable: 2], Fetch(1): 0 to 3 [usable: 3], Offsets(2): 0 to 1 [usable: 1], Metadata(3): 0 to 2 [usable: 2], LeaderAndIsr(4): 0 [usable: 0], StopReplica(5): 0 [usable: 0], UpdateMetadata(6): 0 to 3 [usable: 3], ControlledShutdown(7): 1 [usable: 1], OffsetCommit(8): 0 to 2 [usable: 2], OffsetFetch(9): 0 to 2 [usable: 2], GroupCoordinator(10): 0 [usable: 0], JoinGroup(11): 0 to 1 [usable: 1], Heartbeat(12): 0 [usable: 0], LeaveGroup(13): 0 [usable: 0], SyncGroup(14): 0 [usable: 0], DescribeGroups(15): 0 [usable: 0], ListGroups(16): 0 [usable: 0], SaslHandshake(17): 0 [usable: 0], ApiVersions(18): 0 [usable: 0], CreateTopics(19): 0 to 1 [usable: 1], DeleteTopics(20): 0 [usable: 0])
14:28:47.192 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name io-time:
14:28:47.192 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name heartbeat-latency
14:28:47.193 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name join-latency
14:28:47.193 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name sync-latency
14:28:47.193 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name commit-latency
14:28:47.193 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-fetched
14:28:47.194 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name records-fetched
14:28:47.194 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name fetch-latency
14:28:47.194 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name records-lag
14:28:47.194 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name fetch-throttle-time
14:28:47.194 [main] INFO o.a.kafka.common.utils.AppInfoParser - Kafka version : 0.10.2.1-cp2
14:28:47.194 [main] INFO o.a.kafka.common.utils.AppInfoParser - Kafka commitId : 5092731dfc53f849
14:28:47.194 [main] DEBUG o.a.k.clients.consumer.KafkaConsumer - Kafka consumer created
14:28:47.194 [main] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-4] State transition from NOT_RUNNING to RUNNING.
14:28:47.194 [main] DEBUG o.apache.kafka.streams.KafkaStreams - stream-client [map-function-scala-example-4694886d-66a8-4f8b-9c31-0447e77b2e76] Starting Kafka Stream process.
14:28:47.195 [main] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 1 to Cluster(id = null, nodes = [localhost:9092 (id: -1 rack: null)], partitions = [])
14:28:47.195 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-closed:
14:28:47.195 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name connections-created:
14:28:47.195 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent-received:
14:28:47.195 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-sent:
14:28:47.196 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name bytes-received:
14:28:47.196 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name select-time:
14:28:47.196 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name io-time:
14:28:47.196 [main] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 1 to Cluster(id = null, nodes = [localhost:9092 (id: -1 rack: null)], partitions = [])
14:28:47.196 [main] DEBUG o.apache.kafka.clients.NetworkClient - Initiating connection to node -1 at localhost:9092.
14:28:47.197 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--1.bytes-sent
14:28:47.197 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--1.bytes-received
14:28:47.197 [main] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--1.latency
14:28:47.198 [main] DEBUG o.a.kafka.common.network.Selector - Created socket with SO_RCVBUF = 326640, SO_SNDBUF = 146988, SO_TIMEOUT = 0 to node -1
14:28:47.198 [main] DEBUG o.apache.kafka.clients.NetworkClient - Completed connection to node -1. Fetching API versions.
14:28:47.198 [main] DEBUG o.apache.kafka.clients.NetworkClient - Initiating API versions fetch from node -1.
14:28:47.199 [main] DEBUG o.apache.kafka.clients.NetworkClient - Recorded API versions for node -1: (Produce(0): 0 to 2 [usable: 2], Fetch(1): 0 to 3 [usable: 3], Offsets(2): 0 to 1 [usable: 1], Metadata(3): 0 to 2 [usable: 2], LeaderAndIsr(4): 0 [usable: 0], StopReplica(5): 0 [usable: 0], UpdateMetadata(6): 0 to 3 [usable: 3], ControlledShutdown(7): 1 [usable: 1], OffsetCommit(8): 0 to 2 [usable: 2], OffsetFetch(9): 0 to 2 [usable: 2], GroupCoordinator(10): 0 [usable: 0], JoinGroup(11): 0 to 1 [usable: 1], Heartbeat(12): 0 [usable: 0], LeaveGroup(13): 0 [usable: 0], SyncGroup(14): 0 [usable: 0], DescribeGroups(15): 0 [usable: 0], ListGroups(16): 0 [usable: 0], SaslHandshake(17): 0 [usable: 0], ApiVersions(18): 0 [usable: 0], CreateTopics(19): 0 to 1 [usable: 1], DeleteTopics(20): 0 [usable: 0])
14:28:47.306 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name connections-closed:
14:28:47.306 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name connections-created:
14:28:47.306 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name bytes-sent-received:
14:28:47.306 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name bytes-sent:
14:28:47.306 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name bytes-received:
14:28:47.306 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name select-time:
14:28:47.306 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name io-time:
14:28:47.306 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name node--1.bytes-sent
14:28:47.306 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name node--1.bytes-received
14:28:47.307 [main] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name node--1.latency
14:28:47.307 [main] INFO o.apache.kafka.streams.KafkaStreams - stream-client [map-function-scala-example-4694886d-66a8-4f8b-9c31-0447e77b2e76] State transition from CREATED to RUNNING.
14:28:47.307 [main] INFO o.apache.kafka.streams.KafkaStreams - stream-client [map-function-scala-example-4694886d-66a8-4f8b-9c31-0447e77b2e76] Started Kafka Stream process
Starting stream...
14:28:47.307 [StreamThread-4] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-4] Starting
14:28:47.307 [StreamThread-4] DEBUG o.a.k.clients.consumer.KafkaConsumer - Subscribed to pattern: test-name
14:28:47.307 [main] DEBUG o.apache.kafka.streams.KafkaStreams - stream-client [map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191] Stopping Kafka Stream process.
14:28:47.307 [main] INFO o.apache.kafka.streams.KafkaStreams - stream-client [map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191] State transition from RUNNING to PENDING_SHUTDOWN.
14:28:47.307 [StreamThread-4] DEBUG o.a.k.c.c.i.AbstractCoordinator - Sending GroupCoordinator request for group map-function-scala-example to broker localhost:9092 (id: -1 rack: null)
14:28:47.307 [StreamThread-4] DEBUG o.apache.kafka.clients.NetworkClient - Initiating connection to node -1 at localhost:9092.
14:28:47.308 [StreamThread-4] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--1.bytes-sent
14:28:47.308 [StreamThread-4] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--1.bytes-received
14:28:47.308 [StreamThread-4] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node--1.latency
14:28:47.309 [StreamThread-4] DEBUG o.a.kafka.common.network.Selector - Created socket with SO_RCVBUF = 342972, SO_SNDBUF = 146988, SO_TIMEOUT = 0 to node -1
14:28:47.309 [StreamThread-4] DEBUG o.apache.kafka.clients.NetworkClient - Completed connection to node -1. Fetching API versions.
14:28:47.309 [StreamThread-4] DEBUG o.apache.kafka.clients.NetworkClient - Initiating API versions fetch from node -1.
14:28:47.309 [kafka-streams-close-thread] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] Informed thread to shut down
14:28:47.309 [kafka-streams-close-thread] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] State transition from RUNNING to PENDING_SHUTDOWN.
14:28:47.310 [StreamThread-4] DEBUG o.apache.kafka.clients.NetworkClient - Recorded API versions for node -1: (Produce(0): 0 to 2 [usable: 2], Fetch(1): 0 to 3 [usable: 3], Offsets(2): 0 to 1 [usable: 1], Metadata(3): 0 to 2 [usable: 2], LeaderAndIsr(4): 0 [usable: 0], StopReplica(5): 0 [usable: 0], UpdateMetadata(6): 0 to 3 [usable: 3], ControlledShutdown(7): 1 [usable: 1], OffsetCommit(8): 0 to 2 [usable: 2], OffsetFetch(9): 0 to 2 [usable: 2], GroupCoordinator(10): 0 [usable: 0], JoinGroup(11): 0 to 1 [usable: 1], Heartbeat(12): 0 [usable: 0], LeaveGroup(13): 0 [usable: 0], SyncGroup(14): 0 [usable: 0], DescribeGroups(15): 0 [usable: 0], ListGroups(16): 0 [usable: 0], SaslHandshake(17): 0 [usable: 0], ApiVersions(18): 0 [usable: 0], CreateTopics(19): 0 to 1 [usable: 1], DeleteTopics(20): 0 [usable: 0])
14:28:47.311 [StreamThread-4] DEBUG o.apache.kafka.clients.NetworkClient - Sending metadata request (type=MetadataRequest, topics=<ALL>) to node -1
14:28:47.314 [StreamThread-4] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 2 to Cluster(id = _Epo8QiSSuGbI1zi8iEXLA, nodes = [10.20.69.86:9092 (id: 0 rack: null)], partitions = [Partition(topic = test-name, partition = 0, leader = 0, replicas = [0], isr = [0])])
14:28:47.315 [StreamThread-4] DEBUG o.a.k.c.c.i.AbstractCoordinator - Received GroupCoordinator response ClientResponse(receivedTimeMs=1500326927315, latencyMs=8, disconnected=false, requestHeader={api_key=10,api_version=0,correlation_id=0,client_id=map-function-scala-example-4694886d-66a8-4f8b-9c31-0447e77b2e76-StreamThread-4-consumer}, responseBody={error_code=0,coordinator={node_id=0,host=10.20.69.86,port=9092}}) for group map-function-scala-example
14:28:47.315 [StreamThread-4] INFO o.a.k.c.c.i.AbstractCoordinator - Discovered coordinator 10.20.69.86:9092 (id: 2147483647 rack: null) for group map-function-scala-example.
14:28:47.316 [StreamThread-4] DEBUG o.apache.kafka.clients.NetworkClient - Initiating connection to node 2147483647 at 10.20.69.86:9092.
14:28:47.316 [StreamThread-4] INFO o.a.k.c.c.i.ConsumerCoordinator - Revoking previously assigned partitions [] for group map-function-scala-example
14:28:47.316 [StreamThread-4] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-4] at state RUNNING: partitions [] revoked at the beginning of consumer rebalance.
14:28:47.317 [StreamThread-4] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-4] State transition from RUNNING to PARTITIONS_REVOKED.
14:28:47.317 [kafka-coordinator-heartbeat-thread | map-function-scala-example] DEBUG o.a.k.c.c.i.AbstractCoordinator - Heartbeat thread for group map-function-scala-example started
14:28:47.317 [StreamThread-4] INFO o.apache.kafka.streams.KafkaStreams - stream-client [map-function-scala-example-4694886d-66a8-4f8b-9c31-0447e77b2e76] State transition from RUNNING to REBALANCING.
14:28:47.317 [StreamThread-4] DEBUG o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-4] suspendTasksAndState: suspending all active tasks [] and standby tasks []
14:28:47.317 [StreamThread-4] DEBUG o.a.k.clients.consumer.KafkaConsumer - Unsubscribed all topics or patterns and assigned partitions
14:28:47.317 [StreamThread-4] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-4] Updating suspended tasks to contain active tasks []
14:28:47.317 [StreamThread-4] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-4] Removing all active tasks []
14:28:47.317 [StreamThread-4] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-4] Removing all standby tasks []
14:28:47.317 [StreamThread-4] INFO o.a.k.c.c.i.AbstractCoordinator - (Re-)joining group map-function-scala-example
14:28:47.318 [StreamThread-4] DEBUG o.a.k.s.p.i.StreamPartitionAssignor - stream-thread [StreamThread-4] found [test-name] topics possibly matching regex
14:28:47.318 [StreamThread-4] DEBUG o.a.k.s.processor.TopologyBuilder - stream-thread [StreamThread-4] updating builder with SubscriptionUpdates{updatedTopicSubscriptions=[test-name]} topic(s) with possible matching regex subscription(s)
14:28:47.318 [StreamThread-4] DEBUG o.a.k.c.c.i.AbstractCoordinator - Sending JoinGroup ((type: JoinGroupRequest, groupId=map-function-scala-example, sessionTimeout=10000, rebalanceTimeout=2147483647, memberId=, protocolType=consumer, groupProtocols=org.apache.kafka.common.requests.JoinGroupRequest$ProtocolMetadata@69a5ef25)) to coordinator 10.20.69.86:9092 (id: 2147483647 rack: null)
14:28:47.318 [StreamThread-4] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node-2147483647.bytes-sent
14:28:47.319 [StreamThread-4] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node-2147483647.bytes-received
14:28:47.319 [StreamThread-4] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node-2147483647.latency
14:28:47.319 [StreamThread-4] DEBUG o.a.kafka.common.network.Selector - Created socket with SO_RCVBUF = 342972, SO_SNDBUF = 146988, SO_TIMEOUT = 0 to node 2147483647
14:28:47.319 [StreamThread-4] DEBUG o.apache.kafka.clients.NetworkClient - Completed connection to node 2147483647. Fetching API versions.
14:28:47.319 [StreamThread-4] DEBUG o.apache.kafka.clients.NetworkClient - Initiating API versions fetch from node 2147483647.
14:28:47.322 [StreamThread-4] DEBUG o.apache.kafka.clients.NetworkClient - Recorded API versions for node 2147483647: (Produce(0): 0 to 2 [usable: 2], Fetch(1): 0 to 3 [usable: 3], Offsets(2): 0 to 1 [usable: 1], Metadata(3): 0 to 2 [usable: 2], LeaderAndIsr(4): 0 [usable: 0], StopReplica(5): 0 [usable: 0], UpdateMetadata(6): 0 to 3 [usable: 3], ControlledShutdown(7): 1 [usable: 1], OffsetCommit(8): 0 to 2 [usable: 2], OffsetFetch(9): 0 to 2 [usable: 2], GroupCoordinator(10): 0 [usable: 0], JoinGroup(11): 0 to 1 [usable: 1], Heartbeat(12): 0 [usable: 0], LeaveGroup(13): 0 [usable: 0], SyncGroup(14): 0 [usable: 0], DescribeGroups(15): 0 [usable: 0], ListGroups(16): 0 [usable: 0], SaslHandshake(17): 0 [usable: 0], ApiVersions(18): 0 [usable: 0], CreateTopics(19): 0 to 1 [usable: 1], DeleteTopics(20): 0 [usable: 0])
14:28:47.355 [StreamThread-1] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] Shutting down at user request
14:28:47.355 [StreamThread-1] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] Shutting down
14:28:47.355 [StreamThread-1] DEBUG o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] shutdownTasksAndState: shutting downactive tasks [], standby tasks [], suspended tasks [], and suspended standby tasks []
14:28:47.357 [StreamThread-1] DEBUG o.a.k.clients.consumer.KafkaConsumer - Unsubscribed all topics or patterns and assigned partitions
14:28:47.357 [StreamThread-1] INFO o.a.k.clients.producer.KafkaProducer - Closing the Kafka producer with timeoutMillis = 9223372036854775807 ms.
14:28:47.358 [kafka-producer-network-thread | map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1-producer] DEBUG o.a.k.c.producer.internals.Sender - Beginning shutdown of Kafka producer I/O thread, sending remaining records.
14:28:47.358 [kafka-producer-network-thread | map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1-producer] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name connections-closed:
14:28:47.358 [kafka-producer-network-thread | map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1-producer] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name connections-created:
14:28:47.358 [kafka-producer-network-thread | map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1-producer] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name bytes-sent-received:
14:28:47.358 [kafka-producer-network-thread | map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1-producer] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name bytes-sent:
14:28:47.359 [kafka-producer-network-thread | map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1-producer] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name bytes-received:
14:28:47.359 [kafka-producer-network-thread | map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1-producer] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name select-time:
14:28:47.359 [kafka-producer-network-thread | map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1-producer] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name io-time:
14:28:47.359 [kafka-producer-network-thread | map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1-producer] DEBUG o.a.k.c.producer.internals.Sender - Shutdown of Kafka producer I/O thread has completed.
14:28:47.359 [StreamThread-1] DEBUG o.a.k.clients.producer.KafkaProducer - The Kafka producer has closed.
14:28:47.359 [kafka-coordinator-heartbeat-thread | map-function-scala-example] DEBUG o.a.k.c.c.i.AbstractCoordinator - Heartbeat thread for group map-function-scala-example has closed
14:28:47.360 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name connections-closed:
14:28:47.360 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name connections-created:
14:28:47.360 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name bytes-sent-received:
14:28:47.360 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name bytes-sent:
14:28:47.361 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name bytes-received:
14:28:47.361 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name select-time:
14:28:47.361 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name io-time:
14:28:47.361 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name node--1.bytes-sent
14:28:47.361 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name node--1.bytes-received
14:28:47.361 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name node--1.latency
14:28:47.361 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name node-2147483647.bytes-sent
14:28:47.361 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name node-2147483647.bytes-received
14:28:47.361 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name node-2147483647.latency
14:28:47.361 [StreamThread-1] DEBUG o.a.k.clients.consumer.KafkaConsumer - The Kafka consumer has closed.
14:28:47.362 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name connections-closed:
14:28:47.362 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name connections-created:
14:28:47.362 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name bytes-sent-received:
14:28:47.362 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name bytes-sent:
14:28:47.362 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name bytes-received:
14:28:47.362 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name select-time:
14:28:47.362 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name io-time:
14:28:47.362 [StreamThread-1] DEBUG o.a.k.clients.consumer.KafkaConsumer - The Kafka consumer has closed.
14:28:47.362 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name connections-closed:
14:28:47.362 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name connections-created:
14:28:47.362 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name bytes-sent-received:
14:28:47.363 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name bytes-sent:
14:28:47.363 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name bytes-received:
14:28:47.363 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name select-time:
14:28:47.363 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name io-time:
14:28:47.363 [StreamThread-1] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] Removing all active tasks []
14:28:47.363 [StreamThread-1] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] Removing all standby tasks []
14:28:47.363 [StreamThread-1] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] Stream thread shutdown complete
14:28:47.363 [StreamThread-1] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] State transition from PENDING_SHUTDOWN to NOT_RUNNING.
14:28:47.363 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name thread.map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1.commit-latency
14:28:47.363 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name thread.map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1.poll-latency
14:28:47.363 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name thread.map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1.process-latency
14:28:47.363 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name thread.map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1.punctuate-latency
14:28:47.364 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name thread.map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1.task-created
14:28:47.364 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name thread.map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1.task-closed
14:28:47.364 [StreamThread-1] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name thread.map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-StreamThread-1.skipped-records
14:28:47.365 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name topic.StreamConfig2.bytes-fetched
14:28:47.365 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name topic.StreamConfig2.records-fetched
14:28:47.366 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name StreamConfig2-0.records-lag
14:28:47.366 [GlobalStreamThread] DEBUG o.a.k.c.consumer.internals.Fetcher - Sending fetch for partitions [StreamConfig2-0] to broker 10.20.69.86:9092 (id: 0 rack: null)
14:28:47.378 [GlobalStreamThread] DEBUG o.a.k.s.p.i.GlobalStreamThread - Shutting down GlobalStreamThread at user request
14:28:47.379 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name connections-closed:
14:28:47.380 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name connections-created:
14:28:47.380 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name bytes-sent-received:
14:28:47.380 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name bytes-sent:
14:28:47.380 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name bytes-received:
14:28:47.380 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name select-time:
14:28:47.380 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name io-time:
14:28:47.380 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name node--1.bytes-sent
14:28:47.380 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name node--1.bytes-received
14:28:47.381 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name node--1.latency
14:28:47.381 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name node-0.bytes-sent
14:28:47.381 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name node-0.bytes-received
14:28:47.381 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name node-0.latency
14:28:47.381 [GlobalStreamThread] DEBUG o.a.k.clients.consumer.KafkaConsumer - The Kafka consumer has closed.
14:28:47.381 [GlobalStreamThread] DEBUG o.a.k.s.p.i.GlobalStateManagerImpl - Closing global storage engine test-store
14:28:47.381 [GlobalStreamThread] DEBUG o.a.k.s.state.internals.ThreadCache - Thread map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191-GlobalStreamThread cache stats on flush: #puts=0, #gets=0, #evicts=0, #flushes=1
14:28:47.383 [GlobalStreamThread] DEBUG o.a.kafka.common.metrics.Metrics - Removed sensor with name -1_-1-test-store-hitRatio
14:28:47.387 [kafka-streams-close-thread] INFO o.apache.kafka.streams.KafkaStreams - stream-client [map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191] Stopped Kafka Streams process.
14:28:47.388 [main] INFO o.apache.kafka.streams.KafkaStreams - stream-client [map-function-scala-example-f215d043-5170-41a5-8bf1-99d7c5873191] State transition from PENDING_SHUTDOWN to NOT_RUNNING.
14:28:49.545 [kafka-streams-test-service-akka.actor.default-dispatcher-3] INFO init - Admin interface listening on /0:0:0:0:0:0:0:0:8001
14:28:49.591 [kafka-streams-test-service-akka.actor.default-dispatcher-3] INFO init - Server binding successful! Starting up.
14:28:49.591 [kafka-streams-test-service-akka.actor.default-dispatcher-3] INFO init - Listening on /0:0:0:0:0:0:0:0:8000
14:28:56.931 [StreamThread-3] DEBUG o.a.k.c.c.i.AbstractCoordinator - Received successful JoinGroup response for group map-function-scala-example: {error_code=0,generation_id=32,group_protocol=stream,leader_id=map-function-scala-example-1af7fd3f-aaef-4b8a-ac1b-904af3cfe4bb-StreamThread-3-consumer-80aa1f3d-370f-4d3e-8353-e8d403f9ad3c,member_id=map-function-scala-example-1af7fd3f-aaef-4b8a-ac1b-904af3cfe4bb-StreamThread-3-consumer-80aa1f3d-370f-4d3e-8353-e8d403f9ad3c,members=[{member_id=map-function-scala-example-1af7fd3f-aaef-4b8a-ac1b-904af3cfe4bb-StreamThread-3-consumer-80aa1f3d-370f-4d3e-8353-e8d403f9ad3c,member_metadata=java.nio.HeapByteBuffer[pos=0 lim=50 cap=421]},{member_id=map-function-scala-example-72622e43-3a62-476d-8232-915f15356b54-StreamThread-2-consumer-06157563-8f80-49ed-b29f-73f877782dc6,member_metadata=java.nio.HeapByteBuffer[pos=0 lim=50 cap=241]},{member_id=map-function-scala-example-4694886d-66a8-4f8b-9c31-0447e77b2e76-StreamThread-4-consumer-9b2b7efc-e084-4b77-a942-689efd1ab1f4,member_metadata=java.nio.HeapByteBuffer[pos=0 lim=61 cap=61]}]}
14:28:56.932 [StreamThread-3] DEBUG o.apache.kafka.clients.NetworkClient - Initialize connection to node 0 for sending metadata request
14:28:56.932 [StreamThread-3] DEBUG o.apache.kafka.clients.NetworkClient - Initiating connection to node 0 at 10.20.69.86:9092.
14:28:56.932 [StreamThread-4] DEBUG o.a.k.c.c.i.AbstractCoordinator - Received successful JoinGroup response for group map-function-scala-example: {error_code=0,generation_id=32,group_protocol=stream,leader_id=map-function-scala-example-1af7fd3f-aaef-4b8a-ac1b-904af3cfe4bb-StreamThread-3-consumer-80aa1f3d-370f-4d3e-8353-e8d403f9ad3c,member_id=map-function-scala-example-4694886d-66a8-4f8b-9c31-0447e77b2e76-StreamThread-4-consumer-9b2b7efc-e084-4b77-a942-689efd1ab1f4,members=[]}
14:28:56.932 [StreamThread-2] DEBUG o.a.k.c.c.i.AbstractCoordinator - Received successful JoinGroup response for group map-function-scala-example: {error_code=0,generation_id=32,group_protocol=stream,leader_id=map-function-scala-example-1af7fd3f-aaef-4b8a-ac1b-904af3cfe4bb-StreamThread-3-consumer-80aa1f3d-370f-4d3e-8353-e8d403f9ad3c,member_id=map-function-scala-example-72622e43-3a62-476d-8232-915f15356b54-StreamThread-2-consumer-06157563-8f80-49ed-b29f-73f877782dc6,members=[]}
14:28:56.932 [StreamThread-4] DEBUG o.a.k.c.c.i.AbstractCoordinator - Sending follower SyncGroup for group map-function-scala-example to coordinator 10.20.69.86:9092 (id: 2147483647 rack: null): (type=SyncGroupRequest, groupId=map-function-scala-example, generationId=32, memberId=map-function-scala-example-4694886d-66a8-4f8b-9c31-0447e77b2e76-StreamThread-4-consumer-9b2b7efc-e084-4b77-a942-689efd1ab1f4, groupAssignment=)
14:28:56.932 [StreamThread-2] DEBUG o.a.k.c.c.i.AbstractCoordinator - Sending follower SyncGroup for group map-function-scala-example to coordinator 10.20.69.86:9092 (id: 2147483647 rack: null): (type=SyncGroupRequest, groupId=map-function-scala-example, generationId=32, memberId=map-function-scala-example-72622e43-3a62-476d-8232-915f15356b54-StreamThread-2-consumer-06157563-8f80-49ed-b29f-73f877782dc6, groupAssignment=)
14:28:56.934 [StreamThread-3] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node-0.bytes-sent
14:28:56.935 [StreamThread-3] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node-0.bytes-received
14:28:56.936 [StreamThread-3] DEBUG o.a.kafka.common.metrics.Metrics - Added sensor with name node-0.latency
14:28:56.936 [StreamThread-3] DEBUG o.a.kafka.common.network.Selector - Created socket with SO_RCVBUF = 342972, SO_SNDBUF = 146988, SO_TIMEOUT = 0 to node 0
14:28:56.936 [StreamThread-3] DEBUG o.apache.kafka.clients.NetworkClient - Completed connection to node 0. Fetching API versions.
14:28:56.936 [StreamThread-3] DEBUG o.apache.kafka.clients.NetworkClient - Initiating API versions fetch from node 0.
14:28:56.938 [StreamThread-3] DEBUG o.apache.kafka.clients.NetworkClient - Recorded API versions for node 0: (Produce(0): 0 to 2 [usable: 2], Fetch(1): 0 to 3 [usable: 3], Offsets(2): 0 to 1 [usable: 1], Metadata(3): 0 to 2 [usable: 2], LeaderAndIsr(4): 0 [usable: 0], StopReplica(5): 0 [usable: 0], UpdateMetadata(6): 0 to 3 [usable: 3], ControlledShutdown(7): 1 [usable: 1], OffsetCommit(8): 0 to 2 [usable: 2], OffsetFetch(9): 0 to 2 [usable: 2], GroupCoordinator(10): 0 [usable: 0], JoinGroup(11): 0 to 1 [usable: 1], Heartbeat(12): 0 [usable: 0], LeaveGroup(13): 0 [usable: 0], SyncGroup(14): 0 [usable: 0], DescribeGroups(15): 0 [usable: 0], ListGroups(16): 0 [usable: 0], SaslHandshake(17): 0 [usable: 0], ApiVersions(18): 0 [usable: 0], CreateTopics(19): 0 to 1 [usable: 1], DeleteTopics(20): 0 [usable: 0])
14:28:56.939 [StreamThread-3] DEBUG o.apache.kafka.clients.NetworkClient - Sending metadata request (type=MetadataRequest, topics=<ALL>) to node 0
14:28:56.944 [StreamThread-3] DEBUG org.apache.kafka.clients.Metadata - Updated cluster metadata version 3 to Cluster(id = _Epo8QiSSuGbI1zi8iEXLA, nodes = [10.20.69.86:9092 (id: 0 rack: null)], partitions = [Partition(topic = test-name, partition = 0, leader = 0, replicas = [0], isr = [0])])
14:28:56.944 [StreamThread-3] DEBUG o.a.k.c.c.i.ConsumerCoordinator - Performing assignment for group map-function-scala-example using strategy stream with subscriptions {map-function-scala-example-4694886d-66a8-4f8b-9c31-0447e77b2e76-StreamThread-4-consumer-9b2b7efc-e084-4b77-a942-689efd1ab1f4=Subscription(topics=[test-name]), map-function-scala-example-1af7fd3f-aaef-4b8a-ac1b-904af3cfe4bb-StreamThread-3-consumer-80aa1f3d-370f-4d3e-8353-e8d403f9ad3c=Subscription(topics=[]), map-function-scala-example-72622e43-3a62-476d-8232-915f15356b54-StreamThread-2-consumer-06157563-8f80-49ed-b29f-73f877782dc6=Subscription(topics=[])}
14:28:56.944 [StreamThread-3] INFO o.a.k.s.p.i.StreamPartitionAssignor - stream-thread [StreamThread-3] Constructed client metadata {4694886d-66a8-4f8b-9c31-0447e77b2e76=ClientMetadata{hostInfo=null, consumers=[map-function-scala-example-4694886d-66a8-4f8b-9c31-0447e77b2e76-StreamThread-4-consumer-9b2b7efc-e084-4b77-a942-689efd1ab1f4], state=[activeTasks: ([]) assignedTasks: ([]) prevActiveTasks: ([]) prevAssignedTasks: ([0_0]) capacity: 1.0 cost: 0.0]}, 72622e43-3a62-476d-8232-915f15356b54=ClientMetadata{hostInfo=null, consumers=[map-function-scala-example-72622e43-3a62-476d-8232-915f15356b54-StreamThread-2-consumer-06157563-8f80-49ed-b29f-73f877782dc6], state=[activeTasks: ([]) assignedTasks: ([]) prevActiveTasks: ([]) prevAssignedTasks: ([0_0]) capacity: 1.0 cost: 0.0]}, 1af7fd3f-aaef-4b8a-ac1b-904af3cfe4bb=ClientMetadata{hostInfo=null, consumers=[map-function-scala-example-1af7fd3f-aaef-4b8a-ac1b-904af3cfe4bb-StreamThread-3-consumer-80aa1f3d-370f-4d3e-8353-e8d403f9ad3c], state=[activeTasks: ([]) assignedTasks: ([]) prevActiveTasks: ([]) prevAssignedTasks: ([0_0]) capacity: 1.0 cost: 0.0]}} from the member subscriptions.
14:28:56.945 [StreamThread-3] DEBUG o.a.k.s.p.i.StreamPartitionAssignor - stream-thread [StreamThread-3] Starting to validate internal topics in partition assignor.
14:28:56.945 [StreamThread-3] INFO o.a.k.s.p.i.StreamPartitionAssignor - stream-thread [StreamThread-3] Completed validating internal topics in partition assignor
14:28:56.945 [StreamThread-3] DEBUG o.a.k.s.p.i.StreamPartitionAssignor - stream-thread [StreamThread-3] Created repartition topics [] from the parsed topology.
14:28:56.945 [StreamThread-3] INFO o.a.k.s.p.DefaultPartitionGrouper - Skipping assigning topic source-topic2 to tasks since its metadata is not available yet
14:28:56.945 [StreamThread-3] WARN o.a.k.s.p.i.StreamPartitionAssignor - stream-thread [StreamThread-3] No partitions found for topic source-topic2
14:28:56.945 [StreamThread-3] DEBUG o.a.k.s.p.i.StreamPartitionAssignor - stream-thread [StreamThread-3] Starting to validate internal topics in partition assignor.
14:28:56.945 [StreamThread-3] INFO o.a.k.s.p.i.StreamPartitionAssignor - stream-thread [StreamThread-3] Completed validating internal topics in partition assignor
14:28:56.945 [StreamThread-3] DEBUG o.a.k.s.p.i.StreamPartitionAssignor - stream-thread [StreamThread-3] Created state changelog topics {} from the parsed topology.
14:28:56.945 [StreamThread-3] DEBUG o.a.k.s.p.i.StreamPartitionAssignor - stream-thread [StreamThread-3] Assigning tasks [] to clients {4694886d-66a8-4f8b-9c31-0447e77b2e76=[activeTasks: ([]) assignedTasks: ([]) prevActiveTasks: ([]) prevAssignedTasks: ([0_0]) capacity: 1.0 cost: 0.0], 72622e43-3a62-476d-8232-915f15356b54=[activeTasks: ([]) assignedTasks: ([]) prevActiveTasks: ([]) prevAssignedTasks: ([0_0]) capacity: 1.0 cost: 0.0], 1af7fd3f-aaef-4b8a-ac1b-904af3cfe4bb=[activeTasks: ([]) assignedTasks: ([]) prevActiveTasks: ([]) prevAssignedTasks: ([0_0]) capacity: 1.0 cost: 0.0]} with number of replicas 0
14:28:56.945 [StreamThread-3] INFO o.a.k.s.p.i.StreamPartitionAssignor - stream-thread [StreamThread-3] Assigned tasks to clients as {4694886d-66a8-4f8b-9c31-0447e77b2e76=[activeTasks: ([]) assignedTasks: ([]) prevActiveTasks: ([]) prevAssignedTasks: ([0_0]) capacity: 1.0 cost: 0.0], 72622e43-3a62-476d-8232-915f15356b54=[activeTasks: ([]) assignedTasks: ([]) prevActiveTasks: ([]) prevAssignedTasks: ([0_0]) capacity: 1.0 cost: 0.0], 1af7fd3f-aaef-4b8a-ac1b-904af3cfe4bb=[activeTasks: ([]) assignedTasks: ([]) prevActiveTasks: ([]) prevAssignedTasks: ([0_0]) capacity: 1.0 cost: 0.0]}.
14:28:56.946 [StreamThread-3] WARN o.a.k.c.c.i.ConsumerCoordinator - The following subscribed topics are not assigned to any members in the group map-function-scala-example : [test-name]
14:28:56.946 [StreamThread-3] DEBUG o.a.k.c.c.i.ConsumerCoordinator - Finished assignment for group map-function-scala-example: {map-function-scala-example-4694886d-66a8-4f8b-9c31-0447e77b2e76-StreamThread-4-consumer-9b2b7efc-e084-4b77-a942-689efd1ab1f4=Assignment(partitions=[]), map-function-scala-example-1af7fd3f-aaef-4b8a-ac1b-904af3cfe4bb-StreamThread-3-consumer-80aa1f3d-370f-4d3e-8353-e8d403f9ad3c=Assignment(partitions=[]), map-function-scala-example-72622e43-3a62-476d-8232-915f15356b54-StreamThread-2-consumer-06157563-8f80-49ed-b29f-73f877782dc6=Assignment(partitions=[])}
14:28:56.946 [StreamThread-3] DEBUG o.a.k.c.c.i.AbstractCoordinator - Sending leader SyncGroup for group map-function-scala-example to coordinator 10.20.69.86:9092 (id: 2147483647 rack: null): (type=SyncGroupRequest, groupId=map-function-scala-example, generationId=32, memberId=map-function-scala-example-1af7fd3f-aaef-4b8a-ac1b-904af3cfe4bb-StreamThread-3-consumer-80aa1f3d-370f-4d3e-8353-e8d403f9ad3c, groupAssignment=map-function-scala-example-4694886d-66a8-4f8b-9c31-0447e77b2e76-StreamThread-4-consumer-9b2b7efc-e084-4b77-a942-689efd1ab1f4,map-function-scala-example-1af7fd3f-aaef-4b8a-ac1b-904af3cfe4bb-StreamThread-3-consumer-80aa1f3d-370f-4d3e-8353-e8d403f9ad3c,map-function-scala-example-72622e43-3a62-476d-8232-915f15356b54-StreamThread-2-consumer-06157563-8f80-49ed-b29f-73f877782dc6)
14:28:56.949 [StreamThread-3] INFO o.a.k.c.c.i.AbstractCoordinator - Successfully joined group map-function-scala-example with generation 32
14:28:56.949 [StreamThread-3] INFO o.a.k.c.c.i.ConsumerCoordinator - Setting newly assigned partitions [] for group map-function-scala-example
14:28:56.949 [StreamThread-3] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-3] at state PARTITIONS_REVOKED: new partitions [] assigned at the end of consumer rebalance.
14:28:56.949 [StreamThread-3] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-3] State transition from PARTITIONS_REVOKED to ASSIGNING_PARTITIONS.
14:28:56.949 [StreamThread-3] INFO o.apache.kafka.streams.KafkaStreams - stream-client [map-function-scala-example-1af7fd3f-aaef-4b8a-ac1b-904af3cfe4bb] State transition from REBALANCING to REBALANCING.
14:28:56.949 [StreamThread-3] DEBUG o.a.k.clients.consumer.KafkaConsumer - Unsubscribed all topics or patterns and assigned partitions
14:28:56.949 [StreamThread-3] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-3] State transition from ASSIGNING_PARTITIONS to RUNNING.
14:28:56.950 [StreamThread-3] INFO o.apache.kafka.streams.KafkaStreams - stream-client [map-function-scala-example-1af7fd3f-aaef-4b8a-ac1b-904af3cfe4bb] State transition from REBALANCING to RUNNING.
14:28:56.950 [StreamThread-2] INFO o.a.k.c.c.i.AbstractCoordinator - Successfully joined group map-function-scala-example with generation 32
14:28:56.950 [StreamThread-2] INFO o.a.k.c.c.i.ConsumerCoordinator - Setting newly assigned partitions [] for group map-function-scala-example
14:28:56.950 [StreamThread-2] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-2] at state PARTITIONS_REVOKED: new partitions [] assigned at the end of consumer rebalance.
14:28:56.950 [StreamThread-2] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-2] State transition from PARTITIONS_REVOKED to ASSIGNING_PARTITIONS.
14:28:56.950 [StreamThread-2] INFO o.apache.kafka.streams.KafkaStreams - stream-client [map-function-scala-example-72622e43-3a62-476d-8232-915f15356b54] State transition from REBALANCING to REBALANCING.
14:28:56.951 [StreamThread-2] DEBUG o.a.k.clients.consumer.KafkaConsumer - Unsubscribed all topics or patterns and assigned partitions
14:28:56.951 [StreamThread-2] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-2] State transition from ASSIGNING_PARTITIONS to RUNNING.
14:28:56.951 [StreamThread-2] INFO o.apache.kafka.streams.KafkaStreams - stream-client [map-function-scala-example-72622e43-3a62-476d-8232-915f15356b54] State transition from REBALANCING to RUNNING.
14:28:56.952 [StreamThread-4] INFO o.a.k.c.c.i.AbstractCoordinator - Successfully joined group map-function-scala-example with generation 32
14:28:56.952 [StreamThread-4] INFO o.a.k.c.c.i.ConsumerCoordinator - Setting newly assigned partitions [] for group map-function-scala-example
14:28:56.952 [StreamThread-4] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-4] at state PARTITIONS_REVOKED: new partitions [] assigned at the end of consumer rebalance.
14:28:56.952 [StreamThread-4] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-4] State transition from PARTITIONS_REVOKED to ASSIGNING_PARTITIONS.
14:28:56.952 [StreamThread-4] INFO o.apache.kafka.streams.KafkaStreams - stream-client [map-function-scala-example-4694886d-66a8-4f8b-9c31-0447e77b2e76] State transition from REBALANCING to REBALANCING.
14:28:56.953 [StreamThread-4] DEBUG o.a.k.clients.consumer.KafkaConsumer - Unsubscribed all topics or patterns and assigned partitions
14:28:56.953 [StreamThread-4] INFO o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-4] State transition from ASSIGNING_PARTITIONS to RUNNING.
14:28:56.953 [StreamThread-4] INFO o.apache.kafka.streams.KafkaStreams - stream-client [map-function-scala-example-4694886d-66a8-4f8b-9c31-0447e77b2e76] State transition from REBALANCING to RUNNING.
14:28:59.950 [kafka-coordinator-heartbeat-thread | map-function-scala-example] DEBUG o.a.k.c.c.i.AbstractCoordinator - Sending Heartbeat request for group map-function-scala-example to coordinator 10.20.69.86:9092 (id: 2147483647 rack: null)
14:28:59.959 [StreamThread-3] DEBUG o.a.k.c.c.i.AbstractCoordinator - Received successful Heartbeat response for group map-function-scala-example
14:29:00.047 [kafka-coordinator-heartbeat-thread | map-function-scala-example] DEBUG o.a.k.c.c.i.AbstractCoordinator - Sending Heartbeat request for group map-function-scala-example to coordinator 10.20.69.86:9092 (id: 2147483647 rack: null)
14:29:00.048 [kafka-coordinator-heartbeat-thread | map-function-scala-example] DEBUG o.a.k.c.c.i.AbstractCoordinator - Sending Heartbeat request for group map-function-scala-example to coordinator 10.20.69.86:9092 (id: 2147483647 rack: null)
14:29:00.050 [StreamThread-4] DEBUG o.a.k.c.c.i.AbstractCoordinator - Received successful Heartbeat response for group map-function-scala-example
14:29:00.050 [StreamThread-2] DEBUG o.a.k.c.c.i.AbstractCoordinator - Received successful Heartbeat response for group map-function-scala-example
14:29:03.035 [kafka-coordinator-heartbeat-thread | map-function-scala-example] DEBUG o.a.k.c.c.i.AbstractCoordinator - Sending Heartbeat request for group map-function-scala-example to coordinator 10.20.69.86:9092 (id: 2147483647 rack: null)
14:29:03.038 [StreamThread-3] DEBUG o.a.k.c.c.i.AbstractCoordinator - Received successful Heartbeat response for group map-function-scala-example
14:29:03.128 [kafka-coordinator-heartbeat-thread | map-function-scala-example] DEBUG o.a.k.c.c.i.AbstractCoordinator - Sending Heartbeat request for group map-function-scala-example to coordinator 10.20.69.86:9092 (id: 2147483647 rack: null)
14:29:03.130 [StreamThread-4] DEBUG o.a.k.c.c.i.AbstractCoordinator - Received successful Heartbeat response for group map-function-scala-example
14:29:03.135 [kafka-coordinator-heartbeat-thread | map-function-scala-example] DEBUG o.a.k.c.c.i.AbstractCoordinator - Sending Heartbeat request for group map-function-scala-example to coordinator 10.20.69.86:9092 (id: 2147483647 rack: null)
14:29:03.137 [StreamThread-2] DEBUG o.a.k.c.c.i.AbstractCoordinator - Received successful Heartbeat response for group map-function-scala-example
14:29:06.134 [kafka-coordinator-heartbeat-thread | map-function-scala-example] DEBUG o.a.k.c.c.i.AbstractCoordinator - Sending Heartbeat request for group map-function-scala-example to coordinator 10.20.69.86:9092 (id: 2147483647 rack: null)
14:29:06.135 [kafka-coordinator-heartbeat-thread | map-function-scala-example] DEBUG o.a.k.c.c.i.AbstractCoordinator - Sending Heartbeat request for group map-function-scala-example to coordinator 10.20.69.86:9092 (id: 2147483647 rack: null)
14:29:06.139 [StreamThread-2] DEBUG o.a.k.c.c.i.AbstractCoordinator - Received successful Heartbeat response for group map-function-scala-example
14:29:06.140 [StreamThread-3] DEBUG o.a.k.c.c.i.AbstractCoordinator - Received successful Heartbeat response for group map-function-scala-example
14:29:06.207 [kafka-coordinator-heartbeat-thread | map-function-scala-example] DEBUG o.a.k.c.c.i.AbstractCoordinator - Sending Heartbeat request for group map-function-scala-example to coordinator 10.20.69.86:9092 (id: 2147483647 rack: null)
14:29:06.210 [StreamThread-4] DEBUG o.a.k.c.c.i.AbstractCoordinator - Received successful Heartbeat response for group map-function-scala-example
14:29:09.137 [kafka-coordinator-heartbeat-thread | map-function-scala-example] DEBUG o.a.k.c.c.i.AbstractCoordinator - Sending Heartbeat request for group map-function-scala-example to coordinator 10.20.69.86:9092 (id: 2147483647 rack: null)
14:29:09.138 [kafka-coordinator-heartbeat-thread | map-function-scala-example] DEBUG o.a.k.c.c.i.AbstractCoordinator - Sending Heartbeat request for group map-function-scala-example to coordinator 10.20.69.86:9092 (id: 2147483647 rack: null)
14:29:09.142 [StreamThread-3] DEBUG o.a.k.c.c.i.AbstractCoordinator - Received successful Heartbeat response for group map-function-scala-example
14:29:09.142 [StreamThread-2] DEBUG o.a.k.c.c.i.AbstractCoordinator - Received successful Heartbeat response for group map-function-scala-example
14:29:09.294 [kafka-coordinator-heartbeat-thread | map-function-scala-example] DEBUG o.a.k.c.c.i.AbstractCoordinator - Sending Heartbeat request for group map-function-scala-example to coordinator 10.20.69.86:9092 (id: 2147483647 rack: null)
14:29:09.295 [StreamThread-4] DEBUG o.a.k.c.c.i.AbstractCoordinator - Received successful Heartbeat response for group map-function-scala-example
14:29:12.191 [kafka-coordinator-heartbeat-thread | map-function-scala-example] DEBUG o.a.k.c.c.i.AbstractCoordinator - Sending Heartbeat request for group map-function-scala-example to coordinator 10.20.69.86:9092 (id: 2147483647 rack: null)
14:29:12.193 [StreamThread-3] DEBUG o.a.k.c.c.i.AbstractCoordinator - Received successful Heartbeat response for group map-function-scala-example
14:29:12.213 [kafka-coordinator-heartbeat-thread | map-function-scala-example] DEBUG o.a.k.c.c.i.AbstractCoordinator - Sending Heartbeat request for group map-function-scala-example to coordinator 10.20.69.86:9092 (id: 2147483647 rack: null)
14:29:12.214 [StreamThread-2] DEBUG o.a.k.c.c.i.AbstractCoordinator - Received successful Heartbeat response for group map-function-scala-example
14:29:12.398 [kafka-coordinator-heartbeat-thread | map-function-scala-example] DEBUG o.a.k.c.c.i.AbstractCoordinator - Sending Heartbeat request for group map-function-scala-example to coordinator 10.20.69.86:9092 (id: 2147483647 rack: null)
14:29:12.399 [StreamThread-4] DEBUG o.a.k.c.c.i.AbstractCoordinator - Received successful Heartbeat response for group map-function-scala-example
Raw
kafka server logs
[2017-07-17 14:28:23,799] INFO KafkaConfig values:
advertised.host.name = null
advertised.listeners = null
advertised.port = null
authorizer.class.name =
auto.create.topics.enable = true
auto.leader.rebalance.enable = true
background.threads = 10
broker.id = 0
broker.id.generation.enable = true
broker.rack = null
compression.type = producer
connections.max.idle.ms = 600000
controlled.shutdown.enable = true
controlled.shutdown.max.retries = 3
controlled.shutdown.retry.backoff.ms = 5000
controller.socket.timeout.ms = 30000
create.topic.policy.class.name = null
default.replication.factor = 1
delete.topic.enable = false
fetch.purgatory.purge.interval.requests = 1000
group.max.session.timeout.ms = 300000
group.min.session.timeout.ms = 6000
host.name =
inter.broker.listener.name = null
inter.broker.protocol.version = 0.10.2-IV0
leader.imbalance.check.interval.seconds = 300
leader.imbalance.per.broker.percentage = 10
listener.security.protocol.map = SSL:SSL,SASL_PLAINTEXT:SASL_PLAINTEXT,TRACE:TRACE,SASL_SSL:SASL_SSL,PLAINTEXT:PLAINTEXT
listeners = null
log.cleaner.backoff.ms = 15000
log.cleaner.dedupe.buffer.size = 134217728
log.cleaner.delete.retention.ms = 86400000
log.cleaner.enable = true
log.cleaner.io.buffer.load.factor = 0.9
log.cleaner.io.buffer.size = 524288
log.cleaner.io.max.bytes.per.second = 1.7976931348623157E308
log.cleaner.min.cleanable.ratio = 0.5
log.cleaner.min.compaction.lag.ms = 0
log.cleaner.threads = 1
log.cleanup.policy = [delete]
log.dir = /tmp/kafka-logs
log.dirs = /tmp/kafka-logs
log.flush.interval.messages = 9223372036854775807
log.flush.interval.ms = null
log.flush.offset.checkpoint.interval.ms = 60000
log.flush.scheduler.interval.ms = 9223372036854775807
log.index.interval.bytes = 4096
log.index.size.max.bytes = 10485760
log.message.format.version = 0.10.2-IV0
log.message.timestamp.difference.max.ms = 9223372036854775807
log.message.timestamp.type = CreateTime
log.preallocate = false
log.retention.bytes = -1
log.retention.check.interval.ms = 300000
log.retention.hours = 168
log.retention.minutes = null
log.retention.ms = null
log.roll.hours = 168
log.roll.jitter.hours = 0
log.roll.jitter.ms = null
log.roll.ms = null
log.segment.bytes = 1073741824
log.segment.delete.delay.ms = 60000
max.connections.per.ip = 2147483647
max.connections.per.ip.overrides =
message.max.bytes = 1000012
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
min.insync.replicas = 1
num.io.threads = 8
num.network.threads = 3
num.partitions = 1
num.recovery.threads.per.data.dir = 1
num.replica.fetchers = 1
offset.metadata.max.bytes = 4096
offsets.commit.required.acks = -1
offsets.commit.timeout.ms = 5000
offsets.load.buffer.size = 5242880
offsets.retention.check.interval.ms = 600000
offsets.retention.minutes = 1440
offsets.topic.compression.codec = 0
offsets.topic.num.partitions = 50
offsets.topic.replication.factor = 3
offsets.topic.segment.bytes = 104857600
port = 9092
principal.builder.class = class org.apache.kafka.common.security.auth.DefaultPrincipalBuilder
producer.purgatory.purge.interval.requests = 1000
queued.max.requests = 500
quota.consumer.default = 9223372036854775807
quota.producer.default = 9223372036854775807
quota.window.num = 11
quota.window.size.seconds = 1
replica.fetch.backoff.ms = 1000
replica.fetch.max.bytes = 1048576
replica.fetch.min.bytes = 1
replica.fetch.response.max.bytes = 10485760
replica.fetch.wait.max.ms = 500
replica.high.watermark.checkpoint.interval.ms = 5000
replica.lag.time.max.ms = 10000
replica.socket.receive.buffer.bytes = 65536
replica.socket.timeout.ms = 30000
replication.quota.window.num = 11
replication.quota.window.size.seconds = 1
request.timeout.ms = 30000
reserved.broker.max.id = 1000
sasl.enabled.mechanisms = [GSSAPI]
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.principal.to.local.rules = [DEFAULT]
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.mechanism.inter.broker.protocol = GSSAPI
security.inter.broker.protocol = PLAINTEXT
socket.receive.buffer.bytes = 102400
socket.request.max.bytes = 104857600
socket.send.buffer.bytes = 102400
ssl.cipher.suites = null
ssl.client.auth = none
ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
ssl.endpoint.identification.algorithm = null
ssl.key.password = null
ssl.keymanager.algorithm = SunX509
ssl.keystore.location = null
ssl.keystore.password = null
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.location = null
ssl.truststore.password = null
ssl.truststore.type = JKS
unclean.leader.election.enable = true
zookeeper.connect = localhost:2181
zookeeper.connection.timeout.ms = 6000
zookeeper.session.timeout.ms = 6000
zookeeper.set.acl = false
zookeeper.sync.time.ms = 2000
(kafka.server.KafkaConfig)
[2017-07-17 14:28:23,840] INFO starting (kafka.server.KafkaServer)
[2017-07-17 14:28:23,842] INFO Connecting to zookeeper on localhost:2181 (kafka.server.KafkaServer)
[2017-07-17 14:28:23,854] INFO Starting ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread)
[2017-07-17 14:28:23,861] INFO Client environment:zookeeper.version=3.4.9-1757313, built on 08/23/2016 06:50 GMT (org.apache.zookeeper.ZooKeeper)
[2017-07-17 14:28:23,861] INFO Client environment:host.name=10.20.69.86 (org.apache.zookeeper.ZooKeeper)
[2017-07-17 14:28:23,861] INFO Client environment:java.version=1.8.0_45 (org.apache.zookeeper.ZooKeeper)
[2017-07-17 14:28:23,861] INFO Client environment:java.vendor=Oracle Corporation (org.apache.zookeeper.ZooKeeper)
[2017-07-17 14:28:23,861] INFO Client environment:java.home=/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre (org.apache.zookeeper.ZooKeeper)
[2017-07-17 14:28:23,861] INFO Client environment:java.class.path=:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/aopalliance-repackaged-2.5.0-b05.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/argparse4j-0.7.0.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/connect-api-0.10.2.1.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/connect-file-0.10.2.1.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/connect-json-0.10.2.1.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/connect-runtime-0.10.2.1.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/connect-transforms-0.10.2.1.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/guava-18.0.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/hk2-api-2.5.0-b05.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/hk2-locator-2.5.0-b05.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/hk2-utils-2.5.0-b05.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/jackson-annotations-2.8.0.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/jackson-annotations-2.8.5.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/jackson-core-2.8.5.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/jackson-databind-2.8.5.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/jackson-jaxrs-base-2.8.5.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/jackson-jaxrs-json-provider-2.8.5.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/jackson-module-jaxb-annotations-2.8.5.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/javassist-3.20.0-GA.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/javax.annotation-api-1.2.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/javax.inject-1.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/javax.inject-2.5.0-b05.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/javax.servlet-api-3.1.0.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/javax.ws.rs-api-2.0.1.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/jersey-client-2.24.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/jersey-common-2.24.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/jersey-container-servlet-2.24.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/jersey-container-servlet-core-2.24.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/jersey-guava-2.24.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/jersey-media-jaxb-2.24.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/jersey-server-2.24.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/jetty-continuation-9.2.15.v20160210.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/jetty-http-9.2.15.v20160210.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/jetty-io-9.2.15.v20160210.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/jetty-security-9.2.15.v20160210.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/jetty-server-9.2.15.v20160210.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/jetty-servlet-9.2.15.v20160210.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/jetty-servlets-9.2.15.v20160210.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/jetty-util-9.2.15.v20160210.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/jopt-simple-5.0.3.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/kafka-clients-0.10.2.1.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/kafka-log4j-appender-0.10.2.1.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/kafka-streams-0.10.2.1.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/kafka-streams-examples-0.10.2.1.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/kafka-tools-0.10.2.1.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/kafka_2.11-0.10.2.1-sources.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/kafka_2.11-0.10.2.1-test-sources.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/kafka_2.11-0.10.2.1.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/log4j-1.2.17.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/lz4-1.3.0.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/metrics-core-2.2.0.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/osgi-resource-locator-1.0.1.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/reflections-0.9.10.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/rocksdbjni-5.0.1.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/scala-library-2.11.8.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/scala-parser-combinators_2.11-1.0.4.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/slf4j-api-1.7.21.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/slf4j-log4j12-1.7.21.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/snappy-java-1.1.2.6.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/validation-api-1.1.0.Final.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/zkclient-0.10.jar:/Users/cchen/Downloads/kafka_2.11-0.10.2.1/bin/../libs/zookeeper-3.4.9.jar (org.apache.zookeeper.ZooKeeper)
[2017-07-17 14:28:23,861] INFO Client environment:java.library.path=/Users/cchen/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:. (org.apache.zookeeper.ZooKeeper)
[2017-07-17 14:28:23,861] INFO Client environment:java.io.tmpdir=/var/folders/bp/xb967qvs0nzf3qgj7ffjzpyxfvnd4r/T/ (org.apache.zookeeper.ZooKeeper)
[2017-07-17 14:28:23,861] INFO Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper)
[2017-07-17 14:28:23,861] INFO Client environment:os.name=Mac OS X (org.apache.zookeeper.ZooKeeper)
[2017-07-17 14:28:23,861] INFO Client environment:os.arch=x86_64 (org.apache.zookeeper.ZooKeeper)
[2017-07-17 14:28:23,861] INFO Client environment:os.version=10.11.6 (org.apache.zookeeper.ZooKeeper)
[2017-07-17 14:28:23,861] INFO Client environment:user.name=cchen (org.apache.zookeeper.ZooKeeper)
[2017-07-17 14:28:23,861] INFO Client environment:user.home=/Users/cchen (org.apache.zookeeper.ZooKeeper)
[2017-07-17 14:28:23,861] INFO Client environment:user.dir=/Users/cchen/Downloads/kafka_2.11-0.10.2.1 (org.apache.zookeeper.ZooKeeper)
[2017-07-17 14:28:23,862] INFO Initiating client connection, connectString=localhost:2181 sessionTimeout=6000 watcher=org.I0Itec.zkclient.ZkClient@13e39c73 (org.apache.zookeeper.ZooKeeper)
[2017-07-17 14:28:23,875] INFO Waiting for keeper state SyncConnected (org.I0Itec.zkclient.ZkClient)
[2017-07-17 14:28:23,878] INFO Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2017-07-17 14:28:23,953] INFO Socket connection established to localhost/0:0:0:0:0:0:0:1:2181, initiating session (org.apache.zookeeper.ClientCnxn)
[2017-07-17 14:28:23,964] INFO Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x15d4327c712010f, negotiated timeout = 6000 (org.apache.zookeeper.ClientCnxn)
[2017-07-17 14:28:23,966] INFO zookeeper state changed (SyncConnected) (org.I0Itec.zkclient.ZkClient)
[2017-07-17 14:28:24,097] INFO Cluster ID = _Epo8QiSSuGbI1zi8iEXLA (kafka.server.KafkaServer)
[2017-07-17 14:28:24,114] INFO [ThrottledRequestReaper-Fetch], Starting (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
[2017-07-17 14:28:24,116] INFO [ThrottledRequestReaper-Produce], Starting (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
[2017-07-17 14:28:24,174] INFO Loading logs. (kafka.log.LogManager)
[2017-07-17 14:28:24,248] INFO Completed load of log __consumer_offsets-0 with 1 log segments and log end offset 4 in 58 ms (kafka.log.Log)
[2017-07-17 14:28:24,254] INFO Completed load of log __consumer_offsets-1 with 1 log segments and log end offset 0 in 1 ms (kafka.log.Log)
[2017-07-17 14:28:24,259] INFO Completed load of log __consumer_offsets-10 with 1 log segments and log end offset 0 in 2 ms (kafka.log.Log)
[2017-07-17 14:28:24,264] INFO Completed load of log __consumer_offsets-11 with 1 log segments and log end offset 0 in 2 ms (kafka.log.Log)
[2017-07-17 14:28:24,268] INFO Completed load of log __consumer_offsets-12 with 1 log segments and log end offset 0 in 2 ms (kafka.log.Log)
[2017-07-17 14:28:24,273] INFO Completed load of log __consumer_offsets-13 with 1 log segments and log end offset 0 in 2 ms (kafka.log.Log)
[2017-07-17 14:28:24,277] INFO Completed load of log __consumer_offsets-14 with 1 log segments and log end offset 0 in 1 ms (kafka.log.Log)
[2017-07-17 14:28:24,281] INFO Completed load of log __consumer_offsets-15 with 1 log segments and log end offset 0 in 1 ms (kafka.log.Log)
[2017-07-17 14:28:24,285] INFO Completed load of log __consumer_offsets-16 with 1 log segments and log end offset 0 in 2 ms (kafka.log.Log)
[2017-07-17 14:28:24,289] INFO Completed load of log __consumer_offsets-17 with 1 log segments and log end offset 0 in 1 ms (kafka.log.Log)
[2017-07-17 14:28:24,292] INFO Completed load of log __consumer_offsets-18 with 1 log segments and log end offset 0 in 2 ms (kafka.log.Log)
[2017-07-17 14:28:24,296] INFO Completed load of log __consumer_offsets-19 with 1 log segments and log end offset 0 in 2 ms (kafka.log.Log)
[2017-07-17 14:28:24,299] INFO Completed load of log __consumer_offsets-2 with 1 log segments and log end offset 0 in 1 ms (kafka.log.Log)
[2017-07-17 14:28:24,301] INFO Completed load of log __consumer_offsets-20 with 1 log segments and log end offset 0 in 1 ms (kafka.log.Log)
[2017-07-17 14:28:24,305] INFO Completed load of log __consumer_offsets-21 with 1 log segments and log end offset 3 in 2 ms (kafka.log.Log)
[2017-07-17 14:28:24,310] INFO Completed load of log __consumer_offsets-22 with 1 log segments and log end offset 7 in 1 ms (kafka.log.Log)
[2017-07-17 14:28:24,314] INFO Completed load of log __consumer_offsets-23 with 1 log segments and log end offset 3 in 2 ms (kafka.log.Log)
[2017-07-17 14:28:24,316] INFO Completed load of log __consumer_offsets-24 with 1 log segments and log end offset 0 in 1 ms (kafka.log.Log)
[2017-07-17 14:28:24,320] INFO Completed load of log __consumer_offsets-25 with 1 log segments and log end offset 4 in 2 ms (kafka.log.Log)
[2017-07-17 14:28:24,324] INFO Completed load of log __consumer_offsets-26 with 1 log segments and log end offset 0 in 1 ms (kafka.log.Log)
[2017-07-17 14:28:24,327] INFO Completed load of log __consumer_offsets-27 with 1 log segments and log end offset 0 in 2 ms (kafka.log.Log)
[2017-07-17 14:28:24,330] INFO Completed load of log __consumer_offsets-28 with 1 log segments and log end offset 4 in 2 ms (kafka.log.Log)
[2017-07-17 14:28:24,334] INFO Completed load of log __consumer_offsets-29 with 1 log segments and log end offset 0 in 1 ms (kafka.log.Log)
[2017-07-17 14:28:24,337] INFO Completed load of log __consumer_offsets-3 with 1 log segments and log end offset 0 in 2 ms (kafka.log.Log)
[2017-07-17 14:28:24,341] INFO Completed load of log __consumer_offsets-30 with 1 log segments and log end offset 0 in 2 ms (kafka.log.Log)
[2017-07-17 14:28:24,346] INFO Completed load of log __consumer_offsets-31 with 1 log segments and log end offset 0 in 3 ms (kafka.log.Log)
[2017-07-17 14:28:24,350] INFO Completed load of log __consumer_offsets-32 with 1 log segments and log end offset 15 in 2 ms (kafka.log.Log)
[2017-07-17 14:28:24,353] INFO Completed load of log __consumer_offsets-33 with 1 log segments and log end offset 0 in 1 ms (kafka.log.Log)
[2017-07-17 14:28:24,360] INFO Completed load of log __consumer_offsets-34 with 1 log segments and log end offset 0 in 4 ms (kafka.log.Log)
[2017-07-17 14:28:24,363] INFO Completed load of log __consumer_offsets-35 with 1 log segments and log end offset 0 in 1 ms (kafka.log.Log)
[2017-07-17 14:28:24,367] INFO Completed load of log __consumer_offsets-36 with 1 log segments and log end offset 0 in 2 ms (kafka.log.Log)
[2017-07-17 14:28:24,371] INFO Completed load of log __consumer_offsets-37 with 1 log segments and log end offset 4 in 2 ms (kafka.log.Log)
[2017-07-17 14:28:24,375] INFO Completed load of log __consumer_offsets-38 with 1 log segments and log end offset 0 in 1 ms (kafka.log.Log)
[2017-07-17 14:28:24,378] INFO Completed load of log __consumer_offsets-39 with 1 log segments and log end offset 0 in 2 ms (kafka.log.Log)
[2017-07-17 14:28:24,384] INFO Completed load of log __consumer_offsets-4 with 1 log segments and log end offset 0 in 4 ms (kafka.log.Log)
[2017-07-17 14:28:24,388] INFO Completed load of log __consumer_offsets-40 with 1 log segments and log end offset 0 in 2 ms (kafka.log.Log)
[2017-07-17 14:28:24,391] INFO Completed load of log __consumer_offsets-41 with 1 log segments and log end offset 0 in 2 ms (kafka.log.Log)
[2017-07-17 14:28:24,394] INFO Completed load of log __consumer_offsets-42 with 1 log segments and log end offset 0 in 2 ms (kafka.log.Log)
[2017-07-17 14:28:24,397] INFO Completed load of log __consumer_offsets-43 with 1 log segments and log end offset 121 in 2 ms (kafka.log.Log)
[2017-07-17 14:28:24,400] INFO Completed load of log __consumer_offsets-44 with 1 log segments and log end offset 0 in 1 ms (kafka.log.Log)
[2017-07-17 14:28:24,404] INFO Completed load of log __consumer_offsets-45 with 1 log segments and log end offset 0 in 2 ms (kafka.log.Log)
[2017-07-17 14:28:24,407] INFO Completed load of log __consumer_offsets-46 with 1 log segments and log end offset 0 in 2 ms (kafka.log.Log)
[2017-07-17 14:28:24,410] INFO Completed load of log __consumer_offsets-47 with 1 log segments and log end offset 0 in 1 ms (kafka.log.Log)
[2017-07-17 14:28:24,416] INFO Completed load of log __consumer_offsets-48 with 1 log segments and log end offset 0 in 4 ms (kafka.log.Log)
[2017-07-17 14:28:24,429] INFO Completed load of log __consumer_offsets-49 with 1 log segments and log end offset 0 in 11 ms (kafka.log.Log)
[2017-07-17 14:28:24,432] INFO Completed load of log __consumer_offsets-5 with 1 log segments and log end offset 0 in 1 ms (kafka.log.Log)
[2017-07-17 14:28:24,436] INFO Completed load of log __consumer_offsets-6 with 1 log segments and log end offset 0 in 2 ms (kafka.log.Log)
[2017-07-17 14:28:24,439] INFO Completed load of log __consumer_offsets-7 with 1 log segments and log end offset 0 in 1 ms (kafka.log.Log)
[2017-07-17 14:28:24,441] INFO Completed load of log __consumer_offsets-8 with 1 log segments and log end offset 0 in 1 ms (kafka.log.Log)
[2017-07-17 14:28:24,444] INFO Completed load of log __consumer_offsets-9 with 1 log segments and log end offset 4 in 1 ms (kafka.log.Log)
[2017-07-17 14:28:24,469] INFO Completed load of log json-config-0 with 1 log segments and log end offset 367 in 24 ms (kafka.log.Log)
[2017-07-17 14:28:24,473] INFO Completed load of log OriginalAndUppercasedTopic-0 with 1 log segments and log end offset 0 in 2 ms (kafka.log.Log)
[2017-07-17 14:28:24,481] INFO Completed load of log OriginalAndUppercasedTopic2-0 with 1 log segments and log end offset 365 in 7 ms (kafka.log.Log)
[2017-07-17 14:28:24,496] INFO Completed load of log StreamConfig2-0 with 1 log segments and log end offset 2 in 14 ms (kafka.log.Log)
[2017-07-17 14:28:24,519] INFO Completed load of log test-name-0 with 1 log segments and log end offset 57 in 22 ms (kafka.log.Log)
[2017-07-17 14:28:24,521] INFO Logs loading complete in 347 ms. (kafka.log.LogManager)
[2017-07-17 14:28:24,607] INFO Starting log cleanup with a period of 300000 ms. (kafka.log.LogManager)
[2017-07-17 14:28:24,608] INFO Starting log flusher with a default period of 9223372036854775807 ms. (kafka.log.LogManager)
[2017-07-17 14:28:24,642] INFO Awaiting socket connections on 0.0.0.0:9092. (kafka.network.Acceptor)
[2017-07-17 14:28:24,644] INFO [Socket Server on Broker 0], Started 1 acceptor threads (kafka.network.SocketServer)
[2017-07-17 14:28:24,657] INFO [ExpirationReaper-0], Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-07-17 14:28:24,657] INFO [ExpirationReaper-0], Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-07-17 14:28:24,688] INFO Creating /controller (is it secure? false) (kafka.utils.ZKCheckedEphemeral)
[2017-07-17 14:28:24,695] INFO Result of znode creation is: OK (kafka.utils.ZKCheckedEphemeral)
[2017-07-17 14:28:24,696] INFO 0 successfully elected as leader (kafka.server.ZookeeperLeaderElector)
[2017-07-17 14:28:25,483] INFO New leader is 0 (kafka.server.ZookeeperLeaderElector$LeaderChangeListener)
[2017-07-17 14:28:25,487] INFO [ExpirationReaper-0], Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-07-17 14:28:25,490] INFO [ExpirationReaper-0], Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-07-17 14:28:25,491] INFO [ExpirationReaper-0], Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-07-17 14:28:25,533] INFO [GroupCoordinator 0]: Starting up. (kafka.coordinator.GroupCoordinator)
[2017-07-17 14:28:25,534] INFO [GroupCoordinator 0]: Startup complete. (kafka.coordinator.GroupCoordinator)
[2017-07-17 14:28:25,535] INFO [Group Metadata Manager on Broker 0]: Removed 0 expired offsets in 1 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:25,560] INFO Will not load MX4J, mx4j-tools.jar is not in the classpath (kafka.utils.Mx4jLoader$)
[2017-07-17 14:28:25,896] INFO Creating /brokers/ids/0 (is it secure? false) (kafka.utils.ZKCheckedEphemeral)
[2017-07-17 14:28:25,902] INFO Result of znode creation is: OK (kafka.utils.ZKCheckedEphemeral)
[2017-07-17 14:28:25,905] INFO Registered broker 0 at path /brokers/ids/0 with addresses: EndPoint(10.20.69.86,9092,ListenerName(PLAINTEXT),PLAINTEXT) (kafka.utils.ZkUtils)
[2017-07-17 14:28:25,915] INFO Kafka version : 0.10.2.1 (org.apache.kafka.common.utils.AppInfoParser)
[2017-07-17 14:28:25,915] INFO Kafka commitId : e89bffd6b2eff799 (org.apache.kafka.common.utils.AppInfoParser)
[2017-07-17 14:28:25,916] INFO [Kafka Server 0], started (kafka.server.KafkaServer)
[2017-07-17 14:28:26,120] INFO [ReplicaFetcherManager on broker 0] Removed fetcher for partitions __consumer_offsets-22,__consumer_offsets-30,__consumer_offsets-8,__consumer_offsets-21,__consumer_offsets-4,__consumer_offsets-27,__consumer_offsets-7,__consumer_offsets-9,__consumer_offsets-46,OriginalAndUppercasedTopic-0,StreamConfig2-0,__consumer_offsets-25,__consumer_offsets-35,__consumer_offsets-41,__consumer_offsets-33,__consumer_offsets-23,__consumer_offsets-49,__consumer_offsets-47,__consumer_offsets-16,__consumer_offsets-28,test-name-0,__consumer_offsets-31,__consumer_offsets-36,__consumer_offsets-42,__consumer_offsets-3,__consumer_offsets-18,__consumer_offsets-37,__consumer_offsets-15,__consumer_offsets-24,__consumer_offsets-38,__consumer_offsets-17,__consumer_offsets-48,__consumer_offsets-19,__consumer_offsets-11,__consumer_offsets-13,__consumer_offsets-2,__consumer_offsets-43,__consumer_offsets-6,__consumer_offsets-14,__consumer_offsets-20,__consumer_offsets-0,__consumer_offsets-44,__consumer_offsets-39,__consumer_offsets-12,__consumer_offsets-45,__consumer_offsets-1,__consumer_offsets-5,__consumer_offsets-26,__consumer_offsets-29,OriginalAndUppercasedTopic2-0,__consumer_offsets-34,__consumer_offsets-10,__consumer_offsets-32,__consumer_offsets-40,json-config-0 (kafka.server.ReplicaFetcherManager)
[2017-07-17 14:28:26,373] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-22 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,413] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-22 in 39 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,414] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-25 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,421] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-25 in 6 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,421] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-28 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,430] INFO [ReplicaFetcherManager on broker 0] Removed fetcher for partitions __consumer_offsets-22,__consumer_offsets-30,__consumer_offsets-8,__consumer_offsets-21,__consumer_offsets-4,__consumer_offsets-27,__consumer_offsets-7,__consumer_offsets-9,__consumer_offsets-46,OriginalAndUppercasedTopic-0,StreamConfig2-0,__consumer_offsets-25,__consumer_offsets-35,__consumer_offsets-41,__consumer_offsets-33,__consumer_offsets-23,__consumer_offsets-49,__consumer_offsets-47,__consumer_offsets-16,__consumer_offsets-28,test-name-0,__consumer_offsets-31,__consumer_offsets-36,__consumer_offsets-42,__consumer_offsets-3,__consumer_offsets-18,__consumer_offsets-37,__consumer_offsets-15,__consumer_offsets-24,__consumer_offsets-38,__consumer_offsets-17,__consumer_offsets-48,__consumer_offsets-19,__consumer_offsets-11,__consumer_offsets-13,__consumer_offsets-2,__consumer_offsets-43,__consumer_offsets-6,__consumer_offsets-14,__consumer_offsets-20,__consumer_offsets-0,__consumer_offsets-44,__consumer_offsets-39,__consumer_offsets-12,__consumer_offsets-45,__consumer_offsets-1,__consumer_offsets-5,__consumer_offsets-26,__consumer_offsets-29,OriginalAndUppercasedTopic2-0,__consumer_offsets-34,__consumer_offsets-10,__consumer_offsets-32,__consumer_offsets-40,json-config-0 (kafka.server.ReplicaFetcherManager)
[2017-07-17 14:28:26,447] INFO [GroupCoordinator 0]: Loading group metadata for map-function-scala-example-1500317447828 with generation 3 (kafka.coordinator.GroupCoordinator)
[2017-07-17 14:28:26,448] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-28 in 27 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,448] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-31 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,455] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-31 in 7 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,455] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-34 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,461] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-34 in 5 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,461] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-37 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,467] INFO [GroupCoordinator 0]: Loading group metadata for map-function-scala-example-1500317577177 with generation 3 (kafka.coordinator.GroupCoordinator)
[2017-07-17 14:28:26,467] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-37 in 6 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,467] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-40 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,472] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-40 in 5 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,472] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-43 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,495] INFO [GroupCoordinator 0]: Loading group metadata for map-function-scala-example with generation 30 (kafka.coordinator.GroupCoordinator)
[2017-07-17 14:28:26,496] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-43 in 24 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,496] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-46 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,500] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-46 in 4 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,500] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-49 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,504] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-49 in 4 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,504] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-41 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,508] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-41 in 4 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,508] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-44 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,513] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-44 in 5 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,513] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-47 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,517] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-47 in 4 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,517] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-1 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,521] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-1 in 4 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,522] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-4 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,526] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-4 in 4 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,526] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-7 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,530] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-7 in 4 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,530] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-10 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,535] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-10 in 5 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,535] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-13 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,544] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-13 in 9 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,549] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-16 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,555] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-16 in 6 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,555] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-19 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,560] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-19 in 4 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,561] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-2 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,565] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-2 in 4 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,566] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-5 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,570] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-5 in 4 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,572] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-8 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,579] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-8 in 7 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,579] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-11 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,593] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-11 in 13 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,594] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-14 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,608] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-14 in 13 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,608] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-17 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,626] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-17 in 18 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,626] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-20 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,631] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-20 in 5 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,631] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-23 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,639] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-23 in 8 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,639] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-26 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,648] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-26 in 9 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,649] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-29 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,656] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-29 in 7 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,658] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-32 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,681] INFO [GroupCoordinator 0]: Loading group metadata for map-function-scala-example-testing with generation 11 (kafka.coordinator.GroupCoordinator)
[2017-07-17 14:28:26,681] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-32 in 23 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,682] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-35 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,696] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-35 in 10 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,699] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-38 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,722] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-38 in 23 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,722] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-0 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,745] INFO [GroupCoordinator 0]: Loading group metadata for map-function-scala-example-1500317519231 with generation 3 (kafka.coordinator.GroupCoordinator)
[2017-07-17 14:28:26,745] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-0 in 23 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,745] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-3 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,752] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-3 in 7 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,753] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-6 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,758] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-6 in 5 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,758] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-9 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,766] INFO [GroupCoordinator 0]: Loading group metadata for map-function-scala-example-1500326387352 with generation 3 (kafka.coordinator.GroupCoordinator)
[2017-07-17 14:28:26,766] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-9 in 8 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,766] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-12 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,786] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-12 in 20 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,786] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-15 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,787] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-15 in 1 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,787] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-18 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,790] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-18 in 3 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,790] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-21 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,792] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-21 in 2 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,792] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-24 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,793] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-24 in 1 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,793] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-27 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,794] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-27 in 1 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,794] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-30 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,796] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-30 in 2 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,796] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-33 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,809] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-33 in 13 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,809] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-36 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,811] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-36 in 2 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,811] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-39 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,812] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-39 in 1 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,812] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-42 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,834] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-42 in 22 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,834] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-45 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,835] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-45 in 1 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,835] INFO [Group Metadata Manager on Broker 0]: Loading offsets and group metadata from __consumer_offsets-48 (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:26,837] INFO [Group Metadata Manager on Broker 0]: Finished loading offsets from __consumer_offsets-48 in 2 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2017-07-17 14:28:46,861] INFO [GroupCoordinator 0]: Preparing to restabilize group map-function-scala-example with old generation 30 (kafka.coordinator.GroupCoordinator)
[2017-07-17 14:28:46,867] INFO [GroupCoordinator 0]: Stabilized group map-function-scala-example generation 31 (kafka.coordinator.GroupCoordinator)
[2017-07-17 14:28:46,885] INFO [GroupCoordinator 0]: Assignment received from leader for group map-function-scala-example for generation 31 (kafka.coordinator.GroupCoordinator)
[2017-07-17 14:28:47,063] INFO [GroupCoordinator 0]: Preparing to restabilize group map-function-scala-example with old generation 31 (kafka.coordinator.GroupCoordinator)
[2017-07-17 14:28:56,930] INFO [GroupCoordinator 0]: Stabilized group map-function-scala-example generation 32 (kafka.coordinator.GroupCoordinator)
[2017-07-17 14:28:56,947] INFO [GroupCoordinator 0]: Assignment received from leader for group map-function-scala-example for generation 32 (kafka.coordinator.GroupCoordinator)
[2017-07-17 14:29:18,092] INFO [Kafka Server 0], shutting down (kafka.server.KafkaServer)
[2017-07-17 14:29:18,095] INFO [Kafka Server 0], Starting controlled shutdown (kafka.server.KafkaServer)
[2017-07-17 14:29:18,128] INFO [Kafka Server 0], Controlled shutdown succeeded (kafka.server.KafkaServer)
[2017-07-17 14:29:18,131] INFO [Socket Server on Broker 0], Shutting down (kafka.network.SocketServer)
[2017-07-17 14:29:18,139] INFO [Socket Server on Broker 0], Shutdown completed (kafka.network.SocketServer)
[2017-07-17 14:29:18,139] INFO [Kafka Request Handler on Broker 0], shutting down (kafka.server.KafkaRequestHandlerPool)
[2017-07-17 14:29:18,142] INFO [Kafka Request Handler on Broker 0], shut down completely (kafka.server.KafkaRequestHandlerPool)
[2017-07-17 14:29:18,145] INFO [ThrottledRequestReaper-Fetch], Shutting down (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
[2017-07-17 14:29:18,252] INFO [ThrottledRequestReaper-Fetch], Stopped (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
[2017-07-17 14:29:18,252] INFO [ThrottledRequestReaper-Fetch], Shutdown completed (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
[2017-07-17 14:29:18,252] INFO [ThrottledRequestReaper-Produce], Shutting down (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
[2017-07-17 14:29:19,256] INFO [ThrottledRequestReaper-Produce], Stopped (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
[2017-07-17 14:29:19,256] INFO [ThrottledRequestReaper-Produce], Shutdown completed (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
[2017-07-17 14:29:19,256] INFO [KafkaApi-0] Shutdown complete. (kafka.server.KafkaApis)
[2017-07-17 14:29:19,258] INFO [Replica Manager on Broker 0]: Shutting down (kafka.server.ReplicaManager)
[2017-07-17 14:29:19,259] INFO [ReplicaFetcherManager on broker 0] shutting down (kafka.server.ReplicaFetcherManager)
[2017-07-17 14:29:19,262] INFO [ReplicaFetcherManager on broker 0] shutdown completed (kafka.server.ReplicaFetcherManager)
[2017-07-17 14:29:19,262] INFO [ExpirationReaper-0], Shutting down (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-07-17 14:29:19,293] INFO [ExpirationReaper-0], Stopped (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-07-17 14:29:19,293] INFO [ExpirationReaper-0], Shutdown completed (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-07-17 14:29:19,293] INFO [ExpirationReaper-0], Shutting down (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-07-17 14:29:19,337] INFO [ExpirationReaper-0], Stopped (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-07-17 14:29:19,337] INFO [ExpirationReaper-0], Shutdown completed (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-07-17 14:29:19,341] INFO [Replica Manager on Broker 0]: Shut down completely (kafka.server.ReplicaManager)
[2017-07-17 14:29:19,342] INFO [ExpirationReaper-0], Shutting down (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-07-17 14:29:19,372] INFO [ExpirationReaper-0], Stopped (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-07-17 14:29:19,372] INFO [ExpirationReaper-0], Shutdown completed (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-07-17 14:29:19,375] INFO [GroupCoordinator 0]: Shutting down. (kafka.coordinator.GroupCoordinator)
[2017-07-17 14:29:19,375] INFO [ExpirationReaper-0], Shutting down (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-07-17 14:29:19,538] INFO [ExpirationReaper-0], Stopped (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-07-17 14:29:19,538] INFO [ExpirationReaper-0], Shutdown completed (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-07-17 14:29:19,538] INFO [ExpirationReaper-0], Shutting down (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-07-17 14:29:19,595] INFO [ExpirationReaper-0], Stopped (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-07-17 14:29:19,595] INFO [ExpirationReaper-0], Shutdown completed (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2017-07-17 14:29:19,597] INFO [GroupCoordinator 0]: Shutdown complete. (kafka.coordinator.GroupCoordinator)
[2017-07-17 14:29:19,598] INFO Shutting down. (kafka.log.LogManager)
[2017-07-17 14:29:19,641] INFO Shutdown complete. (kafka.log.LogManager)
[2017-07-17 14:29:19,649] INFO Terminate ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread)
[2017-07-17 14:29:19,653] INFO Session: 0x15d4327c712010f closed (org.apache.zookeeper.ZooKeeper)
[2017-07-17 14:29:19,655] INFO [Kafka Server 0], shut down completed (kafka.server.KafkaServer)
[2017-07-17 14:29:19,655] INFO EventThread shut down for session: 0x15d4327c712010f (org.apache.zookeeper.ClientCnxn)
Raw
when the KStream is created properly on startup, I see these logs
11:13:00.665 [StreamThread-4] DEBUG o.a.k.c.c.i.AbstractCoordinator - Received successful Heartbeat response for group map-function-scala-example
11:13:00.898 [StreamThread-4] DEBUG o.a.k.c.consumer.internals.Fetcher - Sending fetch for partitions [test-name-0] to broker 10.20.69.86:9092 (id: 0 rack: null)
11:13:01.400 [StreamThread-4] DEBUG o.a.k.c.consumer.internals.Fetcher - Sending fetch for partitions [test-name-0] to broker 10.20.69.86:9092 (id: 0 rack: null)
11:13:01.903 [StreamThread-4] DEBUG o.a.k.c.consumer.internals.Fetcher - Sending fetch for partitions [test-name-0] to broker 10.20.69.86:9092 (id: 0 rack: null)
11:13:02.408 [StreamThread-4] DEBUG o.a.k.c.consumer.internals.Fetcher - Sending fetch for partitions [test-name-0] to broker 10.20.69.86:9092 (id: 0 rack: null)
11:13:02.910 [StreamThread-4] DEBUG o.a.k.c.consumer.internals.Fetcher - Sending fetch for partitions [test-name-0] to broker 10.20.69.86:9092 (id: 0 rack: null)
11:13:03.412 [StreamThread-4] DEBUG o.a.k.c.consumer.internals.Fetcher - Sending fetch for partitions [test-name-0] to broker 10.20.69.86:9092 (id: 0 rack: null)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment