tips, tricks and common errors with kafka and spring-kafka

kafka_ref.md

Schema registry console-consumer exception

• When using ./usr/bin/kafka-avro-console-consumer an exception happens with trace like:

ERROR Unknown error when running consumer:  (kafka.tools.ConsoleConsumer$)
org.apache.kafka.common.errors.SerializationException: Error deserializing Avro message for id 2
Caused by: java.net.ConnectException: Connection refused (Connection refused)

So the command like: ./kafka-avro-console-consumer --bootstrap-server kafka:9092 --topic raw-stock-quote-tick

would fail due to client try to connect by default with default port from schema-registry and in this case it's possibly not there the fix is to specify the schema.registry.url pointing to proper schema-registry address as shown below:

./kafka-avro-console-consumer --bootstrap-server kafka:9092 --topic raw-stock-quote-tick --property schema.registry.url=http://localhost:9010

When using spring-kafka

• When avro schema is configured it might cause an incompatibility error using spring-kafka, the following configuration fixes it:

spring:
  kafka:
    consumer:
      properties:
        specific:
          avro:
            reader: true