kafka-python example with Kerberos auth

kafka-python-sasl-gssapi.py

# Requirements: kafka-python gssapi krbticket
import os
import time
from kafka import KafkaConsumer, KafkaProducer
from krbticket import KrbConfig, KrbCommand

try:
    os.environ['KRB5CCNAME'] = '/tmp/krb5cc_<myusername>'
    kconfig = KrbConfig(principal='araujo', keytab='/path/to/<myusername>.keytab')
    KrbCommand.kinit(kconfig)

    # Kafka broker
    BROKERS = ['host.cloudera.site:9093']
    # Kafka topics
    TOPIC = 'demo'

    producer = KafkaProducer(
                   bootstrap_servers=BROKERS,
                   security_protocol='SASL_SSL',
                   ssl_cafile='/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_cacerts.pem',
                   sasl_mechanism='GSSAPI',
                   sasl_kerberos_service_name='kafka',
               )

    producer.send(TOPIC, ('Hello, World! %s' % (time.time(),)).encode())
    producer.flush()

    consumer = KafkaConsumer(
                   bootstrap_servers=BROKERS,
                   security_protocol='SASL_SSL',
                   ssl_cafile='/var/lib/cloudera-scm-agent/agent-cert/cm-auto-global_cacerts.pem',
                   sasl_mechanism='GSSAPI',
                   sasl_kerberos_service_name='kafka',
                   auto_offset_reset='earliest',
               )
    consumer.subscribe([TOPIC])

    for message in consumer:
        print(message)

finally:
    print("Destroying ticket")
    KrbCommand.kdestroy(kconfig)

Hi there, thanks a lot for the script, can you tell me what this line number 8 is meant for? Do i have to add the environment variable? sorry i am new to kafka. Is "KRB5CCNAME" fixed variable name? or it changes upon the server configuration?

Hi there, thanks a lot for the script, can you tell me what this line number 8 is meant for? Do i have to add the environment variable? sorry i am new to kafka. Is "KRB5CCNAME" fixed variable name? or it changes upon the server configuration?

@manishr38
what this line number 8 is meant for? - it is used to set kerberos default credential cache name
Do i have to add the environment variable? - you don't. you can check the default cache name using the command klist
Is "KRB5CCNAME" fixed variable name? - yes
or it changes upon the server configuration? - no

Thanks for answering this, @aiganymus !
Even though you don't need to set KRB5CCNAME, I'd highly recommend you to do so. If you have multiple scripts using the default cache, the actions of a script (kinit/kdestroy) can affect the other scripts. If you set one cache file for each script, using the KRB5CCNAME variable, then you're safe.

Hello @asdaraujo,
I'm trying to implement this to be able to connect to Kafka from a Windows Server Application which is not supported by the KrbTicket.
Do you have any idea of what could be a possible solution?
Thanks