Skip to content

Instantly share code, notes, and snippets.

@iamshreeram
Last active October 11, 2019 17:53
Show Gist options
  • Select an option

  • Save iamshreeram/57b0644a1c1698303646f0356f722697 to your computer and use it in GitHub Desktop.

Select an option

Save iamshreeram/57b0644a1c1698303646f0356f722697 to your computer and use it in GitHub Desktop.
Sending data to kafka
from pykafka import KafkaClient
path="/path/of/log/file.log"
host="ip.ad.dr.ess"
port=9092
topic="name-of-the-topic"
client = KafkaClient(host:port)
topic = client.topics[topic]
producer = topic.get_producer(sync=True,delivery_reports=True)
def sendkafka(msg):
producer.produce(msg.encode())
with open(path) as f:
for line in f:
sendkafka(line)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment