FlumeToKafka

install kafka:  https://archive.cloudera.com/kafka/kafka/2/kafka/quickstart.html
start zookeeper if not started(one comes with kafka):
bin/zookeeper-server-start.sh config/zookeeper.properties

start kafka server:
bin/kafka-server-start.sh config/server.properties

create topic:
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic ftoKafka_topic
bin/kafka-topics.sh --list --zookeeper localhost:2181

ftokafka.conf
ftokafka.sources = logsource
ftokafka.sinks = kafkasink
ftokafka.channels = mchannel

# Describe/configure the source
ftokafka.sources.logsource.type = exec
ftokafka.sources.logsource.command = tail -F /opt/gen_logs/logs/access.log

# Describe the sink
ftokafka.sinks.kafkasink.type = org.apache.flume.sink.kafka.KafkaSink 
ftokafka.sinks.kafkasink.brokerList = localhost:9092 
ftokafka.sinks.kafkasink.topic = ftokafka_topic 

# Use a channel which buffers events in memory
ftokafka.channels.mchannel.type = memory
ftokafka.channels.mchannel.capacity = 1000
ftokafka.channels.mchannel.transactionCapacity = 100

# Bind the source and sink to the channel
ftokafka.sources.logsource.channels = mchannel
ftokafka.sinks.kafkasink.channel = mchannel
Start flume agent that moves data from log files to kafka sink
flume-ng agent -n ftokafka -c . -f ftokafka.conf

consume on the console: