Skip to content

Instantly share code, notes, and snippets.

@salekseev
Created July 25, 2014 20:46
Show Gist options
  • Save salekseev/310c94d7f12d40cd97d9 to your computer and use it in GitHub Desktop.
Save salekseev/310c94d7f12d40cd97d9 to your computer and use it in GitHub Desktop.
# Name the components on this agent
a1.sources = r1
a1.sinks = k1
a1.channels = c1
# Describe/configure the source
a1.sources.r1.type = spooldir
a1.sources.r1.spoolDir = ~/source
a1.sources.r1.fileHeader = true
a1.sources.r1.basenameHeader = true
a1.sources.r1.batchSize = 1
a1.sources.r1.deletePolicy = immediate
a1.sources.r1.deserializer = org.apache.flume.sink.solr.morphline.BlobDeserializer$Builder
# Describe the sink
a1.sinks.k1.type = file_roll
a1.sinks.k1.batchSize = 1
a1.sinks.k1.sink.directory = ~/dest
a1.sinks.k1.rollInterval = 0
# Use a channel which buffers events in memory
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100
# Bind the source and sink to the channel
a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment