1

下载

http://flume.apache.org/download.html下载二进制版本

安装

tar zvxf apache-flume-1.6.0-bin.tar.gz

目录结构

  • bin - 包含启动脚本

  • conf - 包含配置文件模板

  • docs - 包含文档

  • lib - 依赖的jar包

  • tools - 包含log4j appender的实现

配置文件结构

1)主要结构

图片描述

2)顶层

<agent-name>.sources=<source> <source>
<agent-name>.channels=<channel> <channel>
<agent-name>.sinks=<sink> <sink>

3)底层

<agent-name>.<component-type>.<component-name>.<configuration-parameter>=<value>

4)实例single-node-demo.properties

a1.sources=src1
a1.sinks=sink1
a1.channels=ch1

# Configure the source
a1.sources.src1.type=netcat
a1.sources.src1.bind=localhost
a1.sources.src1.port=51000

# Sink Configuration
a1.sinks.sink1.type=logger

# Channel configuration
a1.channels.ch1.type=memory
a1.channels.ch1.capacity=1000
a1.channels.ch1.transactionCapacity=100

# Link stuff together
a1.sources.src1.channels=ch1
a1.sinks.sink1.channel=ch1

启动agent

$bin/flume-ng agent 
--conf-file conf/single-node-demo.properties 
--name a1 
--conf ./conf/ 
-Dflume.root.logger=INFO,console

也可以简写为

bin/flume-ng agent 
-c conf/single-node-demo.properties 
-n a1 
-f ./conf/ 
-Dflume.root.logger=INFO,console
  • -c 相当于--conf-file

  • -f 相当于--conf

  • -n 相当于--name

netcat发消息

nc localhost 51000
hello
OK
world
OK

看flume的输出

2016-01-12 22:11:20,603 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:94)] Event: { headers:{} body: 68 65 6C 6C 6F                                  hello }
2016-01-12 22:11:20,603 (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:94)] Event: { headers:{} body: 77 6F 72 6C 64                                  world }

codecraft
11.9k 声望2k 粉丝

当一个代码的工匠回首往事时,不因虚度年华而悔恨,也不因碌碌无为而羞愧,这样,当他老的时候,可以很自豪告诉世人,我曾经将代码注入生命去打造互联网的浪潮之巅,那是个很疯狂的时代,我在一波波的浪潮上留下...


引用和评论

0 条评论