? ?數(shù)據(jù)采集是大數(shù)據(jù)分析全流程的重要環(huán)節(jié),典型的數(shù)據(jù)采集工具包括ETL工具、日志采集工具、數(shù)據(jù)遷移工具等。
為壽縣等地區(qū)用戶提供了全套網(wǎng)頁設(shè)計(jì)制作服務(wù),及壽縣網(wǎng)站建設(shè)行業(yè)解決方案。主營業(yè)務(wù)為成都網(wǎng)站制作、成都網(wǎng)站設(shè)計(jì)、壽縣網(wǎng)站設(shè)計(jì),以傳統(tǒng)方式定制建設(shè)網(wǎng)站,并提供域名空間備案等一條龍服務(wù),秉承以專業(yè)、用心的態(tài)度為用戶提供真誠的服務(wù)。我們深信只要達(dá)到每一位用戶的要求,就會(huì)得到認(rèn)可,從而選擇與我們長期合作。這樣,我們也可以走得更遠(yuǎn)!
? ?Flume是一個(gè)高可用的、高可靠的、分布式的海量日志采集、聚合和傳輸?shù)南到y(tǒng)。
1.安裝Flume
下載:http://www.apache.org/dist/flume/
hadoop@dblab:/usr/local$ sudo wgethttp://www.apache.org/dist/flume/1.7.0/apache-flume-1.7.0-bin.tar.gz
hadoop@dblab:/usr/local$ sudo tar -zxvf apache-flume-1.7.0-bin.tar.gz?
hadoop@dblab:/usr/local$ sudo mv apache-flume-1.7.0-bin ./flume
2.配置環(huán)境變量
hadoop@dblab:/usr/local$ sudo vim ~/.bashrc
export FLUME_HOME=/usr/local/flume
export FLUME_CONF_DIR=$FLUME_HOME/conf
export JAVA_HOME=/usr/lib/jvm/default-java
export PATH=$PATH:$HIVE_HOME/bin
hadoop@dblab:/usr/local$ source ~/.bashrc
hadoop@dblab:/usr/local/flume/conf$ mv flume-env.sh.template? flume-env.sh
hadoop@dblab:/usr/local/flume/conf$ sudo vim? ?flume-env.sh??
#在flume-env.sh文件開頭加入如下語句:? ? ? ? ? ? ? ? ? ??
export JAVA_HOME=/usr/lib/jvm/default-java
3.啟動(dòng)Flume
hadoop@dblab:/usr/local/flume$ cd /usr/local/flume
hadoop@dblab:/usr/local/flume$ ./bin/flume-ng version
錯(cuò)誤: 找不到或無法加載主類 org.apache.flume.tools.GetJavaProperty
Flume 1.7.0
Source code repository: https://git-wip-us.apache.org/repos/asf/flume.git
Revision: 511d868555dd4d16e6ce4fedc72c2d1454546707
Compiled by bessbd on Wed Oct 12 20:51:10 CEST 2016
From source with checksum 0d21b3ffdc55a07e1d08875872c00523
hadoop@dblab:/usr/local/flume$ cd /usr/local/hbase/conf
hadoop@dblab:/usr/local/hbase/conf$ sudo vim hbase-env.sh? ?
#export HBASE_CLASSPATH=/usr/local/hadoop/conf #注釋該行,即解決上述問題
hadoop@dblab:/usr/local/flume$ ./bin/flume-ng version
Flume 1.7.0
Source code repository: https://git-wip-us.apache.org/repos/asf/flume.git
Revision: 511d868555dd4d16e6ce4fedc72c2d1454546707
Compiled by bessbd on Wed Oct 12 20:51:10 CEST 2016
From source with checksum 0d21b3ffdc55a07e1d08875872c00523