前言:
前面幾篇博客主要介紹了MapReduce與Yarn的架構(gòu)設(shè)計(jì)及簡(jiǎn)單工作流程,本篇文章將以wordcount程序?yàn)槔?,?jiǎn)單介紹下Yarn的使用。
[root@hadoop000 ~]# su - hadoop
[hadoop@hadoop000 ~]$ jps
9201 SecondaryNameNode
9425 ResourceManager
13875 Jps
9540 NodeManager
8852 NameNode
8973 DataNode
# 創(chuàng)建wordcount目錄
[hadoop@hadoop000 ~]$ hdfs dfs -mkdir -p /wordcount/input
[hadoop@hadoop000 ~]$ vi test.log
jepson ruoze
hero yimi xjp
123
a b a
[hadoop@hadoop000 ~]$ hdfs dfs -put test.log /wordcount/input
[hadoop@hadoop000 ~]$ hdfs dfs -ls /wordcount/input
Found 1 items
-rw-r--r-- 1 hadoop supergroup 37 2018-05-29 20:38 /wordcount/input/test.log
# 執(zhí)行wordcount示例jar包
[hadoop@hadoop000 ~]$ yarn jar \
> /opt/software/hadoop-2.8.1/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.8.1.jar \
> wordcount \
> /wordcount/input \
> /wordcount/output
18/05/29 20:40:59 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
18/05/29 20:40:59 INFO input.FileInputFormat: Total input files to process : 1
18/05/29 20:41:00 INFO mapreduce.JobSubmitter: number of splits:1
18/05/29 20:41:00 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1526991305992_0001
18/05/29 20:41:01 INFO impl.YarnClientImpl: Submitted application application_1526991305992_0001
18/05/29 20:41:01 INFO mapreduce.Job: The url to track the job: http://hadoop000:8088/proxy/application_1526991305992_0001/
18/05/29 20:41:01 INFO mapreduce.Job: Running job: job_1526991305992_0001
18/05/29 20:41:14 INFO mapreduce.Job: Job job_1526991305992_0001 running in uber mode : false
18/05/29 20:41:14 INFO mapreduce.Job: map 0% reduce 0%
18/05/29 20:41:23 INFO mapreduce.Job: map 100% reduce 0%
18/05/29 20:41:29 INFO mapreduce.Job: map 100% reduce 100%
18/05/29 20:41:30 INFO mapreduce.Job: Job job_1526991305992_0001 completed successfully
18/05/29 20:41:30 INFO mapreduce.Job: Counters: 49
# 查看結(jié)果
[hadoop@hadoop000 ~]$ hdfs dfs -ls /wordcount/output
Found 2 items
-rw-r--r-- 1 hadoop supergroup 0 2018-05-29 20:41 /wordcount/output/_SUCCESS
-rw-r--r-- 1 hadoop supergroup 51 2018-05-29 20:41 /wordcount/output/part-r-00000
[hadoop@hadoop000 ~]$ hdfs dfs -cat /wordcount/output/part-r-00000
123 1
a 2
b 1
hero 1
jepson 1
ruoze 1
xjp 1
yimi 1
登錄網(wǎng)頁(yè)查看相關(guān)信息:http://192.168.6.217:8088/cluster
yarn jar --run a jar file
yarn application -list --列出在跑的job
yarn application -kill application_1526991305992_0001(job的id) --殺掉在跑的job
參考:https://blog.csdn.net/yczws1/article/details/21794873
另外有需要云服務(wù)器可以了解下創(chuàng)新互聯(lián)scvps.cn,海內(nèi)外云服務(wù)器15元起步,三天無(wú)理由+7*72小時(shí)售后在線(xiàn),公司持有idc許可證,提供“云服務(wù)器、裸金屬服務(wù)器、高防服務(wù)器、香港服務(wù)器、美國(guó)服務(wù)器、虛擬主機(jī)、免備案服務(wù)器”等云主機(jī)租用服務(wù)以及企業(yè)上云的綜合解決方案,具有“安全穩(wěn)定、簡(jiǎn)單易用、服務(wù)可用性高、性?xún)r(jià)比高”等特點(diǎn)與優(yōu)勢(shì),專(zhuān)為企業(yè)上云打造定制,能夠滿(mǎn)足用戶(hù)豐富、多元化的應(yīng)用場(chǎng)景需求。