分布式安裝(至少三臺主機(jī)):
環(huán)境所需軟件:
CentOS7
hadoop-2.7.3.tar.gz
jdk-8u102-linux-x64.tar.gz
安裝前準(zhǔn)備工作:
配置免密鑰登陸
cd
ssh-keygen -t rsa
一直回車,直到結(jié)束
ssh-copy-id .ssh/id_rsa.pub bigdata1
ssh-copy-id .ssh/id_rsa.pub bigdata2
ssh-copy-id .ssh/id_rsa.pub bigdata3
同步時間
通過設(shè)置計劃任務(wù)實現(xiàn)各主機(jī)間的時間同步
vim /etc/crontab
0 0 1 root ntpdate -s time.windows.com
或者部署一個時間服務(wù)器實現(xiàn)同步,這里就不詳細(xì)講解了
(*)hdfs-site.xml
dfs.replication
2
core-site.xml
fs.defaultFS
hdfs://bigdata1:9000
hadoop.tmp.dir
/root/training/hadoop-2.7.3/tmp
mapred-site.xml
mapreduce.framework.name
yarn
yarn-site.xml
yarn.resourcemanager.hostname
bigdata1
yarn.nodemanager.aux-services
mapreduce_shuffle
對NameNode進(jìn)行格式化: hdfs namenode -format
日志:Storage directory /root/training/hadoop-2.7.3/tmp/dfs/name has been successfully formatted.
scp -r /root/training/hadoop-2.7.3 bigdata2:/root/training/hadoop-2.7.3
scp -r /root/training/hadoop-2.7.3 bigdata3:/root/training/hadoop-2.7.3
啟動:start-all.sh = start-dfs.sh + start-yarn.sh
驗證
(*)命令行:hdfs dfsadmin -report
(*)網(wǎng)頁:HDFS:http://192.168.157.12:50070/
Yarn:http://192.168.157.12:8088
(*)Demo:測試MapReduce程序
example: /root/training/hadoop-2.7.3/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar
hadoop jar hadoop-mapreduce-examples-2.7.3.jar wordcount /input/data.txt /output/wc1204
另外有需要云服務(wù)器可以了解下創(chuàng)新互聯(lián)scvps.cn,海內(nèi)外云服務(wù)器15元起步,三天無理由+7*72小時售后在線,公司持有idc許可證,提供“云服務(wù)器、裸金屬服務(wù)器、高防服務(wù)器、香港服務(wù)器、美國服務(wù)器、虛擬主機(jī)、免備案服務(wù)器”等云主機(jī)租用服務(wù)以及企業(yè)上云的綜合解決方案,具有“安全穩(wěn)定、簡單易用、服務(wù)可用性高、性價比高”等特點與優(yōu)勢,專為企業(yè)上云打造定制,能夠滿足用戶豐富、多元化的應(yīng)用場景需求。