這篇文章主要介紹如何查看HDFS文件系統(tǒng)空間使用情況，文中介紹的非常詳細(xì)，具有一定的參考價(jià)值，感興趣的小伙伴們一定要看完！
創(chuàng)新互聯(lián)建站一直秉承“誠信做人，踏實(shí)做事”的原則，不欺瞞客戶，是我們最起碼的底線！ 以服務(wù)為基礎(chǔ)，以質(zhì)量求生存，以技術(shù)求發(fā)展，成交一個(gè)客戶多一個(gè)朋友！為您提供成都網(wǎng)站設(shè)計(jì)、做網(wǎng)站、成都網(wǎng)頁設(shè)計(jì)、成都小程序開發(fā)、成都網(wǎng)站開發(fā)、成都網(wǎng)站制作、成都軟件開發(fā)、app開發(fā)定制是成都本地專業(yè)的網(wǎng)站建設(shè)和網(wǎng)站設(shè)計(jì)公司，等你一起來見證！
1、查看集群的空間使用狀態(tài)
[hduser@master1 bin]$ ./hdfs dfsadmin -report
Configured Capacity: 8453768839168 (7.69 TB)
Present Capacity: 7959274510394 (7.24 TB)
DFS Remaining: 4128882869718 (3.76 TB)
DFS Used: 3830391640676 (3.48 TB)
DFS Used%: 48.12%
Under replicated blocks: 18
Blocks with corrupt replicas: 0
Missing blocks: 0
Missing blocks (with replication factor 1): 0
-------------------------------------------------
Live datanodes (5):
Name: 192.168.11.137:50010 (slave5)
Hostname: slave5
Decommission Status : Normal
Configured Capacity: 1056617381888 (984.05 GB)
DFS Used: 736041209958 (685.49 GB)
Non DFS Used: 66876043079 (62.28 GB)
DFS Remaining: 253700128851 (236.28 GB)
DFS Used%: 69.66%
DFS Remaining%: 24.01%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 30
Last contact: Tue Jul 03 15:06:51 CST 2018
Name: 192.168.11.134:50010 (slave2)
Hostname: slave2
Decommission Status : Normal
Configured Capacity: 2113511358464 (1.92 TB)
DFS Used: 749551308902 (698.07 GB)
Non DFS Used: 120793546401 (112.50 GB)
DFS Remaining: 1243166503161 (1.13 TB)
DFS Used%: 35.46%
DFS Remaining%: 58.82%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 23
Last contact: Tue Jul 03 15:06:50 CST 2018
Name: 192.168.11.133:50010 (slave1)
Hostname: slave1
Decommission Status : Normal
Configured Capacity: 2113511358464 (1.92 TB)
DFS Used: 792363753676 (737.95 GB)
Non DFS Used: 118847225403 (110.69 GB)
DFS Remaining: 1202300379385 (1.09 TB)
DFS Used%: 37.49%
DFS Remaining%: 56.89%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 21
Last contact: Tue Jul 03 15:06:50 CST 2018
Name: 192.168.11.135:50010 (slave3)
Hostname: slave3
Decommission Status : Normal
Configured Capacity: 2113511358464 (1.92 TB)
DFS Used: 752023101542 (700.38 GB)
Non DFS Used: 121481747963 (113.14 GB)
DFS Remaining: 1240006508959 (1.13 TB)
DFS Used%: 35.58%
DFS Remaining%: 58.67%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 26
Last contact: Tue Jul 03 15:06:50 CST 2018
Name: 192.168.11.136:50010 (slave4)
Hostname: slave4
Decommission Status : Normal
Configured Capacity: 1056617381888 (984.05 GB)
DFS Used: 800412266598 (745.44 GB)
Non DFS Used: 66495765928 (61.93 GB)
DFS Remaining: 189709349362 (176.68 GB)
DFS Used%: 75.75%
DFS Remaining%: 17.95%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 30
2、分析192.168.11.133
$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/VolGroup00-LVroot
17G 12G 4.9G 70% /
tmpfs 7.8G 0 7.8G 0% /dev/shm
/dev/sda1 190M 125M 56M 70% /boot
/dev/mapper/VolGroup00-LVhome
477M 2.4M 449M 1% /home
/dev/mapper/VolGroup00-LVcloud
2.9G 247M 2.5G 9% /opt/cloud
/dev/mapper/VolGroup00-LVtmp
7.8G 19M 7.4G 1% /tmp
/dev/mapper/VolGroup00-LVvar
25G 786M 23G 4% /var
/dev/mapper/VolGroup1-LVdata1
985G 384G 551G 42% /data
/dev/mapper/VolGroup2-LVdata2
985G 365G 570G 40% /data2
數(shù)據(jù)統(tǒng)計(jì)：
/data和/data2為數(shù)據(jù)存放目錄
總的容量 = 985G + 985G = 1970G = 1.92T
已使用容量 = 384G + 365G = 749G
未使用容量 = 551G + 570G = 1121G
未顯示容量 = 1970G - （749G + 1121G）= 100G
/data目錄中的hadoop和hbase的解壓文件占用10G左右
dfs文件系統(tǒng)分析：
Name: 192.168.11.133:50010 (slave1)
Hostname: slave1
Decommission Status : Normal
Configured Capacity: 2113511358464 (1.92 TB)
DFS Used: 792363753676 (737.95 GB)
Non DFS Used: 118847225403 (110.69 GB)
DFS Remaining: 1202300379385 (1.09 TB)
DFS Used%: 37.49%
DFS Remaining%: 56.89%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 21
Last contact: Tue Jul 03 15:06:50 CST 2018
說明解析：
Configured Capacity:配置容量
DFS Used:DFS已使用的容量
Non DFS Used:非DFS使用的容量
DFS Remaining:DFS剩余的容量
數(shù)據(jù)統(tǒng)計(jì)：
Configured Capacity = 1.92 TB = 1970 GB
DFS Used = 737.95 GB
Non DFS Used = 110.69 GB
DFS Remaining = 1.09 TB = 1116.16 GB
DFS Used + Non DFS Used + DFS Remaining = 737.95 GB + 110.69 GB + 1116.16 GB = 1964.8 GB = Configured Capacity
所以得出結(jié)果：
du -h和./hdfs dfsadmin -report顯示的容量對應(yīng)關(guān)系為：
總的容量 ---> Configured Capacity
已使用容量 ---> DFS Used
未使用容量 ---> DFS Remaining
未顯示容量 + /data下非dfs文件 ---> Non DFS Used
非dfs文件指的是不是同過dfs文件系統(tǒng)上傳的文件，通過mv、cp等方式傳輸?shù)奈募?/p>
統(tǒng)計(jì)192.168.11.136和192.168.11.133類似。
3、尋找消失的100G
Non DFS Used占用了110.69GB，/data下非dfs文件有10GB左右，那么剩下的100GB去哪了？
查看保留空間
# tune2fs -l /dev/mapper/VolGroup1-LVdata1
Reserved block count: 13107148
13107148 * 4 / 1024 /1024 = 49.99
# tune2fs -l /dev/mapper/VolGroup2-LVdata2
Reserved block count: 13107148
13107148 * 4 / 1024 /1024 = 49.99
消失的100G就是兩個(gè)分區(qū)的保留空間之和
以上是“如何查看HDFS文件系統(tǒng)空間使用情況”這篇文章的所有內(nèi)容，感謝各位的閱讀！希望分享的內(nèi)容對大家有幫助，更多相關(guān)知識，歡迎關(guān)注創(chuàng)新互聯(lián)行業(yè)資訊頻道！
重慶分公司
重慶分公司
