18 Feb
Hadoop HDFS: Space Available
I’ve been playing around with Hadoop recently. It’s pretty slick. The EC2 scripts make it incredibly easy to set up and work with. HDFS is pretty neat too. I’ve been working with 10GB data sets and moving the data around and working with them isn’t painful.
I was curious as to how much free space was available in my cluster. It took a bit of digging around to figure out how to get this information, so I figured I’d post it here.
Running bin/hadoop dfsadmin -report will give a breakdown of your node storage:
[root@ip-10-250-147-64 ~]# h dfsadmin -report Configured Capacity: 718626299904 (669.27 GB) Present Capacity: 681713795072 (634.9 GB) DFS Remaining: 681713745920 (634.9 GB) DFS Used: 49152 (48 KB) DFS Used%: 0%
------------------------------------------------- Datanodes available: 2 (2 total, 0 dead)
Name: 10.250.106.176:50010 Decommission Status : Normal Configured Capacity: 359313149952 (334.64 GB) DFS Used: 24576 (24 KB) Non DFS Used: 18456252416 (17.19 GB) DFS Remaining: 340856872960(317.45 GB) DFS Used%: 0% DFS Remaining%: 94.86% Last contact: Wed Feb 18 12:28:16 EST 2009
Name: 10.250.146.127:50010 Decommission Status : Normal Configured Capacity: 359313149952 (334.64 GB) DFS Used: 24576 (24 KB) Non DFS Used: 18456252416 (17.19 GB) DFS Remaining: 340856872960(317.45 GB) DFS Used%: 0% DFS Remaining%: 94.86% Last contact: Wed Feb 18 12:28:18 EST 2009
