HDFS basic commands

Here we will see some basic commands to work on Hadoop Distributed File System (HDFS).

Basic hdfs command syntax is

 $ hdfs dfs -unix_command

1. View hdfs directory

$ hdfs dfs –ls

2. Create a directory

$ hdfs dfs –mkdir /user/hadoop_dir

3. Change permission of directory or files

$ hdfs dfs –chmod –R 777 /user/hadoop_dir

4. Change ownership of file or directory

$ hdfs dfs –chown –R [owner_name] /user/hadoop_dir

5. Copy local file to hdfs directory

$ hdfs dfs –put “/home/local_dir/n1.txt” “/user/hadoop_dir”

Or

$ hdfs dfs –copyFromLocal “/home/local_dir/n1.txt” “/user/hadoop_dir”

6. Create a txt file under a Hadoop folder

$ hdfs dfs –touchz /user/hadoop_dir/n2.txt

7. Determine size of hdfs file (not local file)

$ hdfs dfs –du –h /user/hadoop_dir/n2.txt

8. Print first 5 lines from hdfs file

$ hdfs dfs –cat “/user/hadoop_dir/n2.txt” || head –n 5

9. Copy a file to another or same hdfs directory with different name

$ hdfs dfs –cp “/user/hadoop_dir/n1.txt” “/user/hadoop_dir/n1_copy.txt”

10. Copy hdfs file to local filesystem

$ hdfs dfs –get “/user/hadoop_dir/n1_copy.txt” “/home/local_dir/”

Or

$ hdfs dfs –copyToLocal “/user/hadoop_dir/n1_copy.txt” “/home/local_dir/”

11.  Delete file from hdfs directory

$ hdfs dfs –rm /user/hadoop_dir/n1_copy.txt

12. Delete hdfs directory

$ hdfs dfs –rm –r /user/hadoop_dir