Once you have Hadoop setup,  either single setup or cluster setup, the first thing you want to try is to create files and directories on Hadoop Distributed File System (HDFS), surely you can find whole HDFS commands reference

Below are some examples for mostly used HDFS commands for files and directories management. Hadoop 2.7.3 on SL7(RHEL7/CentOS7)

Create a directory in HDFS

Usage:

$ hdfs dfs -mkdir <paths>

Example:

$ hdfs dfs -mkdir /fibrevillage

List the contents of a directory in HDFS

Usage :

hdfs dfs -ls <args> 

Example:

$ hdfs dfs -ls /
Found 3 items
drwxr-xr-x   - hadoop supergroup          0 2016-12-05 16:19 /fibrevillage
drwxr-xr-x   - hadoop supergroup          0 2016-12-01 15:28 /system
drwxr-xr-x   - hadoop supergroup          0 2016-11-25 00:20 /user 

 
Upload a file to HDFS.

Copy single file, or multiple files from local file system to the Hadoop data file system
 
Usage: 

hdfs dfs -put <localsrc> ... <HDFS_dest_Path>

Example1, copy a whole directory recursively to HDFS:

$ hdfs dfs -put /home/hadoop /fibrevillage
$ hdfs dfs -ls /fibrevillage/hadoop/
Found 6 items
-rw-r--r--   2 hadoop supergroup        575 2016-12-05 16:26 /fibrevillage/hadoop/.bash_profile
-rw-r--r--   2 hadoop supergroup        627 2016-12-05 16:26 /fibrevillage/hadoop/.bashrc
-rw-r--r--   2 hadoop supergroup        334 2016-12-05 16:26 /fibrevillage/hadoop/.emacs
drwxr-xr-x   - hadoop supergroup          0 2016-12-05 16:26 /fibrevillage/hadoop/.ssh
-rw-r--r--   2 hadoop supergroup  339281920 2016-12-05 16:26 /fibrevillage/hadoop/hadoop-2.7.3.tar

Example2, copy multiple files to HDFS

$ hdfs dfs -put /etc/redhat-release /etc/fstab /fibrevillage
$ hdfs dfs -ls /fibrevillage/
Found 3 items
-rw-r--r--   2 hadoop supergroup       4359 2016-12-05 16:34 /fibrevillage/fstab
drwxr-xr-x   - hadoop supergroup          0 2016-12-05 16:26 /fibrevillage/hadoop
-rw-r--r--   2 hadoop supergroup         40 2016-12-05 16:34 /fibrevillage/redhat-release

Download a file from HDFS.

Copies/Downloads files to the local file system
Usage:

hdfs dfs -get <hdfs_src> <localdst> 

Example:

$ hdfs dfs -get /fibrevillage/fstab .
$ ls fstab
fstab

Check a file state in HDFS

Check a file/directory state in HDFS

Usage:

hdfs dfs -state <path>

Example:

$ hdfs dfs -stat '%F %b %n %o' /fibrevillage/fstab
regular file 4359 fstab 134217728

 

See contents of a file in HDFS

Same as unix cat command:
Usage:

hdfs dfs -cat <path[filename]> 

Example:

$ hdfs dfs -cat /fibrevillage/fstab
#
# /etc/fstab
# Created by anaconda on Thu Jul 28 19:30:56 2016
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=23385b5a-c779-4883-81ed-be1f8689bd1b /     xfs     defaults        0 0
UUID=60fd6c6f-c683-402f-b98b-7e022066fe43 /boot xfs     defaults        0 0
/dev/mapper/vg0-home    /home                   xfs     defaults        0 0
/dev/mapper/vg0-opt     /opt                    xfs     defaults        0 0
/dev/mapper/vg0-tmp     /tmp                    xfs     defaults        0 0
...

Copy a file from source to destination in HDFS

This command allows multiple sources as well in which case the destination must be a directory.
 
Usage:

hdfs dfs -cp <source> <dest>

Example:

hdfs dfs -cp /etc/rc.local /fibrevillage/

 

Note: hdfs dfs -cp copy file or directories recursively, all the directory’s files and subdirectories to the bottom of the directory tree are copied.
It is a tool used for large inter/intra-cluster copying

Copy a file from/to Local file system to HDFS

Similar to put/get command, except that the source is restricted to a local file reference.

Usage:

hdfs dfs -copyFromLocal/copyToLocal <localsrc> URI 
hdfs dfs -copyToLocal URI <localsrc>

Example:

hdfs dfs -copyFromLocal /home/hadoop/test1  /fibrevillage/test1
hdfs dfs -copyToLocal /fibrevillage/test /home/hadoop/test1

Move file from source to destination in HDFS.

Note:- Moving files across filesystem is not permitted.
 
Usage :

hdfs dfs -mv <src> <dest>

Example:

 $ hdfs dfs -mv /fibrevillage/redhat-release /fibrevillage/hadoop 

Remove a file or directory in HDFS.

Remove files specified as argument. Deletes directory only when it is empty

Usage :

hdfs dfs -rm <arg>

Example:

hdfs fs -rm /fibrevillage/test1 

Recursive version of delete in HDFS.

Usage :

hdfs dfs -rmdir <arg> 

Example:

hdfs dfs -rmdir /fibrevillage/hadoop

Display last few lines of a file in HDFS.

Similar to tail command in Linux.
Usage :

hdfs dfs -tail <path[filename]> 

Example:

hdfs dfs -tail /fibrevillage/fstab 

Display space usage of a file/directory in HDFS. 

Similar to du command on Linux. Displays size of files and directories contained in the given directory or the size of a file if its just a file.

Usage :

hdfs dfs -du <path> 

Example: 

$ hdfs dfs -du -h /fibrevillage/
4.3 K    /fibrevillage/fstab
323.6 M  /fibrevillage/hadoop
16.7 M   /fibrevillage/usr

Change file mode/permissions in HDFS

Similar to chmod command on Linux

It affects the permissions of the folder or file. Controls who has read/write/execute privileges
Example:

hdfs dfs -chmod 666 -R /fibrevillage

Find Out HDFS filesystem usage

Similar to df command on Linux

$ hdfs dfs -df -h
Filesystem              Size    Used  Available  Use%
hdfs://<namenode>:9000  119.2 T  29.6 G    119.2 T    0%

Search a file in HDFS

Similar to find command

Usage:

hdfs dfs -find <path> <expression>

Example:

$ hdfs dfs -find /fibrevillage -name fstab 
/fibrevillage/fstab

Merge files in HDFS

hdfs dfs -getmerge

Takes a source directory file or files as input and concatenates files in src into the local destination file.
Concatenates files in the same directory or from multiple directories as long as we specify their location and outputs them to the local file system, as can be seen in the Usage below.

Usage:

$ hdfs dfs -getmerge <src> <localdst> [addnl]
$ hdfs dfs -getmerge <src1> <src2> <localdst> [addnl]

Option:

$addnl: can be set to enable adding a newline on end of each file

Example:

hdfs dfs -getmerge /fibrevillage/test1 /fibrevillage/test2 /home/hadoop/testfile_merged

Note: test1 and test2 can be directories or files.

Use Help Command to access Hadoop Command Manual

Help command opens the list of commands supported by Hadoop Data File System (HDFS)

Example:  

hdfs dfs  -help

Hope this short tutorial was useful to get the basics of file management.