CODEWITHSUNDEEP
hadoopvim
Scorpio
Scorpio

Reputation: 511

Why doesn't hadoop allows to use VIM to edit files?

Why doesn't hadoop allows to use VIM to edit files. I want to edit modify files using an editor using HDFS command line.

Upvotes: 4

Views: 3517

Answers (2)

Uri Goren
Uri Goren

Reputation: 13700

you can copy this 5 line script and them use

hvim <your filename>

And the script code:

hadoop fs -text $1>hvim.txt
vim hvim.txt
hadoop fs -rm -skipTrash $1
hadoop fs -copyFromLocal hvim.txt $1
rm hvim.txt

Upvotes: 5

Amal G Jose
Amal G Jose

Reputation: 2546

Hadoop's file system-HDFS is write once, read many file system. We cannot edit any file in HDFS, we can do append if required. This distributed file system is designed for handling huge data. VIM, vi, nano, gedit etc are unix file editors and can be used only for editing files in Unix file system. HDFS is basically is virtual file system and we access the file system using the HDFS URI with HDFS commands. When we read a file in the hdfs location /user/user1, the actual URI is hdfs://namenode:port/user/user1. We cannot use normal unix file system commands to access the hadoop file system. We have hadoop file system commands and related api to access the hadoop file system. The hadoop's file system is stored in the disks of the underlying unix server, but it has separate metadata and blocks stored according to the HDFS architecture.

HDFS file system architecture and format is completely different as compared to unix file system. Since the system is designed to handle huge data, the edit option is not added to avoid the overhead of editing huge files. If you read about the architecture of HDFS and the way the files are stored, you will get a clear idea about why the option to edit files are not present in hdfs.

Appending something to a file has less overhead as it will not involve the change of the complete file and the file metadata. But modifying something in a file is a huge overhead as it involves the complete modification of the data as well as the related metadata.

Upvotes: 3

Related Questions