CODEWITHSUNDEEP
hadoopgrephdfs
oortcloud_domicile
oortcloud_domicile

Reputation: 850

How to search inside HDFS files

I want to be able to search for a pattern of string inside all the folders and files in specific directory in HDFS. not sure how to do it. below does not work.

hadoop fs -ls -R /folder_path/ |grep -i "pattern"

How do i search?.

Upvotes: 1

Views: 7421

Answers (1)

Jagrut Sharma
Jagrut Sharma

Reputation: 4754

For files within a single HDFS folder, you could use the hadoop fs -cat command. This will not search within sub-folders though.

To illustrate, if files are in a HDFS folder called /hive-data, the following command can be used to search within the files in the directory for a pattern:

hadoop fs -cat /hive-data/* | grep -i pattern

Upvotes: 2

Related Questions