CODEWITHSUNDEEP
bash
blehblehblecksheep
blehblehblecksheep

Reputation: 285

Count the number of lines in a large number (100k+) of files

I tried using wc -l, but it says Argument list too long. Is there an alternative? The total number of files in the directory is 100k+ and there are too many lines per file as well (200k+ in each).

Upvotes: 0

Views: 92

Answers (1)

John Kugelman
John Kugelman

Reputation: 362037

If you want a separate count for each file, do:

find /dir -exec wc -l {} +

-exec + is smart and will call wc multiple times with partial file lists so as not to exceed the OS's command line length limit.

If you want one combined count for all files, do:

find /dir -exec cat {} + | wc -l

By concatenating all of the files together only one wc invocation is needed.

Upvotes: 5

Related Questions