CODEWITHSUNDEEP
unix
NewUsr_stat
NewUsr_stat

Reputation: 2571

Cat command fails to concatenate files horizontally

I'm trying to concatenate horizontally around 1000 files composed by a single column each one, to one single file. Since all the files that have to be concatenated are named: myfile1.txt, myfile2.txt, myfile3.txt, ..., I used the following string: 

cat $myfile*.txt > myoutput.txt

Unfortunately, the files are concatenated vertically.

Can anyone help me please?

Upvotes: 0

Views: 1326

Answers (1)

Jonathan Leffler
Jonathan Leffler

Reputation: 753585

The cat command is supposed to concatenate files vertically; it would be broken if it did otherwise. (It would be possible to write a variant of cat with a non-standard option to direct it to change its behaviour and do horizontal concatenation, but why bother?).

The command designed to paste (concatenate) files horizontally is paste:

paste myfile*.txt

By default, the lines from each file will be separated by tabs; you can control that and various other aspects of the behaviour of paste with command line options.

If you have a thousand files and each line in each file contains, say, 10 characters, then your output file will have lines that are 10 KiB or more long. Be aware that POSIX does not mandate that utilities (commands) support such long lines, though the best will. (The minimum value for LINE_MAX is 2048.) GNU does require support for indefinitely long lines, but you should be careful to check that whatever you use to process the output of the paste command will work.

Indeed, with 1000 files, you might have to worry about the per-process file descriptor limit. Use ulimit -n (or ulimit -a) to find the limit on the number of files a single process can have open. If it is 256, you may need to increase it. You'll end up investigating 'hard limits' and 'soft limits' (ulimit -H -a and ulimit -S -a).

(Note that cat $myfile*.txt gives a different list of files from cat myfile*.txt unless the variable assignment myfile=myfile was done first.)

Upvotes: 5

Related Questions