CODEWITHSUNDEEP
bashshellcentosredhat
steveJ
steveJ

Reputation: 2441

Argument list too long for ls while moving files from one dir to other in bash shell

Below is the command I am using for moving files from dir a to dir b 

  ls /<someloc>/a/* | tail -2000 | xargs -I{} mv {} /<someloc>/b/
    -bash: /usr/bin/ls: Argument list too long

folder a has files in millions ..

Need your help to fix this please.

Upvotes: 0

Views: 1531

Answers (2)

pjh
pjh

Reputation: 8054

The ls in the code in the question does nothing useful. The glob (/<someloc>/a/*) produces a sorted list of files, and ls just copies it (after re-sorting it), if it works at all. See “Argument list too long”: How do I deal with it, without changing my command? for the reason why ls is failing.

One way to make the code work is to replace ls with printf:

printf '%s\n' /<someloc>/a/* | tail -2000 | xargs -I{} mv {} /<someloc>/b/

printf is a Bash builtin, so running it doesn't create a subprocess, and the "Argument list too long" problem doesn't occur.

This code will still fail if any of the files contains a newline character in its name. See the answer by kvantour for alternatives that are not vulnerable to this problem.

Upvotes: 1

kvantour
kvantour

Reputation: 26471

If the locations of both directories are on the same disk/partition and folder b is originally empty, you can do the following

$ rmdir /path/to/b
$ mv /other/path/to/a /path/to/b
$ mkdir /other/path/to/a

If folder b is not empty, then you can do something like this:

find /path/to/a/ -type f -exec mv -t /path/to/b {} +

If you just want to move 2000 files, you can do

find /path/to/a/ -type f -print | tail -2000 | xargs mv -t /path/to/b

But this can be problematic with some filenames. A cleaner way would be is to use -print0 of find, but the problem is that head and tail can't process those, so you have to use awk for this.

# first 2000 files (mimick head)
find /path/to/a -type f -print0        \
  | awk 'BEGIN{RS=ORS="\0"}(NR<=2000)' \
  | xargs -0 mv -t /path/to/b

# last 2000 files (mimick tail)
find /path/to/a -type f -print0        \
  | awk 'BEGIN{RS=ORS="\0"}{a[NR%2000]=$0}END{for(i=1;i<=2000;++i) print a[i]}' \
  | xargs -0 mv -t /path/to/b

Upvotes: 2

Related Questions