CODEWITHSUNDEEP
linuxbashshell
Nic Hubbard
Nic Hubbard

Reputation: 42139

Shell script to count files, then remove oldest files

I am new to shell scripting, so I need some help here. I have a directory that fills up with backups. If I have more than 10 backup files, I would like to remove the oldest files, so that the 10 newest backup files are the only ones that are left.

So far, I know how to count the files, which seems easy enough, but how do I then remove the oldest files, if the count is over 10?

if [ls /backups | wc -l > 10]
    then
        echo "More than 10"
fi

Upvotes: 61

Views: 68991

Answers (11)

Morgan
Morgan

Reputation: 14

Based on others suggestions and some awk foo, I got this to work. I know this an old thread, but I didn't find a decent answer here and this sorted it for me. This just deletes the oldest file, but you can change the head -n 1 to 10 and get the oldest 10.

find $DIR -type f -printf '%T+ %p\n' | sort | head -n 1 | awk '{first =$1; $1 =""; print $0}' | xargs -d '\n' rm

Upvotes: 0

indiana
indiana

Reputation: 21

I just found this topic and the solution from mikecolley helped me in a first step. As I needed a solution for a single line homematic (raspberrymatic) script, I ran into a problem that this command only gave me the fileames and not the whole path which is needed for "rm". My used CUxD Exec command can not start in a selected folder.

So here is my solution:

ls -A1t $(find /media/usb0/backup/ -type f -name homematic-raspi*.sbk) | tail -n +11 | xargs rm

Explaining:

  • find /media/usb0/backup/ -type f -name homematic-raspi*.sbk searching only files -type f whiche are named like -name homematic-raspi*.sbk (case sensitive) or use -iname (case insensitive) in folder /media/usb0/backup/
  • ls -A1t $(...) list the files given by find without files starting with "." or ".." -A sorted by mtime -t and with a return of only one column -1
  • tail -n +11 return of only the last 10 -n +11 lines for following rm
  • xargs rm and finally remove the raiming files in the list

Maybe this helps others from longer searching and makes the solution more flexible.

Upvotes: 2

miron
miron

Reputation: 1450

On a very limited chroot environment, we had only a couple of programs available to achieve what was initially asked. We solved it that way:

MIN_FILES=5
FILE_COUNT=$(ls -l | grep -c ^d )


if [ $MIN_FILES -lt $FILE_COUNT  ]; then
  while [ $MIN_FILES -lt $FILE_COUNT ]; do
    FILE_COUNT=$[$FILE_COUNT-1]
    FILE_TO_DEL=$(ls -t | tail -n1)
    # be careful with this one
    rm -rf "$FILE_TO_DEL"
  done
fi

Explanation:

  • FILE_COUNT=$(ls -l | grep -c ^d ) counts all files in the current folder. Instead of grep we could use also wc -l but wc was not installed on that host.
  • FILE_COUNT=$[$FILE_COUNT-1] update the current $FILE_COUNT
  • FILE_TO_DEL=$(ls -t | tail -n1) Save the oldest file name in the $FILE_TO_DEL variable. tail -n1 returns the last element in the list.

Upvotes: 1

mahyard
mahyard

Reputation: 1238

find is the common tool for this kind of task :

find ./my_dir -mtime +10 -type f -delete

EXPLANATIONS

  • ./my_dir your directory (replace with your own)
  • -mtime +10 older than 10 days
  • -type f only files
  • -delete no surprise. Remove it to test your find filter before executing the whole command

And take care that ./my_dir exists to avoid bad surprises !

Upvotes: 32

John Andrea
John Andrea

Reputation: 21

Straightforward file counter:

max=12
n=0
ls -1t *.dat |
while read file; do
    n=$((n+1))
    if [[ $n -gt $max ]]; then
        rm -f "$file"
    fi
done

Upvotes: 2

mikecolley
mikecolley

Reputation: 149

Make sure your pwd is the correct directory to delete the files then(assuming only regular characters in the filename):

ls -A1t | tail -n +11 | xargs rm

keeps the newest 10 files. I use this with camera program 'motion' to keep the most recent frame grab files. Thanks to all proceeding answers because you showed me how to do it.

Upvotes: 14

Daniel Haley
Daniel Haley

Reputation: 52858

I like the answers from @Dennis Williamson and @Dale Hagglund. (+1 to each)

Here's another way to do it using find (with the -newer test) that is similar to what you started with.

This was done in bash on cygwin...

if [[ $(ls /backups | wc -l) > 10 ]]
then
  find /backups ! -newer $(ls -t | sed '11!d') -exec rm {} \;
fi

Upvotes: 3

bta
bta

Reputation: 45057

stat -c "%Y %n" * | sort -rn | head -n +10 | \
        cut -d ' ' -f 1 --complement | xargs -d '\n' rm

Breakdown: Get last-modified times for each file (in the format "time filename"), sort them from oldest to newest, keep all but the last ten entries, and then keep all but the first field (keep only the filename portion).

Edit: Using cut instead of awk since the latter is not always available

Edit 2: Now handles filenames with spaces

Upvotes: 1

bashfu
bashfu

Reputation: 1

Using inode numbers via stat & find command (to avoid pesky-chars-in-file-name issues):

stat -f "%m %i" * | sort -rn -k 1,1 | tail -n +11 | cut -d " " -f 2 | \
   xargs -n 1 -I '{}' find "$(pwd)" -type f -inum '{}' -print

#stat -f "%m %i" * | sort -rn -k 1,1 | tail -n +11 | cut -d " " -f 2 | \
#   xargs -n 1 -I '{}' find "$(pwd)" -type f -inum '{}' -delete

Upvotes: -2

Dennis Williamson
Dennis Williamson

Reputation: 360095

The proper way to do this type of thing is with logrotate.

Upvotes: 10

Dale Hagglund
Dale Hagglund

Reputation: 16440

Try this:

ls -t | sed -e '1,10d' | xargs -d '\n' rm

This should handle all characters (except newlines) in a file name.

What's going on here?

  • ls -t lists all files in the current directory in decreasing order of modification time. Ie, the most recently modified files are first, one file name per line.
  • sed -e '1,10d' deletes the first 10 lines, ie, the 10 newest files. I use this instead of tail because I can never remember whether I need tail -n +10 or tail -n +11.
  • xargs -d '\n' rm collects each input line (without the terminating newline) and passes each line as an argument to rm.

As with anything of this sort, please experiment in a safe place.

Upvotes: 104

Related Questions