CODEWITHSUNDEEP
bashunixshellscripting
trobrock
trobrock

Reputation: 47387

Copy folder recursively, excluding some folders

I am trying to write a simple bash script that will copy the entire contents of a folder including hidden files and folders into another folder, but I want to exclude certain specific folders. How could I achieve this?

Upvotes: 230

Views: 160176

Answers (10)

Panin Maxim
Panin Maxim

Reputation: 1

If there is no way to use rsync, you can use this (if you are in the directory being copied):

cp -R $(ls -a | grep -v '\.$' | grep -v exclude_directory) directory_to_copy

Upvotes: 0

Jack
Jack

Reputation: 427

Quick Start

Run:

rsync -av --exclude='path1/in/source' --exclude='path2/in/source' [source]/ [destination]

Notes

  • -avr will create a new directory named [destination].
  • source and source/ create different results:
    • source — copy the contents of source into destination.
    • source/ — copy the folder source into destination.
  • To exclude many files:
    • --exclude-from=FILE — FILE is the name of a file containing other files or directories to exclude.
  • --exclude may also contain wildcards:
    • e.g. --exclude=*/.svn*

Modified from: https://stackoverflow.com/a/2194500/749232

Example

Starting folder structure:

.
├── destination
└── source
    ├── fileToCopy.rtf
    └── fileToExclude.rtf

Run:

rsync -av --exclude='fileToCopy.rtf' source/ destination

Ending folder structure:

.
├── destination
│   └── fileToExclude.rtf
└── source
    ├── fileToCopy.rtf
    └── fileToExclude.rtf

Upvotes: 8

Johannes
Johannes

Reputation: 3099

Simple solution (but I would still prefer the bash pattern matching from the top comments):

touch /path/to/target/.git
cp -n -ax * /path/to/target/
rm /path/to/target/.git

This exploits the -n option of cp, which forces cp to not overwrite existing targets.

Drawback: Works with GNU cp. If you don't have GNU cp, then the cp operation might return an error code (1), which is annoying because then you can't tell if it was a real failure.

Upvotes: 2

go2null
go2null

Reputation: 2308

inspired by @SteveLazaridis's answer, which would fail, here is a POSIX shell function - just copy and paste into a file named cpx in yout $PATH and make it executible (chmod a+x cpr). [Source is now maintained in my GitLab.

#!/bin/sh

# usage: cpx [-n|--dry-run] "from_path" "to_path" "newline_separated_exclude_list"
# limitations: only excludes from "from_path", not it's subdirectories

cpx() {
# run in subshell to avoid collisions
  (_CopyWithExclude "$@")
}

_CopyWithExclude() {
  case "$1" in
    -n|--dry-run) { DryRun='echo'; shift; } ;;
  esac

  from="$1"
  to="$2"
  exclude="$3"

  $DryRun mkdir -p "$to"

  if [ -z "$exclude" ]; then
      cp "$from" "$to"
      return
  fi

  ls -A1 "$from" \
    | while IFS= read -r f; do
        unset excluded
        if [ -n "$exclude" ]; then
          for x in $(printf "$exclude"); do
          if [ "$f" = "$x" ]; then
              excluded=1
              break
          fi
          done
        fi
        f="${f#$from/}"
        if [ -z "$excluded" ]; then
          $DryRun cp -R "$f" "$to"
        else
          [ -n "$DryRun" ] && echo "skip '$f'"
        fi
      done
}

# Do not execute if being sourced
[ "${0#*cpx}" != "$0" ] && cpx "$@"

Example usage

EXCLUDE="
.git
my_secret_stuff
"
cpr "$HOME/my_stuff" "/media/usb" "$EXCLUDE"

Upvotes: 1

Kaleb Pederson
Kaleb Pederson

Reputation: 46509

Use rsync:

rsync -av --exclude='path1/to/exclude' --exclude='path2/to/exclude' source destination

Note that using source and source/ are different. A trailing slash means to copy the contents of the folder source into destination. Without the trailing slash, it means copy the folder source into destination.

Alternatively, if you have lots of directories (or files) to exclude, you can use --exclude-from=FILE, where FILE is the name of a file containing files or directories to exclude.

--exclude may also contain wildcards, such as --exclude=*/.svn*

Upvotes: 384

Kyle Butt
Kyle Butt

Reputation: 9810

Use tar along with a pipe.

cd /source_directory
tar cf - --exclude=dir_to_exclude . | (cd /destination && tar xvf - )

You can even use this technique across ssh.

Upvotes: 56

Dennis Williamson
Dennis Williamson

Reputation: 360693

You can use find with the -prune option.

An example from man find:

       cd /source-dir
       find . -name .snapshot -prune -o \( \! -name *~ -print0 \)|
       cpio -pmd0 /dest-dir

       This command copies the contents of /source-dir to /dest-dir, but omits
       files  and directories named .snapshot (and anything in them).  It also
       omits files or directories whose name ends in ~,  but  not  their  con‐
       tents.  The construct -prune -o \( ... -print0 \) is quite common.  The
       idea here is that the expression before -prune matches things which are
       to  be  pruned.  However, the -prune action itself returns true, so the
       following -o ensures that the right hand side  is  evaluated  only  for
       those  directories  which didn't get pruned (the contents of the pruned
       directories are not even visited, so their  contents  are  irrelevant).
       The  expression on the right hand side of the -o is in parentheses only
       for clarity.  It emphasises that the -print0 action  takes  place  only
       for  things  that  didn't  have  -prune  applied  to them.  Because the
       default `and' condition between tests binds more tightly than -o,  this
       is  the  default anyway, but the parentheses help to show what is going
       on.

Upvotes: 13

ghostdog74
ghostdog74

Reputation: 343201

you can use tar, with --exclude option , and then untar it in destination. eg

cd /source_directory
tar cvf test.tar --exclude=dir_to_exclude *
mv test.tar /destination 
cd /destination  
tar xvf test.tar

see the man page of tar for more info

Upvotes: 4

Steve Lazaridis
Steve Lazaridis

Reputation: 2210

EXCLUDE="foo bar blah jah"                                                                             
DEST=$1

for i in *
do
    for x in $EXCLUDE
    do  
        if [ $x != $i ]; then
            cp -a $i $DEST
        fi  
    done
done

Untested...

Upvotes: 0

Matthew Flaschen
Matthew Flaschen

Reputation: 285077

Similar to Jeff's idea (untested):

find . -name * -print0 | grep -v "exclude" | xargs -0 -I {} cp -a {} destination/

Upvotes: 3

Related Questions