Git bash `cp` – Exclude "protected operating system files" (and what are those?)

Question

If I try to copy my home directory using cp in Git bash (on Windows 7), more than I want will be copied. The following is an example to demonstrate the issue. In reality I want to copy recursively, but I don't in this example to speed things up.

Simon@SIMONS-DATOR ~
$ mkdir /c/test

Simon@SIMONS-DATOR ~
$ cp ./* /c/test
cp: omitting directory `./AppData'
cp: omitting directory `./Contacts'
cp: omitting directory `./Cookies'
cp: omitting directory `./Desktop'
cp: omitting directory `./Documents'
cp: omitting directory `./Downloads'
cp: omitting directory `./Dropbox'
cp: omitting directory `./Favorites'
cp: omitting directory `./GodMode.{ED7BA470-8E54-465E-825C-99712043E01C}'
cp: omitting directory `./Links'
cp: omitting directory `./Lokala inst\303\244llningar'
cp: omitting directory `./Mallar'
cp: omitting directory `./Mina dokument'
cp: omitting directory `./Music'
cp: `./NTUSER.DAT.exe' and `/c/test/NTUSER.DAT' are the same file
cp: omitting directory `./N\303\244tverket'
cp: omitting directory `./Pictures'
cp: omitting directory `./Programdata'
cp: omitting directory `./Recent'
cp: omitting directory `./Saved Games'
cp: omitting directory `./Searches'
cp: omitting directory `./SendTo'
cp: omitting directory `./Skrivare'
cp: omitting directory `./Start-meny'
cp: omitting directory `./Videos'
cp: `./ntuser.dat.LOG1.exe' and `/c/test/ntuser.dat.LOG1' are the same file
cp: `./ntuser.dat.LOG2.exe' and `/c/test/ntuser.dat.LOG2' are the same file

Simon@SIMONS-DATOR ~
$ ls /c/test
NTUSER.DAT{016888bd-6c6f-11de-8d1d-001e0bcde3ec}.TM.blf
NTUSER.DAT{016888bd-6c6f-11de-8d1d-001e0bcde3ec}.TMContainer00000000000000000001.regtrans-ms
NTUSER.DAT{016888bd-6c6f-11de-8d1d-001e0bcde3ec}.TMContainer00000000000000000002.regtrans-ms
NTUSER.DAT{6a7ede06-4c18-11e0-86c3-00262d9124f7}.TM.blf
NTUSER.DAT{6a7ede06-4c18-11e0-86c3-00262d9124f7}.TMContainer00000000000000000001.regtrans-ms
NTUSER.DAT{6a7ede06-4c18-11e0-86c3-00262d9124f7}.TMContainer00000000000000000002.regtrans-ms
NTUSER.DAT{8ef111c8-e816-11df-8bd5-00262d9124f7}.TM.blf
NTUSER.DAT{8ef111c8-e816-11df-8bd5-00262d9124f7}.TMContainer00000000000000000001.regtrans-ms
NTUSER.DAT{8ef111c8-e816-11df-8bd5-00262d9124f7}.TMContainer00000000000000000002.regtrans-ms
NTUSER.DAT{e9e59e5b-940d-11e0-8662-00262d9124f7}.TM.blf
NTUSER.DAT{e9e59e5b-940d-11e0-8662-00262d9124f7}.TMContainer00000000000000000001.regtrans-ms
NTUSER.DAT{e9e59e5b-940d-11e0-8662-00262d9124f7}.TMContainer00000000000000000002.regtrans-ms
_.swp
_viminfo
ntuser.ini
php_errors.log

I want none of those ntuser files to be copied. And if I copy recursively, I do not want the following folders to be copied (translation attempts to the right):

~/Contacts
~/Cookies
~/Lokala inställningar    (Local settings)
~/Mallar                  (Templates)
~/Mina dokument           (My documents)
~/Nätverket               (The network)
~/Programdata
~/Recent
~/SendTo
~/Skrivare                (Printers)
~/Start-meny              (Start menu)

If I do try to copy recursivley, all of the folders mentioned above will be created at the destination, but they will remain empty since permission to them seems to be denied:

cp: cannot access `/c/Users/Simon/Cookies': Permission denied
cp: cannot access `/c/Users/Simon/Documents/Min musik': Permission denied
cp: cannot access `/c/Users/Simon/Documents/Mina bilder': Permission denied
cp: cannot access `/c/Users/Simon/Documents/Mina videoklipp': Permission denied
cp: cannot access `/c/Users/Simon/Lokala inst\303\244llningar': Permission denied
cp: cannot access `/c/Users/Simon/Mallar': Permission denied
cp: cannot access `/c/Users/Simon/Mina dokument': Permission denied
`/c/Users/Simon/NTUSER.DAT' -> `/e/Backup/Simon/NTUSER.DAT'
cp: `/c/Users/Simon/NTUSER.DAT.exe' and `/e/Backup/Simon/NTUSER.DAT' are the same file
cp: cannot access `/c/Users/Simon/N\303\244tverket': Permission denied
cp: cannot access `/c/Users/Simon/Programdata': Permission denied
cp: cannot access `/c/Users/Simon/Recent': Permission denied
cp: cannot access `/c/Users/Simon/SendTo': Permission denied
cp: cannot access `/c/Users/Simon/Skrivare': Permission denied
cp: cannot access `/c/Users/Simon/Start-meny': Permission denied
`/c/Users/Simon/ntuser.dat.LOG1' -> `/e/Backup/Simon/ntuser.dat.LOG1'
cp: `/c/Users/Simon/ntuser.dat.LOG1.exe' and `/e/Backup/Simon/ntuser.dat.LOG1' are the same file
`/c/Users/Simon/ntuser.dat.LOG2' -> `/e/Backup/Simon/ntuser.dat.LOG2'
cp: `/c/Users/Simon/ntuser.dat.LOG2.exe' and `/e/Backup/Simon/ntuser.dat.LOG2' are the same file

As seen from the above log, there seems to be additional unwanted folders in ~/Documents (translation attempts to the right):

./Min musik          (My music)
./Mina bilder        (My pictures)
./Mina videoklipp    (My video clips)

Anyhow, since the permission is denied, I don't suffer too much from this problem: There's just going to be a few extra unnecessary empty folders and some extra unnecessary files in my destination folder, and some "permission denied" complaining from cp in the log.

Before expermimenting with the cp command I didn't even know about these files. Apparently, one must uncheck "Hide protected operating system files (Recommend)" in the Folder Options, in order for them to show up in Windows Explorer, and then, when they do, they have that little arrow on them, just like shortcuts do, and they are semi-transparent, as hidden files are (like ~/AppData).

The real usage is for a small backup script, which uses cp $src "$dstdir" --update --archive --backup=numbered --verbose to do the actual copying, with shopt -s extglob; shopt -s dotglob and src="/c/Users/Simon/!(AppData)" (notice that --arcgive implies recursive copying), if any of that is of interest. Edit: It should be noted that what I want to achieve with mentioned backup script, is to backup most files in my user folder (most importantly my documents, pictures and videos, but also for example .ssh and .gitconfig which happen to be there (unless I follow the advice from VonC's answer)). I just happen to use Git bash, since I already had it installed, and it offered a very simple way of doing the backup I want. It has nothing to do with Git. Also see additional edit further down.

So how can I prevent the cp command from trying to copy these files? (Alternatively, should I use something else than cp? (Vague question, I know, but it is not the main one, and I can elaborate if needed.))

Originally a sub question: And what are these files? (As far as I remember, many of them exists as more "normal" folders on Windows XP.) Edit: They probably are leftovers from Windows XP, at least Local Settings is.

Edit: For the purpose of backing up files, which seems to be my root problem, I now have plenty of solutions and options, thanks to VonC:

I have set %HOME% to /c/git, which is where I already kept all my repos, so all Git (bash) files are available in one place. Moreover, it eases the backup: Instead of backing up /c/Users/Simon/!(AppData) (as mentioned), I can point to more specific target easily:

"/c/Users/Simon/Documents/*"
"/c/Users/Simon/Music/*"
"/c/Users/Simon/Pictures/*"
"/c/Users/Simon/Videos/*"
"/c/Users/Simon/Desktop/*"

Therefore, I've avoided the problem mostly, there's still the unwanted folders in ~/Documents left (see above).

I now understand that a Windows specific tool, such as xcopy, xxcopy or robocopy, probably is better for backing up, but for now I prefer the simplicity of cp and not having to dive into Windows command line.

I hope these lengthy comments on backing up is useful to someone. I consider the "should I use something else than cp?" question fully answered and my main problem mostly solved.

Still, the "how can I prevent the cp command from trying to copy these/such files?" question remains unsolved (other than the "use a Windows specific tool" kind of answer) and could be relevant for many.

Answer 1

If scripting a backup is what you are after, it would be best to avoid cp, and use windows utilities, like xxcopy:
It can manage all kind of bazckup scenario, including incremental ones, with exclusion list.

---

Original git-centric answer

Both git-cmd.bat and git-bash wil ltry and use %USERPROFILE% if %HOME% is not defined.

I would recommend defining the environment variable %HOME% before calling git commands, in order for your HOME to contain only what you need to use git (.gitconfig, _netrc, .ssh, ...), and develop.

That way, you control exactly what you need to cp, rather than trying to sort out the vast mess that is /c/Users/aLoginName.