Reputation: 85
I am currently trying to create an image of a USB key to create a backup. The USB key is 128G and the target External Hard drive has 300G available approximately.
I am using Cygwin and here is What I do so you understand :
I place myself on the External hard drive by typing : cd E: I end up under /cygdrive/e which is my HDD. The usb key is under/cygdrive/d
Here is what cat /proc/partitions gives :
$ cat /proc/partitions major minor #blocks name win-mounts
8 0 250059096 sda
8 1 266240 sda1
8 2 16384 sda2
8 3 248955904 sda3 C:\
8 4 819200 sda4
8 16 125091840 sdb
8 17 125091584 sdb1 D:\
8 32 312571224 sdc
8 33 312568641 sdc1 E:\
So here is the DD command I launch:
dd if=/dev/sdb1 | pv -s 128G | dd of=backup.usb.img bs=1M
It starts and progress well each time until it fails with the error :
dd: error writing ‘backup.usb.img’: No space left on device
I tried several times, and each time it fails after reading around 4,2G. My HDD has like 300G available so I don't understand...
Why is the dd command failing with a no space left whereas the target has plenty of space available ?
Thanks a lot
Upvotes: 1
Views: 2949
Reputation: 379
Regarding what "@that other guy" mentioned with respect to splitting the file into 4G chunks so it would fit on a FAT32 disk, I had to do something like that earlier this year.
I needed to view the contents of a Mac OSX 2280 SSD, but couldn't on the non-Mac workstation I was on. The only Mac I had at my disposal was behind lock and key, i.e. no physical USB to plug in the adapter. My solution was to dd
the disk image piped into ssh
, sending it to the Mac, and mount it like an image file.
This should've worked:
dd if=${InputDev} bs=1M status=progress | ssh user@${DESTHOST} "dd bs=1M of=${OUTPUTFILENAME}"
The problem I ran into is that something between the two computers caused some network problems, so I would have to restart the entire clone operation. I ended up sending 10 or 100 MB at a time, then whichever chunk failed, I resent manually. I used something like this.
# For demonstration purposes, this example splits ${InputDev}, a 1 GiB file,
# into 10 100 MiB pieces, plus an 11th piece with the remaining data.
# Note that the last couple chunks result in empty files at $DESTHOST
# because it tries to skip past EOF on the input side.
# When chunking with ssh, it's best to use an SSH key for the transfer.
for chunk in $(seq -w 0 12)
# "-w" in seq above ensures leading zeroes to keep files in order
do
echo "Sending chunk #${chunk} of ${InputDev}"
dd \
if=${InputDev} \
bs=100M \
skip=${chunk} \
count=1 \
status=progress | \
ssh root@${DESTHOST} "dd bs=100M of=/tmp/DevSdx_chunk#_${chunk}.dd"
echo ""
done
For your scenario, where you can plug in a USB drive on the same machine, you wouldn't need the pipe into ssh
portion. You would add of=/media/MyUSB/DevSdx_chunk#_${chunk}.dd
to the dd
command.
After the operations are finished, you can compare the source and destination checksums to verify integrity.
# Source side
sha256sum ${InputDev}
5f9c534c0753deb7188ae645928165d16f5e7378c1257532faa1bcb17a69740d ${InputDev}
# Destination side
# if your output files are out of lexicographical order, the checksum won't match
cat /media/MyUSB/DevSdx_chunk*.dd | sha256sum # if local
cat /tmp/DevSdx_chunk*.dd | sha256sum # using the ssh example
If the checksums match on both ends, you can reassemble the whole image on the destination side with
# Destination side
cat /media/MyUSB/DevSdx_chunk*.dd > /media/FileSystemThatAllowsLargerFiles/DevSdx.dd
Upvotes: 0
Reputation: 85
Indeed, reply from "that other guy" solved my issue.
I formatted my DD into exFat to it can be used on MacOS as well, and the DD command went fine.
Also thanks "Matzeri", because I also removed the pv in the middle. We can use : status=progress It just needs sys-apps/coreutils >= 8.24
Thanks !
Upvotes: 1