CODEWITHSUNDEEP
cygwindd
La Hu
La Hu

Reputation: 85

Linux - dd command is failing with error [no space left on device]

I am currently trying to create an image of a USB key to create a backup. The USB key is 128G and the target External Hard drive has 300G available approximately.

I am using Cygwin and here is What I do so you understand :

I place myself on the External hard drive by typing : cd E: I end up under /cygdrive/e which is my HDD. The usb key is under/cygdrive/d

Here is what cat /proc/partitions gives :

$ cat /proc/partitions major minor #blocks name win-mounts

8     0 250059096 sda
8     1    266240 sda1
8     2     16384 sda2
8     3 248955904 sda3   C:\
8     4    819200 sda4
8    16 125091840 sdb
8    17 125091584 sdb1   D:\
8    32 312571224 sdc
8    33 312568641 sdc1   E:\

So here is the DD command I launch:

dd if=/dev/sdb1 | pv -s 128G | dd of=backup.usb.img bs=1M

It starts and progress well each time until it fails with the error :

dd: error writing ‘backup.usb.img’: No space left on device

I tried several times, and each time it fails after reading around 4,2G. My HDD has like 300G available so I don't understand...

Why is the dd command failing with a no space left whereas the target has plenty of space available ?

Thanks a lot

Upvotes: 1

Views: 2949

Answers (2)

user208145
user208145

Reputation: 379

Regarding what "@that other guy" mentioned with respect to splitting the file into 4G chunks so it would fit on a FAT32 disk, I had to do something like that earlier this year.

I needed to view the contents of a Mac OSX 2280 SSD, but couldn't on the non-Mac workstation I was on. The only Mac I had at my disposal was behind lock and key, i.e. no physical USB to plug in the adapter. My solution was to dd the disk image piped into ssh, sending it to the Mac, and mount it like an image file.

This should've worked:

dd if=${InputDev} bs=1M status=progress | ssh user@${DESTHOST} "dd bs=1M of=${OUTPUTFILENAME}"

The problem I ran into is that something between the two computers caused some network problems, so I would have to restart the entire clone operation. I ended up sending 10 or 100 MB at a time, then whichever chunk failed, I resent manually. I used something like this.

# For demonstration purposes, this example splits ${InputDev}, a 1 GiB file,
#   into 10 100 MiB pieces, plus an 11th piece with the remaining data.
# Note that the last couple chunks result in empty files at $DESTHOST 
#   because it tries to skip past EOF on the input side.
# When chunking with ssh, it's best to use an SSH key for the transfer.

for chunk in $(seq -w 0 12)
# "-w" in seq above ensures leading zeroes to keep files in order
do
    echo "Sending chunk #${chunk} of ${InputDev}"
    dd \
        if=${InputDev} \
        bs=100M \
        skip=${chunk} \
        count=1 \
        status=progress | \
            ssh root@${DESTHOST} "dd bs=100M of=/tmp/DevSdx_chunk#_${chunk}.dd"
        echo ""
done

For your scenario, where you can plug in a USB drive on the same machine, you wouldn't need the pipe into ssh portion. You would add of=/media/MyUSB/DevSdx_chunk#_${chunk}.dd to the dd command.

After the operations are finished, you can compare the source and destination checksums to verify integrity.

# Source side
sha256sum ${InputDev}
5f9c534c0753deb7188ae645928165d16f5e7378c1257532faa1bcb17a69740d  ${InputDev}

# Destination side
# if your output files are out of lexicographical order, the checksum won't match
cat /media/MyUSB/DevSdx_chunk*.dd | sha256sum # if local
cat /tmp/DevSdx_chunk*.dd | sha256sum # using the ssh example

If the checksums match on both ends, you can reassemble the whole image on the destination side with

# Destination side
cat /media/MyUSB/DevSdx_chunk*.dd > /media/FileSystemThatAllowsLargerFiles/DevSdx.dd

Upvotes: 0

La Hu
La Hu

Reputation: 85

Indeed, reply from "that other guy" solved my issue.

I formatted my DD into exFat to it can be used on MacOS as well, and the DD command went fine.

Also thanks "Matzeri", because I also removed the pv in the middle. We can use : status=progress It just needs sys-apps/coreutils >= 8.24

Thanks !

Upvotes: 1

Related Questions