slow-ass SATA transfer rate
Here I’m writing a big file full of zeros that I will, on my next step, mark as deleted. All the space that was taken up with that huge file will still exist, but as Gigabytes and Gigabytes of free space filled with zeros. It should all gzip down to a compact archive. I’m using the Linux command line to “Ghost” or “clone” an exact image of my laptop as a baseline for incremental backups. I had no idea I’d need to leave the thing all day though.
You can probably do almost any device I/O thing in the world with dd, but you have to be careful. Despite what anyone might tell you, dd stands for “destroy data”. Screw up the odd syntax and you’ll copy the factory fresh formatted partition all over top of your backup file.
root@StandardMischief:~# time dd if=/dev/zero of=/media/Preload/zerofile2 bs=1024 count=58970800
58970800+0 records in
58970800+0 records out
60386099200 bytes (60 GB) copied, 44091 s, 1.4 MB/s
.
real   734m51.051s
user   2m8.116s
sys   19m59.531s
root@StandardMischief:~#
Look at that speed! The SATA was running at a staggering 0.0109375 Gigabits/second, which is about 137 times slower than the max rate of 1.5 Gigabits/second for first generation SATA . Something is way wrong here.
So it took 12+ hours to create the file and only a second to erase it. I’m pretty sure at least part of the issue here is that I used a ridiculously small block size of 1 KB and did that 58,970,800 times in a row. We’ll see, as next run will use half of my RAM as a block size.
Update: Compression on the fly, with a larger block size
root@StandardMischief:~# time dd if=/dev/sdb1 conv=sync,noerror bs=512K | gzip -c > /home/backup/lappy486-20090905.img.gz
138767+1 records in
138768+0 records out
72754397184 bytes (73 GB) copied, 4765.18 s, 15.3 MB/s
.
real 79m25.198s
user 52m54.734s
sys 21m10.651s
root@StandardMischief:~#
So 15.3 (MB / s) = 0.11953125 Gb / s, which is both much faster and still a less than a tenth of advertised speeds. If this was my broadband connection I’d be on the phone to customer support by now.
I see I did have an error for the second run. I used a blocksize of half a MB instead of half a GB ( bs=512K instead of bs=512M). Well that’s something to try for next week.
