NTFS Sparse Files For Programmers
Posted by jpluimers on 2024/09/25
Need to check this out some day: cs.exe compiled from [Wayback] sparse.zip which you can download from [Wayback/Archive] NTFS Sparse Files For Programmers
Downloads
All the content is provided on the “as is” basis and without any warranty, express or implied. You can use the supplied tools and examples for any commercial or non-commercial purpose without charge. You can copy and redistribute these tools and examples freely, provided that you distribute the original umodified archives.
sparse.zip – Visual C++ source code for a sparse file enabled Copy Stream utility. As the original utility, it performs stream-to-stream copy, however if the source stream is sparse, the target stream will also be sparse.
The reason is that (especially virtual Machine hard-disk image) files on Windows can be sparse files that only occupy the size of the portions that actually are in use.
When you copy them without the sparse attribute, then the copied file will not be sparse and potentially take up way more disk space than anticipated.
I thought this happened when copying (using Robocopy) a bunch of Paragon HDM version 17 disk image backups from a 750 gigabyte drive to a 1 terabyte drive. It failed despite the total size of files being some 560 gigabytes.
After copying some 500 gigabytes of content, the target drive was full?!
In the end, this is a problem in Windows: the target drive had file compression enabled. When that is the case then Windows keeps twice the file size locked until you reboot.
So after a reboot, the 1 terabyte drive had about 500 gigabyte free space again.
Before realising that, I had already started searching how to copy sparse files on Windows with these results:
- [Wayback/Archive] robocopy sparse files – Google Search
- [Wayback/Archive] copy|move sparse files on NTFS with Windows – Super User (thanks [Wayback/Archive] Michael Yasumoto for asking and [Wayback/Archive] Yahor for answering) which pointed me to the above download:
You cannot do it with standard Windows tools.But there is an utility. Go to http://www.flexhex.com/docs/articles/sparse-files.phtml, download and unpack sparse.zip.Then run from from the command line:cs.exe c:\src.dat d:\dest.datThe file will be copied preserving sparseness.There are some other convoluted answers too, but this one seems to be the easiest.
Somehow none of the other results showed me how to do this with Robocopy, so I assume it does not understand sparse files.
–jeroen
The Wiert Corner - irregular stream of stuff 
Leave a comment