Add option to pass through guest trim/UNMAP by punching holes in VDI file to free host disk space

[Donuts]

With VirtualBox 4.2, one of the new features was "Storage: implemented support for discarding unused image blocks through TRIM for SATA and IDE and UNMAP for SCSI when using VDI images"

If my thinking is correct, that means that for expanding VDI files, the guest OS tells VirtualBox (via trim/UNMAP) which regions of the virtual disk can be erased. Then when the guest next writes new data, VirtualBox knows it can re-use those regions of the VDI which were marked as freed-up, instead of increasing the size of the VDI file.

The benefit of that is, the rate of expansion of VDI files is reduced. There is another related feature which would be very useful.

Windows and recent Linux kernels support "punching holes" in files. Unused regions of the file can be made into holes, so the file becomes sparse. The new hole regions read back as all-zero. You can think of hole-punching as analogous to trim/UNMAP, but for files.

It would be great if an option could be added to pass through trim/UNMAP requests from the guest, so that VirtualBox punches holes in the VDI file. Benefits of that:

• When files are deleted in the guest, the VDI file would occupy less disk space on the host. Very useful if you're tight on disk space. The amount of space occupied by the VDI file would be roughly equal to the amount of space actually used in the guest. E.g. 100GB VDI, guest has used 50GB of that -> VDI occupies ~50GB on host, even though the apparent VDI file size may be larger. (Expanding VDI apparent file size will correspond approximately to the high water mark of disk usage in the guest.)

• VDI would always occupy the minimum amount of space on the host drive, without needing to do VBoxManage modifyhd --compact manually.

• If the user has a fixed-size VDI file but needs more disk space on the host, they could enable hole punching and thus free up approximately the amount of free space in the VDI, on the host drive.

[Georgi Chulkov]

Disclaimer: I am not a developer so I may be quite wrong.

The current implementation of discard actually does shrink the VDI file. Write a huge file to a TRIM-enabled guest and delete it - you will see that the VDI will (slowly) shrink to more or less the size it had before you started writing. In fact, I am not sure discard slows expansion in the way that you described.

So technically the feature you want - the ability to shrink a VDI after TRIMming data from it - is already there.

That said, it is quite slow. While I do not know exactly how it works, I have noticed that TRIM-initiated shrinking of a VDI is about as slow as actually writing the data being removed, so it can take a very long time if you delete a very large file. Try the Unix "sync" command in the guest immediately after deleting several gigabytes and watch the VDI shrink.

I do agree that it would be great if hole-punching was a faster way to do the same thing and VB could use it.

[jack lee]

The current implementation of discard on VDI file is not efficient, it has a limit is: "The size of the cleared area must be at least 1MB." That will waste many disk space.

When hole-punching is implemented, there will have no size limit.

[dsmogor]

Given the discard function is next to useless in it's current implementation (it causes regular gues os / vbox crashes) this proposal could be a great first step towards implementing this proposal: https://www.virtualbox.org/ticket/16795#comment:4