Taking a snapshot can revert recent disk writes in the running VM

[Eric Siegerman]

Taking a snapshot with the VM running can revert recent disk writes in the VM. The snapshot itself does contain the changes, but the still-running VM does not. (Rather than trying to explain in this detail in running text, let me refer you to the The problem section below.)

The guest is System Rescue CD 2.0.1 (can be downloaded from ​http://www.sysresccd.org/). For this test, it suffices to boot from the CD image; one does not need to install a guest O/S in the VM.

To reproduce

Setup

This first part is setup, which should all go as expected:

1. Create a VM:
   1. System is Linux/Ubuntu
   2. Accept all further defaults
2. Attach systemrescuecd-x86-2.0.1.iso to the virtual CD-ROM
3. Boot, accepting all defaults
4. Use cfdisk to create a single 5-MB primary partition, /dev/sda1
5. Reboot (probably no longer necessary these days, but for the sake of eliminating extraneous variables, it can't hurt)
6. Take a snapshot, using Machine > Take Snapshot... menu item

The problem

Here's where things get interesting. I'm attaching a screen capture, annotated to show the point where I take the snapshot.

7. od -c /dev/sda1 to verify that it contains all '\0' bytes
8. echo hello >/dev/sda1 (It's not often do you do *that* on purpose :-) )
9. od -c /dev/sda1 again to verify that the "hello" was written
10. Take a second snapshot, again using Machine > Take Snapshot...
11. od -c /dev/sda1 yet again. It should still say "hello\n\0...", as it did at step (9), but in fact it's back to all '\0' bytes!

Further observations

1. The incorrect state survives a reboot:
   1. Reboot the VM from the virtual CD
   2. od -c /dev/sda1; still all '\0's

2. But the snapshot we just took has the correct data:
   1. Power off the VM, and restore it to the most recent snapshot (the second one, taken at step (10)
   2. od -c /dev/sda1; the "hello" is back where it should be

[Eric Siegerman]

Screen capture of the "The probem" section

[Eric Siegerman]

OK, this is even weirder:

20. Perform the steps described in The problem in the initial bug report. At this point, the bug has manifested itself; sda1 has surprisingly been reset to "\0\0\0\0\0\0\0\0"
21. Restore from the most recent snapshot (the one taken in step (10)).
22. Dump sda1; it contains "hello\n\0\0\0\0\0", as expected
23. echo world >/dev/sda1
24. Dump sda1; it contains "world\n\0\0\0\0\0", as expected
25. Take a snapshot
26. Dump sda1. It should contain "world\n\0\0\0\0\0", of course, but given what we've seen, I'd have expected it to erroneously contain "hello\n\0\0\0\0\0". But no! It's been reset all the way back to "\0\0\0\0\0\0\0\0"!

Another point:

From the observations so far, it's plausible that the partition is not being reverted, but just blindly filled with NUL bytes. One can test this by filling the partition with something else to start with. (You can fill it with "."s by inserting a step into the setup section:

5.1. perl -e 'print "." x 100000000' >/dev/sda1

(the large integer is the number of "."s to print. It has to be larger than the length of the partition but, since Perl builds a string that long, it needs to be small enough to avoid paging :-) )

Now, when the bug manifests (both times, i.e. at steps (11) and (26)), sda1 will get reset to all-"." instead of to all-"\0". That shows that it really is being reverted, not just clobbered.

[Frank Mehnert]

Thanks for the report and the testcase! Fixed in VBox 4.0.8.