snapshot to a remote iSCSI volume (builtin initiator) broken since 4.3.0 (verified on Windows and Mac hosts)

[Alban]

Hello,

I used to have workstations running local VirtualBox VMs which base disk are connected to the same remote iSCSI read-only lun. It use to work because each VM also have a default snapshot to the base disk meaning disk writes are redirected to the local differential disk created by the snapshot.
This worked correctly from version 4.0 to latest 4.2.20 and seems broken starting from 4.3.0, including latest 4.3.6.
I could verify this on a Mac host (OSX 10.9) and a Windows host (Windows 7 Prof.). I expect it to be the same on a Linux Host but didn't have a chance to verify.

How to reproduce:
machine 1: iSCSI server (I used ietd on SLES). IP address 192.168.0.19
machine 2: VirtualBox host (Windows or OSX using Virtualbox 4.3.6). IP address 192.168.0.23

1. machine 1:
   • create an iSCSI target and a read-write 20GB Lun
     target: iqn.1995-09.fr.univ-lr:storage.win7
     lun: 2

2. machine 2:
   • create a VB VM 'win7' without a hard disk
   • attach the iSCSI volume from machine 1 with VBoxManage:

     VBoxManage storageattach "win7" --storagectl IDE --port 0 --device 1
       --type hdd --medium iscsi --mtype normal --server "192.168.0.19"
       --target "iqn.1995-09.fr.univ-lr:storage.win7" --lun 2
     

   • install the guest OS (Windows 7 Pro.)
   • start the guest OS once after install to make sure booting read-write from the remote target works fine
   • poweroff the VM and create a snapshot

3. machine 1:
   • make the previous Lun 2 read-only

4. machine 2:
   • start the VM

With VirtualBox 4.3.6, the VM hangs just after starting and the associated host process is using 100% CPU time. The diffential disk never grows and there isn't any iSCSI network traffic on machine 1. Unfortunately, nothing interresting in the VM log file (Windows host or Mac host).

With VirtualBox 4.2.20, the guest OS starts and differential disk of the snapshot starts to grow while block are written

If I downgrade from 4.3.6 to 4.2.20 everything works fine again. So it's clearly related to something changed starting from 4.3.0 Note, I also tried with immutable disk wich avoid creating a snapshot manually and makes the VM revert to its pristine state at each start (which is the desired behaviour) and the results are the same.

Thanks for you help and ready to provide any additional details as needed.

Happy new year

Alban

[Frank Mehnert]

We still would like to see the VBox.log files for the VM session.

[celticnestor]

Hi!

I am able to reproduce this issue on a VirtualBox host running v4.3.12, iSCSI target running on an external NAS box and VM running Ubuntu 14.04.

How to reproduce:

• Network config
  1. machine 1: iSCSI (Thecus N7510), IP address 192.168.1.13
  2. machine 2: virtualBox host (Windows8), IP address 192.168.1.60
• Create an iSCSI target on machine 1 Target: 192.168.1.13|iqn.2014-07.com.thecus.n7510:iscsi.thecussan.raid0|1
• Attach created iSCSI target to VM using command: VBoxManage storageattach VM --storagectl SCSI --port 0 --device 0 --type hdd --medium iscsi --server 192.168.1.13 --target iqn.2014-07.com.thecus.n7510:iscsi.thecussan.raid0 --lun 1
• Install Ubuntu 14.04 on VM over iSCSI volume
• Start the guest OS once after install to make sure booting read-write from the remote target works fine
• Power-off the VM and create a snapshot
• Start the VM

Result: VM hangs and does not boot-up

Attached is the VBox.log generated when VM boots-up after snapshot is created.

Looking forward to hearing your comments.

[aeichner]

Please reopen if still relevant with a recent VirtualBox release.

[Nikolay Kichukov]

Hello, This is still happening on:

app-emulation/virtualbox-5.1.10-r1:0

app-emulation/virtualbox-additions-5.1.10:0

app-emulation/virtualbox-extpack-oracle-5.1.10.112026:0

app-emulation/virtualbox-modules-5.1.10:0

Host is running Gentoo Linux with kernel 4.4.26-gentoo.

The symptoms are the same as the initial description. Once the snapshot on an iscsi disk is made the VM (Windows 7) can no longer boot. One of the CPU Cores is stuck at 100%. The snapshot file is created, but never grows:

260K -rw------- 1 nikolay nikolay 2.0M Dec 14 11:04 {1ca62904-69c2-4890-bd88-b82f7467ae3e}.vdi

The disk was attached to the SATA controller of the VM.

Thank you. -Nikolay

[aeichner]

Please attach the VBox.log of the VM.

[Nikolay Kichukov]

VBox.log - iscsi disk with snapshot