VBoxManage crashes with heap corruption using `guestcontrol run`

[AaronC81]

I recently upgraded VirtualBox and am encountering an issue with the guestcontrol run command of VBoxManage.

Most of the time, the command does not appear to run the process on the guest. (Maybe it is running but not showing me the output, I am not sure.) Instead, VBoxManage fails with exit code -1073740940, which is STATUS_HEAP_CORRUPTION.

> VBoxManage.exe guestcontrol %VM% run --username vboxuser --password changeme -- "C:\Windows\System32\cmd.exe" /c "echo hello"
hello

> echo %errorlevel%
-1073740940

A small amount of the time, the process executes successfully with the expected errorlevel. It seems to be random which occurs, as sometimes the same command will pass or fail between successive attempts.

Debugging

If I use Windows SDK gflags to enable page heap monitoring, and attach WinDbg, I get a heap verifier stop:

===========================================================
VERIFIER STOP 000000000000000F: pid 0x4494: corrupted suffix pattern 

	000002A4F7691000 : Heap handle
	000002A481D32FE0 : Heap block
	0000000000000018 : Block size
	000002A481D32FF8 : corruption address
===========================================================
This verifier stop is not continuable. Process will be terminated 
when you use the `go' debugger command.
===========================================================

The last VBoxManage frame before the stop was VBoxManage + 0xbd04, which called OLEAUT32!SafeArrayDestroy.

I am happy to debug this further if a debug development build, or symbols for the distributed VirtualBox, are available! This is VBoxManage 7.1.0r164728, SHA256 digest of VBoxManage.exe is 5FCFE99C9D231EAA3CC06CBA7AF51E440C065C0F37B08A4D10C69DD978EC284C.

(I tried building from source, but there are some version conflicts with tooling I need for work, so this is a bit tricky for me unfortunately.)

Environment

VirtualBox: I have been able to reproducible this on both VirtualBox 7.0.18 and VirtualBox 7.1.0. The issue is not present on VirtualBox 6.1.28.

Hosts: I have observed this on both of the hosts I have tested, one running Windows 10 and the other Windows 11.

Guests: I have observed this on existing Windows 10 and 11 guests which were originally created in an older VirtualBox version (5.2.16). I also tried creating a brand-new Windows 11 guest on 7.0.18, and the same issue occurred there.

Logs

I have attached the logs from a few attempts at running commands on my newer Windows 11 VM, most of which were unsuccessful.

[pentagonik]

Thanks for the report. Did 7.0.16 work for you in that regard?

[pentagonik]

Can you please try running the same "run" command using VBoxManage and adding "-vvvv" to it? This increases the verbosity and will tell us a little bit more where it appears to crash.

[AaronC81]

(Sorry for the delay, I thought I'd get an email if there were any replies!)

Replying to pentagonik:

Can you please try running the same "run" command using VBoxManage and adding "-vvvv" to it? This increases the verbosity and will tell us a little bit more where it appears to crash.

I tried this and re-ran many times.

I got this output most of the time - usually exiting with the heap corruption exit code, but sometimes exiting with success:

Executing:
  Image : C:\Windows\System32\cmd.exe
  arg[0]: C:\Windows\System32\cmd.exe
  arg[1]: /c
  arg[2]: echo hello
Creating guest session as user 'vboxuser'...
Waiting for guest session to start...
Successfully started guest session (ID 2)
Starting guest process ...
Process 'C:\Windows\System32\cmd.exe' (PID 6648) started
Wait result is 'started' (1)
Wait result is 'waiting flag not supported' (9)
Wait result is 'waiting flag not supported' (9)
Wait result is 'waiting flag not supported' (9)
Wait result is 'waiting flag not supported' (9)
Wait result is 'waiting flag not supported' (9)
hello
Wait result is 'waiting flag not supported' (9)
Wait result is 'terminated' (2)
Process terminated
Exit code=0 (Status=500 [successfully terminated])
Closing guest session ...

So it looks like it's executing to completion in most cases before erroring out.

However, occasionally I saw this output instead:

Executing:
  Image : C:\Windows\System32\cmd.exe
  arg[

Where I suppose it's failing much earlier.

---

Did 7.0.16 work for you in that regard?

I haven't tested on this version - my original choice of 7.0.18 was simply what happened to be the latest during the first attempt at updating VirtualBox.

I can test on this version and/or try to "bisect" the problematic version when I get some time :)