VBoxService.exe crashes when executing a remote command from the host system

[Slava]

When executing a remote command like this

$ sudo -Hu builder vboxmanage --nologo guestcontrol Win7x64Build execute
"c:\python32\python.exe" --username "build" --password "superpuperpassword"
--wait-stdout -- "c:\lmsbuild\build.py"

sent from the host system VBoxService.exe crashes in 100% cases.

The crash occurs because VBoxService.exe executes a hard-coded breakpoint. It seems like it is because an assertion fails in D:\tinderbox\add-4.1\src\VBox\Additions\common\VBoxService\VBoxServiceControlExecThread.cpp, in the function VBoxServiceControlExecThreadGetOutput, supposedly line number 395.

The crash report with the memory dump is attached.

[Technologov]

1. Please attach vbox.log

2. Simple use cases work for me (as covered in my v4.1.6 ​Test Results)

Such as: (XP-on-Win7 host)

VBoxManage --nologo guestcontrol "Windows XP" execute --image "c:\windows\system32\ping.exe"

--username Administrator --password 123456 --wait-exit --wait-stdout -- 127.0.0.1 -n2

-Technologov

[Slava]

Replying to Technologov:

1. Please attach vbox.log

Attached.

2. Simple use cases work for me (as covered in my v4.1.6 ​Test Results)

I do not understand. Use cases work for you, ok. Probably it is some unfamiliar for me English idiom. English is not my first language. Can you please elaborate what actions from me do you expect saying that they work for you.

[Technologov]

Try simple command:

VBoxManage --nologo guestcontrol "[VM name]" execute --image "c:\windows\system32\ping.exe" --username Administrator --password 123456 --wait-exit --wait-stdout -- 127.0.0.1 -n2

Works ?

[Slava]

Yes, simple commands work.

Maybe it is the amount of output that matters, maybe it is a method python.exe uses to write stdout, maybe the combination of the two factors plus the method VBoxService.exe uses to read stdout from pipe.

Can you load the attached process dump into the debugger and check what is going on there (VS2010 supports it, if you don't use VS2010, use Windows Debugging Tools to load the process dump)? I pointed you to the exact line where an assertion fails and causes the process to end abnormally. I believe it is an assertion after the pipe read. Can you then compare how you read stdout in the version 3 of virtualbox? Version 3 worked fine.

[Slava]

Ping, ping, ping.

The assertion fails on executing the VBoxServicePipeBufIsEnabled function. As far as I can tell this function just checks some value in memory, obviously a flag that tells data was written to the pipe buffer. After some timeout if this flag is still unset VBoxServiceControlExecThreadGetOutput generates a debugger trap.

How do you fill the buffer and how do you set this flag?

Were you able to load the process dump into debugger and check what's going on?

What else help do you need?

[Slava]

Alright, I at last found the source code. The assertion fails after

VBoxServicePipeBufRead(pPipeBuf, pBuf, cbSize, &cbRead);

returns with cbRead equal to zero and pPipeBuf->fEnabled was true before that call. Can it be a race condition, i.e. the buffer was filled after you read fEnabled in

bool fEnabled = VBoxServicePipeBufIsEnabled(pPipeBuf);

and before you call

int rc = RTCritSectEnter(&pBuf->CritSect);

in the VBoxServicePipeBufRead function?

[pentagonik]

As of 4.1.10 there should be no more internal pipe buffering in VBoxService; does the problem still persist for you after upgrading the Guest Additions to 4.1.10?