VBoxSVC crashes/hangs when saving machine config through VirtualBox web service (WS)

[thedisbo1125]

Problem traced to the "<RemoteDisplay>" section missing in the XML settings file of the VM.

If I create a VM using the VirtualBox GUI, but do not go back and enable “Remote Display”, the settings are not added to the XML. Any attempt to modify the VM settings using the WS causes VboxSVC issues. See the attached coredump from the syslog.

Further testing shows “<VRDEProperties>” is also required along with at least one property. I didn’t test all properties, but found if either of the following are present, saving the settings through WS works as expected:

<Property name="Security/Method" value="RDP"/> <Property name="Security/ServerCertificate" value="/home/rfabian/VirtualBox Vms/test/VRDEAutoGeneratedCert.pem"/>

I also tested without the Extension Pack installed, and encountered the same problem, “<RemoteDisplay>” and properties must be there or VboxSVC has issue.

[thedisbo1125]

coredump from syslog

[thedisbo1125]

After searching through the source code, appears the issue is being caused by a variable not being initialized.

In file:

src/VBox/Main/src-server/VRDEServerImpl.cpp

function:

HRESULT VRDEServer::setEnabled(BOOL aEnabled)

If "RemoteDisplay" is not set in the XML file, it crashes on the following line

"if (mData->fEnabled != RT_BOOL(aEnabled))"

[paulson]

Thanks for the report. Out of curiosity how are you interacting with the VirtualBox web service? To help reproduce the issue and bolster our testing what commands are you running? Are you enabling the VRDE server directly or is the VRDEServer::setEnabled() call happening as a consequence of something else? I've given things a quick look so far and tried some VM configuration changes using the web service and haven't run into this yet. Thanks again.

[thedisbo1125]

I've been doing more testing, compiling source code and adding entries to syslog. Currently I have traced it to when I lock the machine to make changes.

If I use findMachine (php):

$machine = $this->vbox->findMachine($argsid);

not an issue.

$machine->VRDEServer->enabled $machine->VRDEServer->getVRDEProperty('TCP/Ports')

work without issue.

When I try and lockMachine to make changes

$machine->lockMachine($this->session, ($vmRunning ? 'Shared' : 'Write'));

using the session->machine causes the problem.

Testing with both "Oracle VirtualBox Manager" (OVM) client and web service. OVM doesn't appear to have this issue.

I'm not too familiar with the source code yet so taking a little time to trace the issue back.

Currently tracked back to function in MachineImpl.cpp:

HRESULT SessionMachine::init(Machine *aMachine)

call

mVRDEServer->init(this, aMachine->mVRDEServer);

seems aMachine->mVRDEServer hasn't been initialized when web service makes the call.

see attached modified code checking if mData is null...

[thedisbo1125]

code changes to "init" in VRDEServerImpl.cpp

[thedisbo1125]

going to have to dig into my code more. I modified your php sample to make the change and it works without RemoteDisplay in the XML file. Just odd that everything I have works fine if RemoteDisplay is there, and only has issues when it isn't. Tested commenting out everything dealing with VRDEServer and still had the issue when I went to savesettings. Also thought maybe because I using HTTPS, nope, your sample also works fine.

[paulson]

Still, if VBoxSVC crashed then something isn't working as it should on the VirtualBox side. Can you share the PHP commands which trigger the failure for you?

[thedisbo1125]

So I tracked down what is causing the problem. I was adding the ability to change the PKI certificate used for VRDE. If "Security/ServerCertificate" and "Security/ServerPrivateKey" are not set in the XML file, attempting to read them causes a problem in VBoxSVC. This shows up when I attempt to either write any setting to VRDEServer or just save settings without changing VRDEServer settings. I have attached a modified version of the PHP sample provided with web services. I verified trying to read "Security/CACertificate" without existing does not cause the problem. If the settings already exist, everything functions as expected.

Also, tracked the failing line to:

• VRDEServerImpl.cpp

function

• HRESULT VRDEServer::init(Machine *aParent, VRDEServer *aThat)

call

• mData.share(aThat->mData);

doesn't work properly after attempting to read the two cert settings.

[thedisbo1125]

getting access denied trying to attach file so here is the php code:

<?php

require_once('./vboxServiceWrappers.php');

Connect to webservice $connection = new SoapClient("vboxwebService.wsdl", array(

'location' => "​http://127.0.0.1:18083/",

'connection_timeout' => 20,

'cache_wsdl' => WSDL_CACHE_BOTH));

Logon to webservice $websessionManager = new IWebsessionManager($connection); Dummy username and password (change to appropriate values or set authentication method to null) $virtualbox = $websessionManager->logon("username","password");

$machine = $virtualbox->findMachine("test"); $test = $machine->VRDEServer->getVRDEProperty('Security/ServerCertificate');

$session = $websessionManager->getSessionObject($virtualbox->handle); $machine->lockMachine($session->handle, "Write");

if($session->machine->VRDEServer) {

$session->machine->VRDEServer->enabled = true;

$session->machine->saveSettings(); }

$session->unlockMachine();

$machine->releaseRemote(); $session->releaseRemote();

$websessionManager->logoff($virtualbox->handle);

?>

[paulson]

Thanks for the sample code. The root cause is an oversight in the new VRDE certificate generation code which was introduced in VirtualBox 7.1. The issue manifests itself when IVRDEServer::getVRDEProperty() is called with either the 'Security/ServerCertificate' or the 'Security/ServerPrivateKey' property before calling IMachine::lockMachine() for a VM which does not have VRDP enabled. If the the VRDE server is enabled (e.g $sessionMachine->getVRDEServer()->setEnabled("true");) and one of the above properties are queried (e.g. $sessionMachine->getVRDEServer()->getVRDEProperty('Security/ServerCertificate');) in the same login session to 'vboxwebsrv' then trouble will still be encountered in a subsequent call to IMachine::lockMachine(). In short, the best workaround option is to avoid querying these two VRDE properties if the VM does not have VRDE enabled (and VRDE wasn't was just enabled in the same 'vboxwebsrv' login sesion. Apologies for this. A fix is in progress.

[Klaus Espenlaub]

This is fixed in the 7.1 test builds now.

[thedisbo1125]

Ok, I'll test it out. Thanks again.

[thedisbo1125]

Issue is resolved.