Using serial host device make guest boot hang

[cderenne]

I'm using the serial host device since a long time. Using Fedora (from 20 now 30) host and XP host (because of sortware only running on Windows XP using serial device) Until VB 5.12, everything worked thing. But now, after update to Fedora 30, I had to update to the last VirtuaBox release (6.0.10). My guest does not boot anymore (it hangs) If i uncheck serial port, it boots. If I keep serial port and disable it, My guest Boots, serial COM is visible but unsuable (which is normal).

VBox.log is showing this : (see file attached)

Failed to change parameters to 9600,EVEN,7,1 -> VERR_DEV_IO_ERROR Failed to change parameters to 1200,EVEN,7,1 -> VERR_DEV_IO_ERROR

[cderenne]

XP guest boot hangs when serial is enable

[Socratis]

Replying to cderenne:

after update to Fedora 30, I had to update to the last VirtuaBox release (6.0.10)

Not really, Fedora30 is supported in the 5.2.x series. Can you try and downgrade to the latest, 5.2.32 for Fedora 29/30 (as of this writing), and see if it's a 6.0.x regression for certain?

[cderenne]

Hi, Thanks for your answer. Not downgraded (it's a server that is in use) but install a VirtualBox 5.22 on an other Fedora 30 PC. No trouble. Then i've upgrade to VB6 on this test workstation : Got the trouble (XP guest don't boot if serial is enable)

For information, i've installed same guest with VB6.010 on a Win 10 Host and everything is fine. So it seems that it's a spécific VB6 bug when using Linux Host

[vavi]

Same problem here, but only with one out of two Win7 VMs on the same host!!

Host is Linux (archlinux 64bit, patches up to date), guests Win7 64bit (all available updates applied).

Guests were installed, when VBox 5.x was the current release.

Starting with 6.0.0 one Win7-VM hung on boot if a serial device was connected to the Serial port, the other did not! As of now I'm von VBox 6.1.0 and the situation has not changed.

The hanging VM consumes 100% of one of the host CPUs until it gets quit.

I'm starting only one Win7-VM at the same time, because my laptop has only one physical serial port.

[aeichner]

Could you please attach a VBox.log of the VM which doesn't work AND the one which works? I want to check whether there are any differences which could explain why one hangs while the other doesn't. Do any of the Windows guests have anything configured to use the serial port (kernel debugging, etc.), what kind of serial hardware do you use on the host? I couldn't reproduce this issue locally so I need as much information as possible in order to try to pin down the cause.

[aeichner]

As I wasn't able to reproduce the issue so far we need some special logs from the serial port emulation. I prepared a testbuild which can be downloaded from here, look for the latest development testbuild (6.1.2+ state). In order to activate the tracing the VM needs some special config which can be added with

VBoxManage setextradata <VM name> "VBoxInternal/PDM/DriverTransformations/SerialTrace/AboveDriver"                         "Host Serial"
VBoxManage setextradata <VM name> "VBoxInternal/PDM/DriverTransformations/SerialTrace/AttachedDriver/Config/TraceFilePath" "<trace/log/file/path>"
VBoxManage setextradata <VM name> "VBoxInternal/PDM/DriverTransformations/SerialTrace/AttachedDriver/Driver"               "IfTrace"

<trace/log/file/path> needs to be substituted with a file path the user as write access to. <VM name> needs to be substituted with the VM name being traced. The current setup doesn't work if you have multiple serial ports configured which are using a host serial port device, so be sure to either disable all but one or leave them disconnected. Depending on the size the resulting logs can either be attached to this defect (they don't contain any data being transfered but only metadata about timings and parameters of certain function calls) or can be sent directly to Alexander (dot) Eichner (at) oracle (dot) com.

[vavi]

Alexander, I've sent you an email containing some log files and a README.txt explaining shortly what I did an what happend.