Windows 10 Guest login on Fedora 36 crashes after mouse movement

[Stephen Rondeau]

Linux cn1-vcl2 5.18.15-200.fc36.x86_64 #1 SMP PREEMPT_DYNAMIC Sun Jul 31 21:30:34 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

VirtualBox 6.1.36 with 6.1.36 extension pack installed

Windows 10 is 21H2, with Microsoft updates up to June 2022.

Linux display manager: gdm

Have been using a script for over 10 years to start the VM and provide it a video hint for a 1920x1080 screen about a minute after starting. The way it is started:

VBoxSDL --startvm win10 --fullscreen --nohostkey

Prior to Fedora 36, we were on Fedora 34 and running VB 6.1.34 with no issues. After upgrading to Fedora 36 and installing VB 6.1.36, we noticed that the Windows 10 VMs previously running could not be logged into, since they crash immediately upon user input, and the script restarts the VM again.

There is nothing in the VBox.log file to indicate what happened. It may be related to the random changes in Linux 5.17.11+, per:

​https://lore.kernel.org/lkml/[email protected]/

and the fixed 20914 ticket https://www.virtualbox.org/ticket/20914

but the Linux 5.18 patches are in 6.1.36.

I have found a workaround for it -- use the VMSVGA graphics controller for the Windows 10 VM. The VBoxVGA and the VBoxSVGA graphics controllers are sensitive to mouse movements. The guest additions can be 6.1.34 or 6.1.36 -- either work with VMSVGA. I had also increased the VRAM to 256 (MB), but feel that is not part of the solution and did not test without it, as I had changed VRAM to 256 before without the graphicscontroller change and it failed, using VBoxVGA.

VBoxManage modifyvm win10 --graphicscontroller VMSVGA --vram 256

before my VBoxSDL command now fixes the problem, but VMSVGA's possible performance problem and its lack of ability to handle video hints via:

VBoxManage controlvm win10 setvideomodehint 1280 1024 32

make it less desirable.

[Stephen Rondeau]

.vbox file with original VBoxVGA graphics controller

[Stephen Rondeau]

.vbox file with original VMSVGA graphics controller

[galitsyn]

Hi srondeau,

Could you please attach VBox.log?

[Stephen Rondeau]

.log file with vboxvga graphics controller enabled, just prior to problem

[Stephen Rondeau]

The newly-uploaded VBox.log file does not change after the problem occurs -- no new lines added. The VM simply restarts.

[Dmitrii Grigorev]

Hi srondeau. Do you have any special reason to ignore the default VBOXSVGA adapter for Windows 10 guest as described here ? https://www.virtualbox.org/manual/ch03.html#settings-display

Could you check if your issue is reproduced with VBOXSVGA and, if it is so, attach VBox.log here.

[Stephen Rondeau]

Replying to DmitrG:

Hi srondeau. Do you have any special reason to ignore the default VBOXSVGA adapter for Windows 10 guest as described here ? https://www.virtualbox.org/manual/ch03.html#settings-display

No. Just 10+ years of using vboxvga without any real problems until now. I might change my default creation scripts to use vboxsvga in the future.

Could you check if your issue is reproduced with VBOXSVGA and, if it is so, attach VBox.log here.

I did, and reported that it failed as well in the initial ticket. The issue is that both vboxvga and vboxsvga fail, but vmsvga works.

[Stephen Rondeau]

.log file with vboxSvga graphics controller enabled, just prior to problem

[Dmitrii Grigorev]

The SDL frontend is not actively supported now.

Could you try the following command line instead:

/usr/lib/virtualbox/VirtualBoxVM --startvm win10 --fullscreen --nohostkey

[Stephen Rondeau]

Replying to DmitrG:

The SDL frontend is not actively supported now.

That's not good. I rely on it heavily.

/usr/lib/virtualbox/VirtualBoxVM --startvm win10 --fullscreen --nohostkey

I did that with vboxsvga as graphicscontroller, but ran into several problems:

1. I can see the VirtualBoxVM process running, but the status using:

VBoxManage showvminfo win10 | grep State

shows "powered off". I manage everything via VBoxManage.

VBoxManage list runningvms

is an empty list.

2. I am working remotely. I don't know if the problem is the VM is not truly started, but VRDP is definitely not working. I cannot tell if there is a VM screen showing on the local computer monitor.

3. The VBox.log file is not updated at all, so I have no way of knowing anything about the VM other than the VirtualBoxVM process is running.

[Stephen Rondeau]

Actually, when I start it outside my script, here is the message I get:

Qt WARNING: could not connect to display Qt FATAL: This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, wayland-egl, wayland, wayland-xcomposite-egl, wayland-xcomposite-glx, xcb.

Aborted (core dumped)

Sorry -- forgot to place this inside a GNOME terminal window. ssh doesn't have a GUI, and that's where I issued the command this time (the other one was from within GNOME).

[galitsyn]

Hi srondeau,

In order to start, VirtualBoxVM frontend requires connection to X Server. Make sure that X is running and DISPLAY environment variable is set in a shell session. Btw,if you are running VMs on remote, you might also consider VBoxHeadless frontend (and access it via remote desktop connection).

[Stephen Rondeau]

Replying to galitsyn:

In order to start, VirtualBoxVM frontend requires connection to X Server.

Yes, I realized the error of my ways. I know about VBoxHeadless, but the VM needs to work both with a head for local workstation display and via VRDP for occasional remote access.

I have switched to using xrdp for debugging so I can gain access to the gnome terminal window.

The original process hang was due to an unseen information dialog box asking if it is okay to switch to fullscreen. I will need to turn that off via some setting prior to starting -- do you know that offhand? I tried adding "--no-startvm-errormsgbox" to the command, but the info dialog box still showed up.

DmitrG was correct: it must be the unsupported VBoxSDL that is causing the problem, as VirtualBoxVM works using either vboxvga, vboxsvga, or vmsvga graphics controllers. Thanks for the help!

[Stephen Rondeau]

To answer my own question about turning off that informational message, prior to VirtualBoxVM:

VBoxManage setextradata global GUI/SuppressMessages confirmGoingFullscreen

It appears that VirtualBoxVM accepts any options, even those it doesn't understand, such as:

--nohostkey

The equivalent functionality for "nohostkey" would be to set this prior to starting the VM:

VBoxManage setextradata global GUI/Input/HostKeyCombination 0