Xorg fails to start after upgrade from snv_101b to snv_111b

[Ken Brucker]

Seeing the following stack trace in Xorg.0.log (full log attached)

Backtrace:
0: /usr/X11/bin/i386/Xorg'xorg_backtrace+0x3c [0x80c266c]
1: /usr/X11/bin/i386/Xorg'xf86SigHandler+0x52 [0x80eeec2]
2: /lib/libc.so.1'__sighndlr+0xf [0xd12ed0af]
3: /lib/libc.so.1'call_user_handler+0x2af [0xd12e01af]
4: /usr/X11/lib/modules/drivers/vboxvideo_drv.so'0xef95 [0xce3b0f95]
5: /usr/X11/lib/modules/libshadowfb.so'ShadowPolyFillRect+0x276 [0xcbc04826]
6: /usr/X11/bin/i386/Xorg'miPaintWindow+0x1b7 [0x8140de7]
7: /usr/X11/bin/i386/Xorg'miWindowExposures+0xc7 [0x8140f77]
8: /usr/X11/bin/i386/Xorg'MapWindow+0x37f [0x80bb06f]
9: /usr/X11/bin/i386/Xorg'InitRootWindow+0x100 [0x80bb2c0]
10: /usr/X11/bin/i386/Xorg'main+0x419 [0x80ac7f9]
11: /usr/X11/bin/i386/Xorg'_start+0x80 [0x8084f40]

Fatal server error:
Caught signal 11.  Server aborting

Reinstalled vboxguest in the Solaris guest:

$ pkginfo -l SUNWvboxguest
   PKGINST:  SUNWvboxguest
      NAME:  Sun VirtualBox Guest Additions
  CATEGORY:  application
      ARCH:  i386
   VERSION:  2.2.2,REV=r46594.2009.04.27.20.14
   BASEDIR:  /
    VENDOR:  Sun Microsystems, Inc.
      DESC:  VirtualBox Guest Additions for Solaris guests
    PSTAMP:  vboxguest20090427201500_r46594
  INSTDATE:  May 20 2009 11:56
   HOTLINE:  Please contact your local service provider
     EMAIL:  [email protected]
    STATUS:  completely installed
     FILES:       45 installed pathnames
                   6 partially installed pathnames
                   2 directories
                  28 executables
               22730 blocks used (approx)

[Ken Brucker]

Xorg.0.log Logfile

[Juergen Keil]

This is a 32-bit opensolaris b111 guest, correct?

Is cpu hardware virtualization (VT-x / AMD-V) enabled or disabled?

A similar problem has been filed as an opensolaris defect, but this is starting to look as if it could be a virtualbox problem. ​http://defect.opensolaris.org/bz/show_bug.cgi?id=8993 (We seem to be getting a protection fault instead of an invalid page fault when accessing memory mapped video card memory)

[Sander van Leeuwen]

Please attach your VBox.log here as well.

[Juergen Keil]

Xorg crash, on host with AMD64 X2 cpu, AMD-V disabled, 16M video mem, 32-bit guest kernel

[Juergen Keil]

Xorg crash, on host with 32-bit Pentium-M cpu, no cpu hardware virtualization, 16M video mem, 32-bit guest kernel

[Juergen Keil]

To reproduce the issue I have to disable hardware cpu virtualization features, and configure 4MB or more video memory for the opensolaris b111 guest. With such a setup the Xorg server is segfaulting at startup quite often, but it isn't 100% reproducible every time.

So far I have not yet observed the Xorg segfault when VT-x or AMD-V is enabled; or (without cpu hardware virtualization) when less than 4 MB of video memory is configured.

It seems that with < 4MB of video memory, the Solaris kernel uses 4K pages to map the video memory; with >= 4MB video memory 4M pages are used. This may or may not be part of the problem.

[Ken Brucker]

Vbox Session Details

[Ken Brucker]

Vbox Runtime Info

[Ken Brucker]

Yes I'm running 32 bit and hardware virtualization is disabled. 16MB video memory configured.

I'm working on recreating right now. I am seeing some oddities though in pkg vs. pkginfo reporting on the system. I've attached session information and will also attach Vbox.log, Xorg.0.log from the session and pkginfo.txt with the pkginfo. pkg appears to display correct and pkginfo wrong data.

[Ken Brucker]

Vbox log file

[Ken Brucker]

Xorg.0.log matching Vbox.log session

[Ken Brucker]

pkg vs. pkginfo data for a few installed Xorg packages

[Ken Brucker]

Turning on HW Virtualization allows X to startup. Now auto-screen resize doesn't work. In previous release could resize VB window and guest display resized. Not working now. Submit a new ticket?

[Ken Brucker]

Previous release == snv_101b. Same version of Vbox (2.2.2)

[Juergen Keil]

Replying to seiryu:

Turning on HW Virtualization allows X to startup. Now auto-screen resize doesn't work. In previous release could resize VB window and guest display resized. Not working now. Submit a new ticket?

Did this guest perhaps had the VBox 2.1.4 guest additions installed at some point in the past, and you've upgraded to VBox 2.2.2 and pkgrm'ed the SUNWvboxguest package and reinstalled the newer version?

This could be bug #3882. There is a bug in the VBox 2.1.4 guest additions installer; it doesn't remove all files when you pkgrm SUNWvboxguest, and this confuses a pkgadd for the VBox 2.2.2 guest additions.

See also ​http://defect.opensolaris.org/bz/show_bug.cgi?id=8515#c29

[Ken Brucker]

Yes - This guest did have earlier Vbox guest additions installed but can't recall if they were from 2.1.4. I did have an odd behavior as I was trying to recreate this problem yesterday. When I first tried, I was getting version mismatch errors when the vbox drivers were loading at X startup. I recognized that as indicative that the 2.2.2 guest additions were not installed and it was using an older release. I did another round of pkgrm/pkgadd and just happened to also check that the /opt/VirtualBoxAdditions directory was empty after the pkgrm and all was good. I don't recall ever getting the noted errors during pkgadd.

I get the X core dump 100% reliably with VT-x disabled. I have latched on to #3656 regarding the auto-resize issue.

[Frank Mehnert]

Please reopen if still relevant with VBox 4.0.6.