Virtual machines crash when started with VirtualBox

[weinreis]

Everytime we use VirtualBox to start virtual machines and put some load on them, the instances crash after a few minutes.
We tried different combinations of host (FC7, Ubuntu 7.04 Server) and guest systems (FC2, FC7, Win2000) and the OSE and closed source edition. No Difference.

When we use VBoxSDL to start the virtual machines they run fine (tested 6 instances >12h with high load)
So it has to be a Problem in VirtualBox. See attached file for some output VBoxSVC throwed.

If you need more information, I will be happy to provide you with the needed.

[weinreis]

Output from VBoxSVC after crash

[Frank Mehnert]

Do you have any chance to test VirtualBox (the Qt GUI) of the current SVN head?

[weinreis]

Oh, I forgot to mention. On Ubuntu 7.04 I also tried the SVN-Head (from yesterday).

[Sander van Leeuwen]

Can you give us a backtrace with a debug build (svn head)? Define the environment variable VBOX_ASSERT=0 to prevent assertions from triggering a breakpoint. Thanks.

[weinreis]

Backtrace after WIN2000 guest crash

[weinreis]

I added a backtrace. Hope it helps

[Frank Mehnert]

Please could you start VirtualBox like this:

 export VBOX_ASSERT=0
 gdb VBoxSVC

and in another terminal

 export VBOX_ASSERT=0
 gdb VirtualBox

and then start your VM session again. When VBox crashes, please dump the backtraces of all threads:

 info threads
 thread 1
 bt
 thread 2
 bt
 ...

And what did you exactly do to reproduce the crash? I have Win2k as guest here on a Ubuntu 7.04/i386 host. If you could find a pattern how to reproduce your problem as quick as possible that would be very helpful.

[Frank Mehnert]

And which guest additions do you have installed, the latest?

[weinreis]

Added backtraces from VBoxSVC.

Again it was the Win2000 vm which crashed, but the other vm also crash.

What we do to provoke the crashes is put some load on the machines. In Win2000 it's simply by starting pinball, the linux instances run a small shell script which in a loop echoes $RANDOM and writes 10 bytes from /dev/zero into a file.

With three or four guest that produces a load of 4 on the host. From that point the machines usually crash after a few minutes.

As I remember on the Windows vm we installed the guest addition from the 1.5 binaries, the linux vm run without additions.

[Frank Mehnert]

Thank you very much for these additional information. However, I doubt that VBoxSVC really crashes but an instance of VirtualBox which is started by VBoxSVC. If you have some more time could you do the following:

export VBOX_ASSERT=0
gdb VBoxSVC

and then

export VBOX_ASSERT=0
gdb -args VirtualBox -startvm <your vm name>

This usage of VirtualBox should immediately start the VM session (a little bit like VBoxSDL). Note -startvm not --startvm. Could you try to provoke a crash again and dump the backtraces of all threads? This time VirtualBox should crash, not VBoxSVC.

[weinreis]

Well that's exactly what happens. The instances crash one after the other.
VirtualBox itself is running fine.
BTW, that's exactly what I said.

Nevertheless, it's a big problem.
It's one of my major tasks to find stable virtualization software, which should be open source and does not need processor extentions. And to be honest Xen sks, and VMWare is closed source.
So I will be happy to help you trace bugs.
Added new backtraces. Win2k instances, crashed short after starting Pinball.

[Frank Mehnert]

Believe me, we trying hard to find the reason for the crashes you observe. My request was meant to get more information. Your previous backtraces from 10/10 showed only the VBoxSVC side. Your last backtraces show that actually VirtualBox (the Qt GUI) crashes (as I presumed). Unfortunately, even the new backtraces don't contain much more information. Thread 1 crashed but there is no usable backtrace. I'm strongly assume that your problem is related to #628.

The major problem here is that we cannot reproduce these crashes ourself. So either the bug hides on our systems or a library of your system is buggy.

Unfortunately, using debugging tools to find the problem is quite difficult. valgrind gives tons of false positives. Out builtin efence (see src/VBox/Runtime/r3/alloc-ef.cpp) does not work together with the Qt GUI because there are simply too much allocations.

[weinreis]

No offense meant. I'm pretty sure you do.
I'm running VirtualBox on a plain Ubuntu 7.04 Server. I installed the latest updates with apt-get dist-upgrade and VirtualBox exactly as described in the build instructions.
The hardware platform is a Fujitsu Siemens PRIMERGY P250, with two Xeon Processors

processor       : 0
vendor_id       : GenuineIntel
cpu family      : 15
model           : 2
model name      : Intel(R) Xeon(TM) CPU 2.60GHz
stepping        : 7
cpu MHz         : 2595.357
cache size      : 512 KB
physical id     : 3
siblings        : 2
core id         : 0
cpu cores       : 1
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 2
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov                                                                       pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe cid
bogomips        : 5193.33
clflush size    : 64

The win2k instance is a plain install with SP4.

I'm no developer and I'm pretty sure it's hard work to debug. If you can tell me what to do, I will be happy to do so.

Thanks

[Sander van Leeuwen]

Please try again with 1.5.2.

[Frank Mehnert]

No response for month and we think we fixed this bug. Closing.