VirtualBox crashes in RDP extension

[Thomas Törnblom]

Occasionally VirtualBox crashes in what appears to be the RDP extension when handling audio. This kills the guest.

Stack trace from affected thread: ---

t@9 (l@9) terminated by signal SEGV (no mapping at the fault address)
Current function is rgnMergeAdjacentRows
dbx: warning: can't find file
"/mnt/tinderbox/extpacks-4.1/src/VBox/RDP/server/regions.cpp"
dbx: warning: see `help finding-files'
(dbx) where
current thread: t@9
=>[1] rgnMergeAdjacentRows(prgn = 0x1221c30), line 822 in "regions.cpp"
[2] shadowBufferGetAction(pAction = 0xfffffd7ffe23de90), line 4168 in
"shadowbuffer.cpp"
[3] VRDPServer::OutputThread(this = 0x95ebb0, self = (nil), pCtx =
0xffffffff), line 392 in "output.cpp"
[4] rtThreadMain(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xfffffd7ff9affc5c
[5] rtThreadNativeMain(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at
0xfffffd7ff9b4cd51
[6] _thrp_setup(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xfffffd7fff0b1331
[7] _lwp_start(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xfffffd7fff0b15d0

---

[Thomas Törnblom]

log

[Thomas Törnblom]

I notice there is a size limit of 256k for attachments, so I can't attach the core dump, which is 156M uncompressed, and 22M gzipped.

Contact me with details if you want the core.

[Thomas Törnblom]

I turned of RDP, and it still crashes with issues in audio. Not in RDP now though; ---

(dbx) where
current thread: t@8
=>[1] clip_natural_int16_t_from_stereo(0x216a42c, 0x2b57000, 0xfff2ca02, 0x0, 0x216a42a, 0x64656c69), at 0xfffffd7ff6969c17 
  [2] solaudio_run_out(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xfffffd7ff6996c8c 
  [3] filteraudio_run_out(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xfffffd7ff696b647 
  [4] audio_timer_helper(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xfffffd7ff6967d21 
  [5] tmR3TimerQueueRun(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xfffffd7ff8bcfaee 
  [6] TMR3TimerQueuesDo(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xfffffd7ff8bcfc16 
  [7] vmR3HaltGlobal1Halt(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xfffffd7ff8bd9110 
  [8] VMR3WaitHalted(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xfffffd7ff8bd97df 
  [9] EMR3ExecuteVM(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xfffffd7ff8c4884f 
  [10] vmR3EmulationThreadWithId(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xfffffd7ff8bd834f 
  [11] rtThreadMain(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xfffffd7ffe91fc5c 
  [12] rtThreadNativeMain(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xfffffd7ffe96cd51 
  [13] _thrp_setup(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xfffffd7ffefe0c99 
  [14] _lwp_start(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xfffffd7ffefe0f40 
(dbx)

My screen is a Sun Ray 2, and audio uses the "libc_ut.so" shared library, which catches writes to /dev/audio.

[Frank Mehnert]

Still relevant with VBox 4.1.22 / 4.2.0?

[Thomas Törnblom]

I have not seen this issue since several months. I hope the issue is gone.

[Frank Mehnert]

Thanks for that information!

[Thomas Törnblom]

Happened again today, with vbox 4.1.8, on s12_06, with an s12_06 VM: --- t@11 (l@11) terminated by signal SEGV (no mapping at the fault address) 0xffff80ffb6b0a0b7: clip_natural_int16_t_from_stereo+0x0007: movq (%rsi),%rax (dbx) where current thread: t@11 =>[1] clip_natural_int16_t_from_stereo(0x2fd404c, 0x5663000, 0xffcc9be3, 0x0, 0x2fd404a, 0x64656c69), at 0xffff80ffb6b0a0b7

[2] solaudio_run_out(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xffff80ffb6b373dc [3] filteraudio_run_out(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xffff80ffb6b0baf7 [4] audio_timer_helper(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xffff80ffb6b081c1 [5] tmR3TimerQueueRun(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xffff80ffb94afb7e [6] TMR3TimerQueuesDo(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xffff80ffb94afca6 [7] emR3ForcedActions(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xffff80ffb9527a08 [8] emR3HwAccExecute(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xffff80ffb952ac9d [9] EMR3ExecuteVM(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xffff80ffb9528550 [10] vmR3EmulationThreadWithId(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xffff80ffb94b83ef [11] rtThreadMain(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xffff80ffbeeafc1c [12] rtThreadNativeMain(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xffff80ffbeefcd11 [13] _thrp_setup(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xffff80ffbf5519a1 [14] _lwp_start(0x0, 0x0, 0x0, 0x0, 0x0, 0x0), at 0xffff80ffbf551c40