Serial port handshake problem

[mirekm]

Hi,

I try use serial port connected to physical port. Communication is OK, but handshake (RTS/CTS, DSR/DTR) signal are not transfered. In VirtualBox CTS and DSR are always active. Physical line RTS and DTR are always off independently of state in VirtualBox.

Host - WinXP Profesional Slave - WinXP Home

[Gregory Nowak]

I'm having the same problem, so I'm adding myself to this ticket.

​http://vbox.innotek.de/pipermail/vbox-users/2007-October/002350.html

[Zakaria]

I also having the same problem.

Host: Ubuntu 7.10
Guest: Win XP Pro

[Martin Moeller]

I seem to be having similar difficulty.

Modem attached to COM1 on physical machine, runnng Windows Vista. Guest set up with 'Host Device' COM1 mapped to COM1. Guest is Windows XP Professional. Let the guest find the ports automatically. I added COM2 as well, but it doesn't seem to really be there after all, though Vista lists it.

Trying to communicate with the modem at all meets with utter failure. I have yet to try different handshaking options, but 'None' doesn't seem to work...

There doesn't seem to be much information about how modems are supposed to be set up in VirtualBox and I know it is probably not that common, but in this case my father actually need it to work.

Let me know if I can provide some data that can help get this solved.

[Gregory Nowak]

I've just upgraded to 1.5.4, and noticed that this problem hasn't been fixed. Although it wasn't mentioned in the 1.5.4 changelog, I was hoping that the problem was fixed, but left out of the changelog by mistake, but this turns out not to be the case. I do however see 2 minor differences from what happened under 1.5.2. These are as follows:

1. If I attempt using a host serial port initially, there is no communication

with the device attached to the host serial port. If I then close vbox, open the port for example with hyperterminal under a winxp pro host, close hyperterminal, and run vbox again, there is communication with the device attached to the host port, until the next reboot of the host machine.

2. Under 1.5.2, when attempting to use window-eyes under a winxp guest (see

my first post), my speech synthesizer responded with "waiting on serial device", whenever window-eyes sent data to it, and the solution was to pull out the serial cable, let my speech synth say whatever window-eyes had sent to it, plug the cable back in, and so on. Under 1.5.4, my speech synth speaks what window-eyes sends to it without having to pull out the serial cable, but handshaking is still not taking place, as was the case in 1.5.2. Finally, if I attempt to configure my 3com 2976 pci modem, which is attached to com3 on the host as uart2 on any guest with vboxmanage modifyvm guest_name_goes_here -uart2 0x02f8 03 -uartmode2 COM3 I get an error, stating that the host port is invalid or empty. However, if I open hyperterminal, and make a connection to com3 directly, I can send at commands to the modem, and get back "ok" responses just fine, I can take the modem off-hook, see "ring" if the phone rings, and so on. Also, it doesn't matter if I type the port as COM3, or as com3, the error is still the same in either case.

I also can't help noting that there seems to have been no response to this ticket by the Innotek staff.

[asnow]

As of 1.5.5 (or possibly 1.5.6-I jumped from .4 to .6), host serial ports have completely broken. Everytime a port is attached to a physical serial port on the host, the error "host path is empty or null" is given. This happens on both linux (slackware 12.1 and Ubuntu 8.04) and windows XP Professional hosts, using Virtualbox 1.5.6, 1.6.0, and 1.6.2.

[bytten]

I'll add to this, please PLEASE unbreak the serial ports under Virtualbox?

I'm running KUBUNTU 8.04 (VBox 1.5.6-OSE) and need to use the serial port, which "kind of" works, on exit it reports "ioctl failed for serial host device "/dev/ttyS0" VERR_INVALID_POINTER, the device will not work properly."

This error pops up if I just start the VM, let Windows boot, then shut down Windows.

I need to use the emulator so that I can a particular app that in turn uses RAS to talk to the EXTERNAL modem attached to ttyS0 to send a file using ZModem (yes ZModem), and no there is no other way to send the file. The app in question will successfully dial the modem and perform the log-in, and then it all goes horribly wrong. The ZModem transfer gets to about 60% and VirtualBox becomes a VirtualBrick, non-responsive, requiring me to kill the process. I really REALLY don't want to repartition the drive and install Windows on the bare-iron... but it is looking like I might have no choice. Interestingly the app worked on an older version (can't remember which one) and an "upgrade" broke it. Sorry, I've rambled on here a bit... been beating my head against this wall for a few days now and am at the end of the line... really guys... fix the serial port problem... PLEASE!

[Michael Hora]

I also having the same problem.

Host: Ubuntu 8.04 Guest: Windows XP VirtualBox 2.0.4

[Lawrence Rust]

VirtualBox 2.0.6 also demonstrates this bug (host Ubuntu 8.04). Surely it can't be difficult to add an ioctl( TIOCMGET/TIOCMBIS/TIOCMBIC ) on Linux or GetCommModemStatus/EscapeCommFunction call on Windows to get this working?

[Frank Mehnert]

Surely it can't be difficult to provide a patch which was properly tested?

[aeichner]

@lawrenc erust: If you would have taken a look at the source code before posting you should have seen that these things you mentioned are already there ;). I think that the problem here is related to timing which is not easy to fix because the time it takes to get the status pin updates from the hardware to the guest is much longer than without virtualization involved.

[Gregory Nowak]

In that case, how is it that qemu and virtual-pc don't have this problem, and vmware supposedly doesn't have this issue either. I still think that the original poster's supposition that handshake is the problem is correct.

[Lawrence Rust]

Did some investigating and it turns out that this is mostly a Linux bug. In kernel 2.6.24 serial_core.c implements uart_wait_modem_status which waits for the h/w driver (8250.c) to report a modem status change. Unfortunately the 8250 driver doesn't correctly handle enabling modem interrupts and so never reports a change unless there's tx or rx data.

The vbox host serial driver code also has a few minor glitches which I've ironed out with this diff: ​ftp://ftp.softsystem.co.uk/vbox/DrvHostSerial.diff. The revised file is availabe here:​ftp://ftp.softsystem.co.uk/vbox/DrvHostSerial.cpp

To get it working for now, I switched to the Darwin code in DrvHostSerial.cpp - drvHostSerialMonitorThread, which polls for modem status changes. The bug workaround is enabled by the macro LINUX_CMIWAIT_BUG currently set to 1 in line 65). IMHO the Darwin code sleeps for too long (500ms). I would prefer 50 ms or less for those apps that expect timely responses.

HTH

[Lawrence Rust]

Linux TIOCMIWAIT workaround

[Lawrence Rust]

Oops. Sorry but the ftp link in my last post needs a password, so I've added the diff attachement above.

[Lawrence Rust]

Linux TIOCMIWAIT workaround #2

[Lawrence Rust]

I managed to find a much better solution without resorting to polling:

In Linux 2.6.24 and earlier, if a thread calls tcsetattr while the monitor thread is waiting in ioctl for a modem status change then 8250.c wrongly disables modem irqs and so the monitor thread never gets released. The workaround is to send a signal after each tcsetattr.

I've done this in the diff attached. This responds far quicker to modem status line changes and avoids all the polling overhead. Who should be contacted about getting this fixed in the Linux kernel sources?

[aeichner]

Thank you very much for your work on finding the problem. I'll have a look at your patch soon. Can you please state that the patch is licensed under the MIT (or you sign the SCA ​http://www.virtualbox.org/wiki/ICA) so that we can apply the patch to the svn?

[Lawrence Rust]

I have just signed the SCA and mailed it back so that this patch can be used.

[Graham]

I am also currently having this problem on a 64-bit Fedora10 host using the 2.6.27.12 Linux kernel. It is preventing me from retiring an old Windows machine.

[Marco Cruz]

Fedora 11 Host : Win XP SP3 Guest Serial Port: COM1, Host Device (/dev/ttyS0)

GetCommModemStatus( HANDLE hFile, LPDWORD lpModemStat ) always returns 0 and I can't test lpModemStat against MS_CTS_ON.

But on:

Win Vista x64 SP2 host : Win XP SP3 Guest Serial Port: COM1, Host Device (COM1)

it works perfectly. I used the same XP Guest on both hosts.

[Marco Cruz]

BTW, VirtualBox version is 3.1.2

[lazyest]

Seems like issue still in 3.2.10 there, at least Debian host+guest OS/2 doing what it wants with DTR and it is totally impossible to use old com-port modem. What can i debug to help improve virtualbox?

[lazyest]

Version 3.2.10r66523, I'm getting ММ╠ММ╠ММ╠ (8C8CCC hex) excepts RING. What can I do to fix it?

[Lawrence Rust]

Fix serial handshake

[Lawrence Rust]

This bug should have been fixed 2 years ago with the patch that I submitted. In case you missed it here's that patch again, re-based to the current OSE top of tree (Revision: 33444).

[lazyest]

I thought that if i install box from the scratch - it should be already patched or so? I've installed PEUL version, whould I switch to OSE?

[Lawrence Rust]

Both versions only get updated when a maintainer decides to update the source tree. It seems that nobody from Oracle reads or cares about this problem (or is it Not Invented Here syndrome?). So the only viable solution is to checkout the OSE source using svn, apply the patch and build it yourself.

[lazyest]

Thanks a lot, I think i find a temp workaround or so.

[Frank Mehnert]

Sorry, will have a look at the patch again.

[Frank Mehnert]

The fix for Linux hosts is contained in VBox 3.2.12.

[hgiritzer]

Problem still exists in 3.2.12 with Host - WinXP Profesional and Slave - WinXP Professional (similar configuration to the one described in the initial report).

I.e. DTR signal asserted in the guest OS is not asserted at the host interface (mapped COM1 to COM1).

[hgiritzer]

Additional information: The DTR signal at the guest OS is asserted by software, no (hardware) handshake active.

[fouadatmeh]

Problem still exists in 4.0.0 with Host - Win7 Profesional and Guest- Ubuntu 10.10

DTR signal from guest OS is not available at the host interface..

[Kevin McGee]

Host: Windows 2003 Server, Guest: NT4 Workstation

We have an application on NT4 Guest that uses an external modem connected to Host COM1:

Everything works OK with VirtualBox-3.1.8-61349-Win.

In July-2010 we upgraded to VirtualBox-3.2.6-63112-Win but found problems with modem access, it dialed & connected OK but DTR signal was missing at the modem, so no dialogue with the remote.

We reverted back to VirtualBox-3.1.8-61349-Win; DTR was raised as soon as application attached modem.

Today we tried latest VirtualBox-4.1.2-73507-Win and found DTR problem still exists, so again have reverted back.

Having searched & found this posting I thought it might be worth adding our experiences.