Problems with Cisco VPN client on 3.x

[Zoltan Fekete]

With Vbox 3.0, and 3.0.2, problems appeared with a NAT networking. (on 2.2 and 2.4 everything worked fine).

Basically not all packages go trough the NAT. Attached is a log and a screenshot, on left a ping command from the guest, on right ping command from the host.

This is a problem for VPN and such connections (Cisco VPN for example) that breaks after couple of seconds or random.

Bridge on the same physical adapter works just fine, only in NAT mode, this problem is evident.

[Zoltan Fekete]

The log file

[Zoltan Fekete]

Ping command, left is guest, right is host.

[vasily Levchenko]

Replying to zoltanf:

With Vbox 3.0, and 3.0.2, problems appeared with a NAT networking. (on 2.2 and 2.4 everything worked fine).

Basically not all packages go trough the NAT. Attached is a log and a screenshot, on left a ping command from the guest, on right ping command from the host.

Ping is ICMP based application, and the picture you've attached could be admired on Windows host only. This behavior caused by problems with ICMP API used only on Windows, we expect that this problem will gone in 3.1 where we will replace ICMP API with sockets like it's done on Unix.

If you suspect packet lost please collect pcap files on guest and host with wireshark (to sent packets was sent from guest and host ). ​Intermediate networking is also will be helpful.

This is a problem for VPN and such connections (Cisco VPN for example) that breaks after couple of seconds or random.

Bridge on the same physical adapter works just fine, only in NAT mode, this problem is evident.

[Zoltan Fekete]

Replying to Hachiman:

Ping is ICMP based application, and the picture you've attached could be admired on Windows host only. This behavior caused by problems with ICMP API used only on Windows, we expect that this problem will gone in 3.1 where we will replace ICMP API with sockets like it's done on Unix.

For ICMP, it is probably true. But, I did not try the PING for fun, I had real problems with the NAT (e.g. cisco VPN breaking up every few seconds to minutes).

If you suspect packet lost please collect pcap files on guest and host with wireshark (to sent packets was sent from guest and host ). ​Intermediate networking is also will be helpful.

I will collect the data today, and attach.

[vasily Levchenko]

Replying to zoltanf:

Replying to Hachiman:

Ping is ICMP based application, and the picture you've attached could be admired on Windows host only. This behavior caused by problems with ICMP API used only on Windows, we expect that this problem will gone in 3.1 where we will replace ICMP API with sockets like it's done on Unix.

For ICMP, it is probably true. But, I did not try the PING for fun, I had real problems with the NAT (e.g. cisco VPN breaking up every few seconds to minutes).

What version of you Cisco VPN client?

If you suspect packet lost please collect pcap files on guest and host with wireshark (to sent packets was sent from guest and host ). ​Intermediate networking is also will be helpful.

I will collect the data today, and attach.

[Zoltan Fekete]

Replying to Hachiman:

What version of you Cisco VPN client?

Cisco VPN client, version is 4.8.02.0010

I have done some further testing today, and also did some network captures that I analyzed (I will attach them here later). It seems that the NAT is working fine with TCP/IP traffic, but with UDP it is a problem where some packets are lost (and a large number, especially if the HOST is connected trough wireless connection to the internet).

Cisco VPN in my configuration uses tunneling on UDP port 4500.

[vasily Levchenko]

thank you for information

[Zoltan Fekete]

Capture, inside of the cisco tunnel. This shows some packets re-transmited inside the tunnel.

[Zoltan Fekete]

I have attached now a file, with a capture of network traffic inside the Cisco VPN tunnel. This is the best case scenario, as my host is currently running on LAN and not on wireless (when the host is on wireless, the number of re-transmited packages grows substantially).

I know that this capture is not of much use, as it is inside the tunnel and not outside where the UDP packages are actually exchanged. I will try now to do a dump of the adapter from VBox.

[Zoltan Fekete]

Capture from virtualbox, nic 1, on NAT, chunk

[Zoltan Fekete]

Attached the nic 1 capture, a junk of it anyway. Since it is UDP, and is encrypted (tunnel data), there is nothing to be seen here.

The trouble is, that UDP, unlike TCP, does not provide a mechanism for detecting dropped packets. You might see the dropped packets on some higher level protocol that uses UDP, but since this chunk is encrypted tunnel traffic, here it is impossible to see. Ah, well, you will probably need to try this with some other app protocol that uses UDP...

[vasily Levchenko]

Replying to zoltanf:

Attached the nic 1 capture, a junk of it anyway. Since it is UDP, and is encrypted (tunnel data), there is nothing to be seen here.

The trouble is, that UDP, unlike TCP, does not provide a mechanism for detecting dropped packets. You might see the dropped packets on some higher level protocol that uses UDP, but since this chunk is encrypted tunnel traffic, here it is impossible to see. Ah, well, you will probably need to try this with some other app protocol that uses UDP...

Right, will try reproduce it on TFTP.

[Zoltan Fekete]

Hichiman,

I don't know what is wrong with comments, but I don't see your last comment here in this ticket history, but did get the notification to email, so I will reply here directly:

---

`i've just thought Cisco clients has logging on board, it might be interesting if Cisco client detects these problems. About retransmit they could be initiated by TCP ... and there could be several reasons for retransmission : expiration, out of order packets deliviring, packet duplication and dropping. please check if it possible to sniff network traffic from Cisco driver with wireshark and logs from Cisco client?

---

About the Cisco wireshark sniffing, I did attach the file earlier. (Cisco Adapter Capture 3.pcap). This was captured inside the VPN tunnel, so you CAN see dropped and retransmitted packages inside this dump. I will check if there is some possibility to enable logging inside Cisco VPN client, and if it loges the dropped packages.

Today, while using the VM in NAT mode, on LAN, I have expirenced several complete networking blackouts, for no apperant reason. I find that here is a ticked open for this already, for 3.0.2., ticket no #4507. Could be releated to this tickets problem...

[Sachin Garg]

I am also seeing a problem with the Nortel Contivity VPN client on a Windows XP SP3 guest on a Ubuntu 9.04 hosts. My VPN client is unable to connect to the server in 3.x series (currently 3.0.2). I was able to use the exact same setup on 2.2.4 earlier.

[vasily Levchenko]

Test build was send for verification

[Zoltan Fekete]

I can confirm that the test build 3.0.3 solves all NAT problems that I had.

[vasily Levchenko]

Replying to zoltanf:

I can confirm that the test build 3.0.3 solves all NAT problems that I had.

Thank you for feedback

[amidsin]

I have also noticed this problem while using VPN Client below there is detailed description just in case if somebody would need to confirm that they have similar symptoms. I was able to work around it by using bridged networking. I will be looking forward using 3.0.3.

After upgrading from VirtualBox version 2.2 to version 3 on Ubuntu host system, IPSec client in Windows guest system has stopped working. At first I blamed IPSec itself or network or Windows install and etc. Latter I narrowed it down to upgrading VirtualBox and have confirmed. I have downgraded it back a couple of time upgraded again. Each time when I was downgrading, problem went away. Each time after upgrading problem was returning.

How it used to work with version 2.2 of Virtual Box before upgrading – connecting to the IPSec gateway for about 10 sec, checking banner text for about 2 sec, connection is established.

Symptoms with version 3 of the VirtualBox after upgrading – Establishing IPSec connection is failing most of the times (~ 1 in 5 times it succeeds). Following scenarios were observed:

• Most frequent - Connecting for a long time (~60 sec), switching over to back-up IPSec gateway, connecting for some time (~30 sec), connections is is failing with a message about failed login.
• Sometimes - Connecting for a long time (~30 secods), checking for banner text for a long time (~30 sec) after which connection is failing (connection lost message).
• Rarely - Connecting for a long time (~30 secods), checking for banner text quickly (~3 sec) after which connection is established.

Additional symptoms on version 3 of VirtualBox: Under Network Connections there is a single network adapter (Local Area Network), which is connected to virtual network adapter of the guest machine (AMD PCNET Family …).

VirtualBox version working with IPSec: 2.2.4 r47978 VirtualBox version failing with IPSec: 3.0.2 r49928

Host Operating System: Ubuntu 9.04 32 bit. Guest operating system: Windows XP MCE 2005 (32 bit). IPSec VPN Software: Nortel Connectivity VPN Client V06_01.054

[ddn]

Cisco VPN Client 32bit last version didn't working (IPSec) in 3.2.0 again with NAT networking. (guest winxp sp3 32bit/ host windows 7 64bit) i used TCP with port lower than 1024. Connection to host can't be established

[vasily Levchenko]

Could you please collect pcap files (from guest and host ) and attach them to defect or send them to me (vasily _dot_ levchenko _at_ Sun _dot_ COM)?

[ddn]

Replying to Hachiman:

Could you please collect pcap files (from guest and host ) and attach them to defect or send them to me (vasily _dot_ levchenko _at_ Sun _dot_ COM)?

i'll try consult with our security. wireshark sniff trace files contains private information and may lead to its disclosure :)

[vasily Levchenko]

i'll try consult with our security. wireshark sniff trace files contains private information and may lead to its disclosure :)

Probably your security could investigate the traces and give some hint what missed in the packets sent from the guest.

[vasily Levchenko]

Could you please ​VBoxDD.dll download debug bits and replace installed dll?

I'll have to ask you to collect trace files again with logs produced with this library.

How to start?

# set VBOX_LOG=drv_nat.e.l2
# set VBOX_LOG_DEST=file=nat.log
# VirtualBox -startvm <your-vm name>

Please send result log with traces to me again.

[ddn]

okay. no prob. but can't download. Browser say HTTP 404. See below:

--20:42:26-- ​http://www.virtualbox.org/www/download/testcase/VBoxDD.dll.4499

=> `VBoxDD.dll.4499'

Resolving www.virtualbox.org... 88.198.19.108 Reusing existing connection to virtualbox.org:80. HTTP request sent, awaiting response... 404 Not Found 20:42:26 ERROR 404: Not Found.

[vasily Levchenko]

Replying to ddn:

okay. no prob. but can't download. Browser say HTTP 404. See below:

--20:42:26-- ​http://www.virtualbox.org/www/download/testcase/VBoxDD.dll.4499

=> `VBoxDD.dll.4499'

Resolving www.virtualbox.org... 88.198.19.108 Reusing existing connection to virtualbox.org:80. HTTP request sent, awaiting response... 404 Not Found 20:42:26 ERROR 404: Not Found.

wiki tags have played a joke on me here the right URL: ​http://www.virtualbox.org/download/testcase/VBoxDD.dll.4499.debug

[ddn]

i reply to your email (again HTTP 301 -> following -> HTTP 404)

[Frank Mehnert]

Please try ​this link again, it works now.

[ddn]

now all working. Im reproduced steps Hachiman wrote and emailed him all collected logs.

Thanks!

[vasily Levchenko]

Replying to ddn:

now all working. Im reproduced steps Hachiman wrote and emailed him all collected logs.

Thanks!

Thanks, reading now.

[vasily Levchenko]

Some intermediate report: bodies of tcp packets are the same from host and host-guest side (no data corruption). But anyway there're several retransmission from the NAT to guest, guest seems just ignore them, the difference with original packet (on host interface) is in PUSH flag, added by NAT. To make sure if it's a reason (and if it's a regression) I've asket ddn to collect the same logs on 3.1, or there're other reasons for such behavior.

[vasily Levchenko]

Could you please verify the ​build?

[ddn]

"This installation package is not supported by this processor type. Contact your product vendor." displayed after executing .msi file i have 32bit system. does it ok?

[vasily Levchenko]

Replying to ddn:

"This installation package is not supported by this processor type. Contact your product vendor." displayed after executing .msi file i have 32bit system. does it ok?

could you please try with 3.2.6 b2?

[ddn]

ok, will try.

P.S. Did you receive mail from me (with attached logs in 3.1 version) ?

[vasily Levchenko]

Replying to ddn:

ok, will try.

P.S. Did you receive mail from me (with attached logs in 3.1 version) ?

yes, thanks.

[ddn]

still no luck with 3.2.6BETA2 and NAT-mode. works only in bridged mode

[Markus Laker]

I'd just like to add that I had a similar problem using vpnc to connect to a Cisco VPN; guest is running 32-bit Kubuntu 9.10, host 64-bit Kubuntu 9.10. The problem I saw was wholesale packet loss, causing (for example) an ssh or telnet session to lock up if I typed 'ls -l', and preventing NoMachine NX from logging in. Moving the network adapter from Nat mode to bridged mode has solved the problem; many thanks to zoltanf for posting the workaround.

The reason I've piped up is that, in my case, VirtualBox 3.1.8 worked cleanly; the problem didn't appear until I tried 3.2.0 and 3.2.4.

[vasily Levchenko]

Replying to mlaker:

I'd just like to add that I had a similar problem using vpnc to connect to a Cisco VPN; guest is running 32-bit Kubuntu 9.10, host 64-bit Kubuntu 9.10. The problem I saw was wholesale packet loss, causing (for example) an ssh or telnet session to lock up if I typed 'ls -l', and preventing NoMachine NX from logging in. Moving the network adapter from Nat mode to bridged mode has solved the problem; many thanks to zoltanf for posting the workaround.

The reason I've piped up is that, in my case, VirtualBox 3.1.8 worked cleanly; the problem didn't appear until I tried 3.2.0 and 3.2.4.

Was it changed for you with 3.2.6? If no please upload your log file to the ticket.

[Frank Mehnert]

No response, closing.