Heavy NAT network usage crashing VBox (hash collision) => Fixed in SVN

[Gene]

I'm running the latest VBOX 1.5.4 on Ubuntu 7.10 (Gutsy) 64-bit version. On a Quad Core Intel Q6600 processor. My guest OS is Win XP SP2 using NAT in the VBox Network Settings. When I do something VERY network intensive on the guest OS such as running a port scanner or proxy scanner for more than a few minutes, the Virtual Machine core dumps (aborts), and I get the following log:

Log created: 2008-01-04T23:07:56.227929000Z
Executable: /usr/lib/virtualbox/VirtualBox
Arg[0]: /usr/lib/virtualbox/VirtualBox
Arg[1]: -comment
Arg[2]: WindowsXP
Arg[3]: -startvm
Arg[4]: f2981c30-2053-45bd-4b97-76e88f9367f9

!!Assertion Failed!!
Expression: <NULL>
Location : /home/vbox/vbox-1.5/src/VBox/Devices/Network/slirp/tcp_subr.c(1341) VBoxU32PtrHashSlow
NAT pointer hash error. pv=00002aaab036c890 cpvHashUsed=321 cpvHashCollisions=198

[Alessandro Battilani]

I have the same problem on my AMD64 dual core with VBOX 1.5.4 on Ubuntu 7.10 (Gutsy) 64-bit version. I have attached the core dump log file.

[Frank Mehnert]

#1170 has been marked as duplicate of this bug.

[Christophe Klopp]

I have the same problem on my INTEL quad with VBOX 1.5.6 on Centos 5 64bits version run a xubuntu 6.06 client. The problem is reproduceable and the virtual machine crashes every ~500 queries.

[Frank Mehnert]

This is a known problem with NAT on 64-bit hosts. The problem is that slirp (the NAT engine VirtualBox is using) is not 64-bit clean. Our hack to work around this problem is obviously not sufficient...

[Christophe Klopp]

Thanks' for this information. Have you any idea when this will be corrected?

Is there the same problem with host interface networking?

[Frank Mehnert]

No ETA when this is corrected. No, this problem does not exist with the host interface networking as this interface relies on the hosts IP stack.

[Frank Mehnert]

Btw, have a look about this topic here (the first few lines): ​http://lists.freebsd.org/pipermail/freebsd-emulation/2007-February/003107.html

Contributions are welcome :)

[Victor Raff]

I have the same problem with VirtualBox 1.5.6 on Kubuntu 8.04 AMD64 on Intel Core 2 Duo CPU. Guest is WinXP SP2.

[Victor Raff]

Btw, whether it should help (at least theoretically) if VM will have no network?

[Frank Mehnert]

Yes, this would help. Or use host interface networking + bridging, see the user manual. Unfortunately, host interface networing is kind of broken in 1.6.0 but we hope that we can release a bugfix release soon (I cannot explain anything as always).

[Victor Raff]

I have installed the 1.6.0 question but the problem remains.

[Victor Raff]

*version

[Frank Mehnert]

The problem was still not fixed yet in 1.6.0.

[Victor Raff]

Could you please let us know whether there is the bug fix for host interface networking for 1.6.0 and how to get it and install? Also maybe you have some news about the original NAT problem. Thank you!

[Frank Mehnert]

Host networking is already fixed in SVN and 1.6.2 (which contains the fix) will be out very soon. The NAT problem is still not fixed. This problem only occurs on 64-bit hosts. As I stated above, we are using the slirp code which has some limitations. We plan to fix this but this will take some more time.

[Victor Raff]

Thank you very much! Will be looking forward to the 1.6.2 release then.

[Masoris]

I have same problem on VirtualBox 1.6.2. I use Ubuntu 8.04 64bit host, and Windows XP guest. When I run a P2P software in guest, VirtualBox randomly crash with NAT pointer hash error.

[Yaniv Aknin]

Just my 0.02$: As a temporary resolution, I've installed the dante SOCKS server, and it works OK with some of the most common P2P programs out there.

[Mush]

Same problem here: Xeon-System running Ubuntu 8.04 as both host and guest:

!!Assertion Failed!! Expression: <NULL> Location : /home/vbox/vbox-1.6.2/src/VBox/Devices/Network/slirp/tcp_subr.c(1341) VBoxU32PtrHashSlow NAT pointer hash error. pv=0000000000ee75d0 cpvHashUsed=305 cpvHashCollisions=682

[Mush]

There are two tickets which are marked as duplicates of this one:
​http://www.virtualbox.org/ticket/557 (11 months ago)
​http://www.virtualbox.org/ticket/1663 (1 month ago)
Remarkably both of these tickets are marked as critical. This one should be, too.

Does anyone know a suitable workaround for an environment where bridging is not an option (i.e. where no additional public IP addresses are available)?

[Frank Mehnert]

The priority of this ticket doesn't matter. We started a big networking overhaul some time ago which will fix this issue among others. Unfortunately, there is no workaround. This bug only happens on AMD64 hosts and only with NAT (buggy slirp engine).

[Victor Raff]

I have the same problem as I mentioned above, but fortunately there is a workaround at least for my specific situation. Since the host interface networking was repaired I have used it in the following way:

1. I've created a new virtual interface vbox0 and told the VirtualBox to use it.
2. Then I have added special rules to iptables to actually enable NAT.

As you probably already noticed I didn't bother with bridge creation. As a result I have NAT working, but also I can forward ports if I need to. And it didn't require additional real IPs or any other changes to host network configuration.

I would be happy to provide more details for anybody interested.

[Frank Mehnert]

Yes, that is a good idea. Use host interface if possible as this interface currently better performs than NAT and it has no known bugs (in contrast to NAT). Perhaps you would like to write some text for our wiki. There is no public write access but I would delighted to put your text on the Wiki pages. But perhaps some of your ideas are already written in either

• ​http://www.virtualbox.org/wiki/PPP_Tunnel
• ​http://www.virtualbox.org/wiki/Advanced_Networking_Linux or
• ​http://www.virtualbox.org/wiki/OpenVPN_on_VBox

(see our HOWTO pages at ​http://www.virtualbox.org/wiki/User_HOWTOS)

[Victor Raff]

You are right - it's not that hard to do such networking and parts of the process are already described in various sources. I will try to explain what I did and maybe it will serve as a simple HOWTO.

In my specific case I am using Ubuntu 8.04 AMD64, my regular Internet connection is via PPP. In the VirtualBox help there is a clear and detailed description of how a virtual network adapter could be added and how to use it in host interface networking. So after it is done I have the following network configuration:

victor@victor-laptop:~$ ifconfig
eth0      Link encap:Ethernet  HWaddr 00:1d:60:9e:c6:cb
          inet addr:192.168.123.135  Bcast:192.168.123.255  Mask:255.255.255.0
          inet6 addr: fe80::21d:60ff:fe9e:c6cb/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:4162777 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1142042 errors:0 dropped:0 overruns:0 carrier:1
          collisions:0 txqueuelen:1000
          RX bytes:2349864401 (2.1 GB)  TX bytes:133942026 (127.7 MB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:28947 errors:0 dropped:0 overruns:0 frame:0
          TX packets:28947 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:2173358 (2.0 MB)  TX bytes:2173358 (2.0 MB)

ppp0      Link encap:Point-to-Point Protocol
          inet addr:77.244.36.169  P-t-P:77.244.44.62  Mask:255.255.255.255
          UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1500  Metric:1
          RX packets:1529469 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1561780 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:3
          RX bytes:2128434933 (1.9 GB)  TX bytes:252926948 (241.2 MB)

vbox0     Link encap:Ethernet  HWaddr 00:ff:f2:ef:ee:6e
          inet addr:172.16.0.1  Bcast:172.16.0.255  Mask:255.255.255.0
          inet6 addr: fe80::2ff:f2ff:feef:ee6e/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:78 errors:0 dropped:10 overruns:0 carrier:0
          collisions:0 txqueuelen:500
          RX bytes:0 (0.0 B)  TX bytes:5750 (5.6 KB)

The vbox0 is the new virtual adapter. The ppp0 is the tunnel, which is opened when I connect to my provider. Then I have set my VM network card to have 172.16.0.2 IP address and 172.16.0.1 as a default gateway. I have set its DNS the same way as it is set on my host system.

Finally, I run the following commands as root to enable NAT:

iptables -A FORWARD -d 172.16.0.2 -j ACCEPT
iptables -A FORWARD -s 172.16.0.2 -j ACCEPT
iptables -t nat -A POSTROUTING -o ppp0 -s 172.16.0.2 -j MASQUERADE
echo 1 > /proc/sys/net/ipv4/ip_forward

Of course if you don't use PPP and, for example, have only eth0 - then just replace ppp0 with eth0 in the code provided.

If then I want to say forward a port for VNC I run something like that:

iptables -t nat -I PREROUTING -p tcp --dport 5899 -i ppp0 -j DNAT --to 172.16.0.2:5900

Basically that's all I do to enable networking for my VM and it works great so far - I am using it intensively for couple of months already. I hope it helps.

[Victor Raff]

I forgot to add - when you will be reading the VirtualBox manual (menu help -> contents -> virtual networking -> Host Interface Networking and bridging on Linux hosts) you will need only the part related to the virtual interface creation (you don't have to specify bridge name here)

sudo VBoxAddIF vbox0 <user>

then set an IP address for the interface and then tell VM to use it (done via virtual machine settings). So, once again, you don't have to create bridge for it.

[Mush]

I can confirm that it works. Thank you!

[Stéphane Charette]

Happening to me with light network usage. Host is Ubuntu 8.04-64, running 64-bit version of VirtualBox 2.0.2.

Left guest Windows Vista 32-bit running overnight with MS Outlook as the only open application. Came back in the morning to find guest had crashed:

!!Assertion Failed!! Expression: <NULL> Location : /home/vbox/vbox-2.0.2/src/VBox/Devices/Network/slirp_tcp_subr.c(1341) VBoxU32PtrhashSlow NAT pointer hash error. pv=0009ab010 cpvHashUsed=13 cpvHashCollisons=17

(in case it contains a typo, note the error text above is retyped from 2nd computer, not cut-and-paste)

[Ari]

Same problem verified with 2.02, I tried downgrading to 1.6 only to find the same issue (for some reason it happens more often now). Do we have an ETA for this?

With 1.6:

Log created: 2008-10-20T21:16:15.580136000Z Executable: /usr/lib/virtualbox/VirtualBox Arg[0]: /usr/lib/virtualbox/VirtualBox Arg[1]: -comment Arg[2]: VBWK16 Arg[3]: -startvm Arg[4]: ba408733-3bbf-4b29-aeb3-ff4b156cff28

!!Assertion Failed!! Expression: <NULL> Location : /home/vbox/vbox-1.6.6/src/VBox/Devices/Network/slirp/tcp_subr.c(1341) VBoxU32PtrHashSlow NAT pointer hash error. pv=00007f0e580c5010 cpvHashUsed=49 cpvHashCollisions=36

With 2.02:

Log created: 2008-10-14T19:28:39.113373000Z Executable: /usr/lib/virtualbox/VirtualBox Arg[0]: /usr/lib/virtualbox/VirtualBox Arg[1]: -comment Arg[2]: VBWK Arg[3]: -startvm Arg[4]: d8d080d9-7aa7-451d-b889-bc3a9116e970

!!Assertion Failed!! Expression: <NULL> Location : /home/vbox/vbox-2.0.2/src/VBox/Devices/Network/slirp/tcp_subr.c(1341) VBoxU32PtrHashSlow NAT pointer hash error. pv=0000000000a88010 cpvHashUsed=66 cpvHashCollisions=198

[TobiasWahl]

This bug seems to be fix within 2.04. I use the new release for some days and had no crashes.

[Frank Mehnert]

No, the bug is not fixed but we are working on it.

[mumhero]

I have the same problem within 2.0.4. 64 bit linux host, 32 bit xp guest.

Log created: 2008-11-08T05:23:36.901638000Z Executable: /opt/virtualbox/VirtualBox Arg[0]: /opt/virtualbox/VirtualBox Arg[1]: -comment Arg[2]: xp Arg[3]: -startvm Arg[4]: 44776758-7cc9-472e-26b8-0c9915f66ff0

!!Assertion Failed!! Expression: <NULL> Location : /home/vbox/tinderbox/2.0-lnx64-rel/src/VBox/Devices/Network/slirp/tcp_subr.c(1341) VBoxU32PtrHashSlow NAT pointer hash error. pv=0000000001bc0cf0 cpvHashUsed=135 cpvHashCollisions=4

[Yaniv Aknin]

Seeing the same on Solaris with VBox 2.0.6. 64bit OpenSolaris host, 32bit XP guest.

00:04:07.242 PCNet#0: Init: ss32=1 GCRDRA=0x02a89420[64] GCTDRA=0x02a89020[64] 00:42:31.379 00:42:31.379 !!Assertion Failed!! 00:42:31.379 Expression: <NULL> 00:42:31.379 Location : /export/home/vbox/tinderbox/2.0-sol64-rel/src/VBox/Devices/Network/slirp/tcp_subr.c(1341) VBoxU32PtrHashSlow 00:42:31.521 NAT pointer hash error. pv=0000000003ffba00 cpvHashUsed=249 cpvHashCollisions=74

[Ale Feltes Quenhan]

Same problem, using NAT and copying a large file through samba:

Sun xVM VirtualBox SDL GUI 2.0.4_OSE built Oct 26 2008 19:32:24

!!Assertion Failed!! Expression: i < 4096 Location : /build/buildd/virtualbox-ose-2.0.4-dfsg/src/VBox/Runtime/r3/linux/semevent-linux.cpp(188) int RTSemEventSignal(RTSEMEVENTINTERNAL*) iCur=0x1 pThis=00000000013d97d0 Trace/breakpoint trap

Host: Ubuntu 8.10, Linux megan 2.6.27-9-generic #1 SMP Thu Nov 20 22:15:32 UTC 2008 x86_64 GNU/Linux

Guest: SysteRescueCd

[Frank Mehnert]

Please don't post any more assertions like this. We are aware of that problem and it seems that we will solve it finally with the next release.

[Ale Feltes Quenhan]

I'm sorry for the last post, I was just trying to help resolve the bug.

I wasn't sure if this qualifies as a new Bug, but I got the same error, using Host interface netwoking type, with bridge, and doing the NAT outside VBox, with iptables on host machine. Could the bug apply to any heavy network activity and not just NAT one?

This is my config:

On host (Ubuntu 8.10 64 bits):

#tunctl -t tap0 -u user

#ifconfig tap0 10.0.1.1

#iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE

#echo "1" > /proc/sys/net/ipv4/ip_forward

On client (SystemRescueCD):

# ifconfig eth0 10.0.1.2

# route add default gw 10.0.1.1

With Host network type I got this log:

---

Log created: 2008-12-12T15:17:45.555113000Z[[BR]]

Executable: /usr/lib/virtualbox/VirtualBox

Arg[0]: /usr/lib/virtualbox/VirtualBox

Arg[1]: -comment

Arg[2]: delphi7-os

Arg[3]: -startvm

Arg[4]: 4a728300-7d01-4f45-954e-977f27d893f3

!!Assertion Failed![[BR]]

Expression: i < 4096

Location : /build/buildd/virtualbox-ose-2.0.4-dfsg/src/VBox/Runtime/r3/linux/semevent-linux.cpp(188) int RTSemEventSignal(RTSEMEVENTINTERNAL*)

iCur=0x1 pThis=00007fad940507e0

---

With NAT networking type, the crash happens after 500MB of transfer, I did the test 3 times, with same result.

With HOST interface networking type, the crash happens around 1.4G of transfer, both times I have tested.