On later RHEL6.8 kernels, hostonly networks stop working after just a few minutes

[ElCoyote]

Hi everyone, Thanks for VirtualBox, it's a great product.

I use VirtualBox as my hypervisor for virtual OpenStack deployments. I have been using it since the early 5.0 times to deploy multi-nodes OpenStack deployments. These deployments typically include 4 to 12 nodes as VBox machines, using multiple e1000 interfaces (4 per vbox machine, 3 of them being host-only network interfaces).

Here's some more detail:

• Large RHEL6.x hypervisor (128Gb or 192Gb RAM). These typically are kept up-to-date with the RedHat kernels.
• One large undercloud VBox VM (16Gb, pagefusion off, nestedpaging on, vtxvpid on, largepages on, paravirt KVM). It has 4 NICs: -- nic0: hostonly to vboxnet0 -- nic1: hostonly to vboxnet1 -- nic2: hostonly to vboxnet2 -- nic3: Bridged to external network (Main Hypervisor interface)

• 8 to 16 overcloud VBox VM's with the same settings as the above but using a slightly different NIC configuration: -- nic0: hostonly to vboxnet0 -- nic1: hostonly to vboxnet1 -- nic2: hostonly to vboxnet1 (to make a BOND with nic1) -- nic3: hostonly to vboxnet2

On nic1/nic2 (connected to vboxnet1), I use a mixture of 802.1Q Tagged VLAN's (this is specific to OpenStack).

As I am testing some OpenStack corner cases, I am deleting/re-creating this setup on a very frequent basis (sometimes 4 times per day).

Description of problem: On latest RHEL6.8 kernels (2.6.32-642.6.1), the vboxnet[0-2] network stop 'working' after just a few minutes when deploying the nodes. No traffic can be seen by tcpdump'ing the adapters on either the Hypervisor or the nodes. On the hypervisor (RHEL6.8 host), I attempt to capture network traffic like this: # tcpdump -nevi vboxnet1 or # tcpdump -nevi vboxnet2

On the undercloud node, I attempt to capture traffic like this: # tcpdump -nevi eth1 (connected to vboxnet1) or # tcpdump -nevi eth2 (connected to vboxnet2)

At the begining of a deployment (PXE'ing the overcloud nodes), things start working properly but stop after just a few minutes.

Things were working properly with: VBox 5.0.26 + Kernel 2.6.32-642.3.1.el6.x86_64 VBox 5.0.26 + Kernel 2.6.32-642.4.2.el6.x86_64 VBox 5.1.6 + Kernel 2.6.32-642.4.2.el6.x86_64

I am also taking great care in making sure that the vbox kernel modules are freshly compiled on the running kernel by cleaning out the /lib/modules and forcing a rebuild (using DKMS on VBox 5.0 and using vboxconfig on VBox 5.1)

Things stop working after just a few minutes with: VBox 5.1.6 + Kernel 2.6.32-642.6.1.el6.x86_64 (the latest RHEL6.8 kernel).

I am currently trying to isolate the issue and see if I can provide more data. Is there a way to put an hostonly network in debug mode? Thanks,