VBoxLinuxAdditions.run fails in Ubuntu Server 16.04.3 -> module signing problem

[Red Viking]

VBoxLinuxAdditions.run fails in Ubuntu Server 16.04.3:

administrator@ubuntu:~$ sudo /media/cdrom/VBoxLinuxAdditions.run
.
.
.
  ld -r -m elf_x86_64 -T ./scripts/module-common.lds --build-id  -o /tmp/vbox.0/vboxvideo.ko /tmp/vbox.0/vboxvideo.o /tmp/vbox.0/vboxvideo.mod.o
make KBUILD_VERBOSE=1 CONFIG_MODULE_SIG= -C /lib/modules/4.4.0-97-generic/build SUBDIRS=/tmp/vbox.0 SRCROOT=/tmp/vbox.0 INSTALL_MOD_PATH= INSTALL_MOD_DIR=misc modules_install
test -e include/generated/autoconf.h -a -e include/config/auto.conf || (                \
echo >&2;                                                       \
echo >&2 "  ERROR: Kernel configuration is invalid.";           \
echo >&2 "         include/generated/autoconf.h or include/config/auto.conf are missing.";\
echo >&2 "         Run 'make oldconfig && make prepare' on kernel src to fix it.";      \
echo >&2 ;                                                      \
/bin/false)
mkdir -p /lib/modules/4.4.0-97-generic/misc
make -f ./scripts/Makefile.modinst
  mkdir -p /lib/modules/4.4.0-97-generic/misc ; cp /tmp/vbox.0/vboxvideo.ko /lib/modules/4.4.0-97-generic/misc ; true /lib/modules/4.4.0-97-generic/misc/vboxvideo.ko ; if (echo "/lib/modules/4.4.0-97-generic/misc/vboxvideo.ko" | egrep -q "\/drivers\/staging\/") && (! egrep -x "vboxvideo.ko" ./drivers/staging/signature-inclusion) ; then echo Not signing "/lib/modules/4.4.0-97-generic/misc/vboxvideo.ko"; else scripts/sign-file "sha512" "certs/signing_key.pem" certs/signing_key.x509 /lib/modules/4.4.0-97-generic/misc/vboxvideo.ko || true && true /lib/modules/4.4.0-97-generic/misc/vboxvideo.ko; fi
At main.c:222:
- SSL error:02001002:system library:fopen:No such file or directory: bss_file.c:175
- SSL error:2006D080:BIO routines:BIO_new_file:no such file: bss_file.c:178
sign-file: certs/signing_key.pem: No such file or directory
  /bin/bash ./scripts/depmod.sh /sbin/depmod 4.4.0-97-generic ""
Creating user for the Guest Additions.
Creating udev rule for the Guest Additions kernel module.
Could not find the X.Org or XFree86 Window System, skipping.

Fails for following:

VBoxGuestAdditions_5.2.1-118447.iso
VBoxGuestAdditions_5.2.1-118468.iso

administrator@ubuntu:~$ uname -a
Linux ubuntu 4.4.0-97-generic #120-Ubuntu SMP Tue Sep 19 17:28:18 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

[Perry G]

The bugtracker is for reporting a bug. All requests for help should be sent to ​https://forums.virtualbox.org

Unless you are manually installing/building Xorg and XFree86 you need to install with the --nox11 tag because server installs don't have DT's. Still all you will achieve is shared folders and not much else.

[Red Viking]

I still think this is a bug. Using "--nox11" did not solve the problem:

administrator@ubuntu:~$ sudo /media/cdrom/VBoxLinuxAdditions.run --nox11
[sudo] password for administrator:
Verifying archive integrity... All good.
Uncompressing VirtualBox 5.2.1 Guest Additions for Linux........
VirtualBox Guest Additions installer
Removing installed version 5.2.1 of VirtualBox Guest Additions...
Copying additional installer modules ...
Installing additional modules ...
VirtualBox Guest Additions: Building the VirtualBox Guest Additions kernel modules.
VirtualBox Guest Additions: Running kernel modules will not be replaced until the system is restarted
VirtualBox Guest Additions: Starting.
VirtualBox Guest Additions: modprobe vboxsf failed

Please note that I've been installing Guest Additions in Ubuntu Server without any options without issue using many earlier versions of VirtualBox.

[Michael Thayer]

You have set up your guest system to require signed kernel modules. That is something that we do not currently support, so you will need to either disable it or take care of it yourself.

[M.Poil]

Same problem, I'm using virtualbox with this VM since version VBox 3 without any problem. (And no I'm not using secureboot, My VM have not UEFI but Bios)

[Socratis]

@M.Poil
I don't believe that this is the same problem. Try to disable 3D acceleration. Also, take a look at #17463, I have a feeling that this might be your problem.