You can disable certain VM visual modes:

$ VBoxManage setextradata VM-name GUI/RestrictedVisualStates property[,property...]

property is one of the following:

Fullscreen

Do not allow to switch the VM into full screen mode.

Seamless

Do not allow to switch the VM into seamless mode.

Scale

Do not allow to switch the VM into scale mode.

This is a per-VM setting. You can specify any combination of properties. To restore the default behavior, use the following command:

$ VBoxManage setextradata VM-name GUI/RestrictedVisualStates