VirtualBox

source: vbox/trunk/doc/manual/en_US/user_Technical.xml@ 49303

Last change on this file since 49303 was 44553, checked in by vboxsync, 12 years ago

Main/doc: prefer <HOME>/.config/VirtualBox over <HOME>/.VirtualBox for new users on platforms where XDG is relevant, user manual and change log changes.

File size: 43.2 KB
Line 
1<?xml version="1.0" encoding="UTF-8"?>
2<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.4//EN"
3"http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd">
4<chapter id="TechnicalBackground">
5 <title>Technical background</title>
6
7 <para>The contents of this chapter are not required to use VirtualBox
8 successfully. The following is provided as additional information for
9 readers who are more familiar with computer architecture and technology and
10 wish to find out more about how VirtualBox works "under the hood".</para>
11
12 <sect1 id="vboxconfigdata">
13 <title>Where VirtualBox stores its files</title>
14
15 <para>In VirtualBox, a virtual machine and its settings are described in a
16 virtual machine settings file in XML format. In addition, most virtual
17 machine have one or more virtual hard disks, which are typically
18 represented by disk images (e.g. in VDI format). Where all these files are
19 stored depends on which version of VirtualBox created the machine.</para>
20
21 <sect2>
22 <title>Machines created by VirtualBox version 4.0 or later</title>
23
24 <para>Starting with version 4.0, by default, each virtual machine has
25 one directory on your host computer where all the files of that machine
26 are stored -- the XML settings file (with a
27 <computeroutput>.vbox</computeroutput> file extension) and its disk
28 images.</para>
29
30 <para>By default, this "machine folder" is placed in a common folder
31 called "VirtualBox VMs", which VirtualBox creates in the current system
32 user's home directory. The location of this home directory depends on
33 the conventions of the host operating system:</para>
34
35 <itemizedlist>
36 <listitem>
37 <para>On Windows, this is
38 <computeroutput>%HOMEDRIVE%%HOMEPATH%</computeroutput>; typically
39 something like <computeroutput>C:\Documents and
40 Settings\Username\</computeroutput>.</para>
41 </listitem>
42
43 <listitem>
44 <para>On Mac OS X, this is
45 <computeroutput>/Users/username</computeroutput>.</para>
46 </listitem>
47
48 <listitem>
49 <para>On Linux and Solaris, this is
50 <computeroutput>/home/username</computeroutput>.</para>
51 </listitem>
52 </itemizedlist>
53
54 <para>For simplicity, we will abbreviate this as
55 <computeroutput>$HOME</computeroutput> below. Using that convention, the
56 common folder for all virtual machines is
57 <computeroutput>$HOME/VirtualBox VMs</computeroutput>.</para>
58
59 <para>As an example, when you create a virtual machine called "Example
60 VM", you will find that VirtualBox creates<orderedlist>
61 <listitem>
62 <para>the folder <computeroutput>$HOME/VirtualBox VMs/Example
63 VM/</computeroutput> and, in that folder,</para>
64 </listitem>
65
66 <listitem>
67 <para>the settings file <computeroutput>Example
68 VM.vbox</computeroutput> and</para>
69 </listitem>
70
71 <listitem>
72 <para>the virtual disk image <computeroutput>Example
73 VM.vdi</computeroutput>.</para>
74 </listitem>
75 </orderedlist></para>
76
77 <para>This is the default layout if you use the "Create new virtual
78 machine" wizard as described in <xref linkend="gui-createvm" />. Once
79 you start working with the VM, additional files will show up: you will
80 find log files in a subfolder called
81 <computeroutput>Logs</computeroutput>, and once you have taken
82 snapshots, they will appear in a
83 <computeroutput>Snapshots</computeroutput> subfolder. For each VM, you
84 can change the location of its snapsnots folder in the VM
85 settings.</para>
86
87 <para>You can change the default machine folder by selecting
88 "Preferences" from the "File" menu in the VirtualBox main window. Then,
89 in the window that pops up, click on the "General" tab. Alternatively,
90 use <computeroutput>VBoxManage setproperty
91 machinefolder</computeroutput>; see <xref
92 linkend="vboxmanage-setproperty" />.</para>
93 </sect2>
94
95 <sect2>
96 <title>Machines created by VirtualBox versions before 4.0</title>
97
98 <para>If you have upgraded to VirtualBox 4.0 from an earlier version of
99 VirtualBox, you probably have settings files and disks in the earlier
100 file system layout.</para>
101
102 <para>Before version 4.0, VirtualBox separated the machine settings
103 files from virtual disk images. The machine settings files had an
104 <computeroutput>.xml</computeroutput> file extension and resided in a
105 folder called "Machines" under the global VirtualBox configuration
106 directory (see the next section). So, for example, on Linux, this was
107 the hidden <computeroutput>$HOME/.VirtualBox/Machines</computeroutput>
108 directory. The default hard disks folder was called "HardDisks" and
109 resided in the <computeroutput>.VirtualBox</computeroutput> folder as
110 well. Both locations could be changed by the user in the global
111 preferences. (The concept of a "default hard disk folder" has been
112 abandoned with VirtualBox 4.0, since disk images now reside in each
113 machine's folder by default.)</para>
114
115 <para>The old layout had several severe disadvantages.<orderedlist>
116 <listitem>
117 <para>It was very difficult to move a virtual machine from one
118 host to another because the files involved did not reside in the
119 same folder. In addition, the virtual media of all machines were
120 registered with a global registry in the central VirtualBox
121 settings file
122 (<computeroutput>$HOME/.VirtualBox/VirtualBox.xml</computeroutput>).</para>
123
124 <para>To move a machine to another host, it was therefore not
125 enough to move the XML settings file and the disk images (which
126 were in different locations), but the hard disk entries from the
127 global media registry XML had to be meticulously copied as well,
128 which was close to impossible if the machine had snapshots and
129 therefore differencing images.</para>
130 </listitem>
131
132 <listitem>
133 <para>Storing virtual disk images, which can grow very large,
134 under the hidden <computeroutput>.VirtualBox</computeroutput>
135 directory (at least on Linux and Solaris hosts) made many users
136 wonder where their disk space had gone.</para>
137 </listitem>
138 </orderedlist></para>
139
140 <para>Whereas new VMs created with VirtualBox 4.0 or later will conform
141 to the new layout, for maximum compatibility, old VMs are
142 <emphasis>not</emphasis> converted to the new layout. Otherwise machine
143 settings would be irrevocably broken if a user downgraded from 4.0 back
144 to an older version of VirtualBox.</para>
145 </sect2>
146
147 <sect2>
148 <title>Global configuration data</title>
149
150 <para>In addition to the files of the virtual machines, VirtualBox
151 maintains global configuration data. On Linux and Solaris as of VirtualBox 4.3, this
152 is in the hidden directory <computeroutput>$HOME/.config/VirtualBox</computeroutput>, although <computeroutput>$HOME/.VirtualBox</computeroutput> will be used if it exists for compatibility with earlier versions; on Windows (and on Linux and Solaris with VirtualBox 4.2 and earlier) this is in <computeroutput>$HOME/.VirtualBox</computeroutput>; on a Mac it resides in
153 <computeroutput>$HOME/Library/VirtualBox</computeroutput>.</para>
154
155 <para>VirtualBox creates this configuration directory automatically if
156 necessary. Optionally, you can supply an alternate configuration
157 directory by setting the
158 <computeroutput><literal>VBOX_USER_HOME</literal></computeroutput>
159 environment variable, or additionally on Linux or Solaris by using the standard <computeroutput><literal>XDG_CONFIG_HOME</literal></computeroutput> variable. (Since the global
160 <computeroutput>VirtualBox.xml</computeroutput> settings file points to
161 all other configuration files, this allows for switching between several
162 VirtualBox configurations entirely.)</para>
163
164 <para>Most importantly, in this directory, VirtualBox stores its global
165 settings file, another XML file called
166 <computeroutput>VirtualBox.xml</computeroutput>. This includes global
167 configuration options and the list of registered virtual machines with
168 pointers to their XML settings files. (Neither the location of this file
169 nor its directory has changed with VirtualBox 4.0.)</para>
170
171 <para>Before VirtualBox 4.0, all virtual media (disk image files) were
172 also contained in a global registry in this settings file. For
173 compatibility, this media registry still exists if you upgrade
174 VirtualBox and there are media from machines which were created with a
175 version before 4.0. If you have no such machines, then there will be no
176 global media registry; with VirtualBox 4.0, each machine XML file has
177 its own media registry.</para>
178
179 <para>Also before VirtualBox 4.0, the default "Machines" folder and the
180 default "HardDisks" folder resided under the VirtualBox configuration
181 directory (e.g.
182 <computeroutput>$HOME/.VirtualBox/Machines</computeroutput> on Linux).
183 If you are upgrading from a VirtualBox version before 4.0, files in
184 these directories are not automatically moved in order not to break
185 backwards compatibility.</para>
186 </sect2>
187
188 <sect2>
189 <title>Summary of 4.0 configuration changes</title>
190
191 <table>
192 <title>ignoreme</title>
193
194 <tgroup cols="3">
195 <tbody>
196 <row>
197 <entry></entry>
198
199 <entry><emphasis role="bold">Before 4.0</emphasis></entry>
200
201 <entry><emphasis role="bold">4.0 or above</emphasis></entry>
202 </row>
203
204 <row>
205 <entry>Default machines folder</entry>
206
207 <entry><computeroutput>$HOME/.VirtualBox/Machines</computeroutput></entry>
208
209 <entry><computeroutput>$HOME/VirtualBox
210 VMs</computeroutput></entry>
211 </row>
212
213 <row>
214 <entry>Default disk image location</entry>
215
216 <entry><computeroutput>$HOME/.VirtualBox/HardDisks</computeroutput></entry>
217
218 <entry>In each machine's folder</entry>
219 </row>
220
221 <row>
222 <entry>Machine settings file extension</entry>
223
224 <entry><computeroutput>.xml</computeroutput></entry>
225
226 <entry><computeroutput>.vbox</computeroutput></entry>
227 </row>
228
229 <row>
230 <entry>Media registry</entry>
231
232 <entry>Global <computeroutput>VirtualBox.xml</computeroutput>
233 file</entry>
234
235 <entry>Each machine settings file</entry>
236 </row>
237
238 <row>
239 <entry>Media registration</entry>
240
241 <entry>Explicit open/close required</entry>
242
243 <entry>Automatic on attach</entry>
244 </row>
245 </tbody>
246 </tgroup>
247 </table>
248 </sect2>
249
250 <sect2>
251 <title>VirtualBox XML files</title>
252
253 <para>VirtualBox uses XML for both the machine settings files and the
254 global configuration file,
255 <computeroutput>VirtualBox.xml</computeroutput>.</para>
256
257 <para>All VirtualBox XML files are versioned. When a new settings file
258 is created (e.g. because a new virtual machine is created), VirtualBox
259 automatically uses the settings format of the current VirtualBox
260 version. These files may not be readable if you downgrade to an earlier
261 version of VirtualBox. However, when VirtualBox encounters a settings
262 file from an earlier version (e.g. after upgrading VirtualBox), it
263 attempts to preserve the settings format as much as possible. It will
264 only silently upgrade the settings format if the current settings cannot
265 be expressed in the old format, for example because you enabled a
266 feature that was not present in an earlier version of
267 VirtualBox.<footnote>
268 <para>As an example, before VirtualBox 3.1, it was only possible to
269 enable or disable a single DVD drive in a virtual machine. If it was
270 enabled, then it would always be visible as the secondary master of
271 the IDE controller. With VirtualBox 3.1, DVD drives can be attached
272 to arbitrary slots of arbitrary controllers, so they could be the
273 secondary slave of an IDE controller or in a SATA slot. If you have
274 a machine settings file from an earlier version and upgrade
275 VirtualBox to 3.1 and then move the DVD drive from its default
276 position, this cannot be expressed in the old settings format; the
277 XML machine file would get written in the new format, and a backup
278 file of the old format would be kept.</para>
279 </footnote> In such cases, VirtualBox backs up the old settings file
280 in the virtual machine's configuration directory. If you need to go back
281 to the earlier version of VirtualBox, then you will need to manually
282 copy these backup files back.</para>
283
284 <para>We intentionally do not document the specifications of the
285 VirtualBox XML files, as we must reserve the right to modify them in the
286 future. We therefore strongly suggest that you do not edit these files
287 manually. VirtualBox provides complete access to its configuration data
288 through its the <computeroutput>VBoxManage</computeroutput> command line
289 tool (see <xref linkend="vboxmanage" />) and its API (see <xref
290 linkend="VirtualBoxAPI" />).</para>
291 </sect2>
292 </sect1>
293
294 <sect1 id="technical-components">
295 <title>VirtualBox executables and components</title>
296
297 <para>VirtualBox was designed to be modular and flexible. When the
298 VirtualBox graphical user interface (GUI) is opened and a VM is started,
299 at least three processes are running:<orderedlist>
300 <listitem>
301 <para><computeroutput>VBoxSVC</computeroutput>, the VirtualBox
302 service process which always runs in the background. This process is
303 started automatically by the first VirtualBox client process (the
304 GUI, <computeroutput>VBoxManage</computeroutput>,
305 <computeroutput>VBoxHeadless</computeroutput>, the web service or
306 others) and exits a short time after the last client exits. The
307 service is responsible for bookkeeping, maintaining the state of all
308 VMs, and for providing communication between VirtualBox components.
309 This communication is implemented via COM/XPCOM.<note>
310 <para>When we refer to "clients" here, we mean the local clients
311 of a particular <computeroutput>VBoxSVC</computeroutput> server
312 process, not clients in a network. VirtualBox employs its own
313 client/server design to allow its processes to cooperate, but
314 all these processes run under the same user account on the host
315 operating system, and this is totally transparent to the
316 user.</para>
317 </note></para>
318 </listitem>
319
320 <listitem>
321 <para>The GUI process, <computeroutput>VirtualBox</computeroutput>,
322 a client application based on the cross-platform Qt library. When
323 started without the <computeroutput>--startvm</computeroutput>
324 option, this application acts as the VirtualBox manager, displaying
325 the VMs and their settings. It then communicates settings and state
326 changes to <computeroutput>VBoxSVC</computeroutput> and also
327 reflects changes effected through other means, e.g.,
328 <computeroutput>VBoxManage</computeroutput>.</para>
329 </listitem>
330
331 <listitem>
332 <para>If the <computeroutput>VirtualBox</computeroutput> client
333 application is started with the
334 <computeroutput>--startvm</computeroutput> argument, it loads the
335 VMM library which includes the actual hypervisor and then runs a
336 virtual machine and provides the input and output for the
337 guest.</para>
338 </listitem>
339 </orderedlist></para>
340
341 <para>Any VirtualBox front-end (client) will communicate with the service
342 process and can both control and reflect the current state. For example,
343 either the VM selector or the VM window or VBoxManage can be used to pause
344 the running VM, and other components will always reflect the changed
345 state.</para>
346
347 <para>The VirtualBox GUI application is only one of several available
348 front ends (clients). The complete list shipped with VirtualBox
349 is:<orderedlist>
350 <listitem>
351 <para><computeroutput>VirtualBox</computeroutput>, the Qt front end
352 implementing the manager and running VMs;</para>
353 </listitem>
354
355 <listitem>
356 <para><computeroutput>VBoxManage</computeroutput>, a less
357 user-friendly but more powerful alternative, described in <xref
358 linkend="vboxmanage" />.</para>
359 </listitem>
360
361 <listitem>
362 <para><computeroutput>VBoxSDL</computeroutput>, a simple graphical
363 front end based on the SDL library; see <xref
364 linkend="vboxsdl" />.</para>
365 </listitem>
366
367 <listitem>
368 <para><computeroutput>VBoxHeadless</computeroutput>, a VM front end
369 which does not directly provide any video output and keyboard/mouse
370 input, but allows redirection via VirtualBox Remote Desktop Extension;
371 see <xref linkend="vboxheadless" />.</para>
372 </listitem>
373
374 <listitem>
375 <para><computeroutput>vboxwebsrv</computeroutput>, the VirtualBox
376 web service process which allows for controlling a VirtualBox host
377 remotely. This is described in detail in the VirtualBox Software
378 Development Kit (SDK) reference; please see <xref
379 linkend="VirtualBoxAPI" /> for details.</para>
380 </listitem>
381
382 <listitem>
383 <para>The VirtualBox Python shell, a Python alternative to
384 VBoxManage. This is also described in the SDK reference.</para>
385 </listitem>
386 </orderedlist></para>
387
388 <para>Internally, VirtualBox consists of many more or less separate
389 components. You may encounter these when analyzing VirtualBox internal
390 error messages or log files. These include:</para>
391
392 <itemizedlist>
393 <listitem>
394 <para>IPRT, a portable runtime library which abstracts file access,
395 threading, string manipulation, etc. Whenever VirtualBox accesses host
396 operating features, it does so through this library for cross-platform
397 portability.</para>
398 </listitem>
399
400 <listitem>
401 <para>VMM (Virtual Machine Monitor), the heart of the
402 hypervisor.</para>
403 </listitem>
404
405 <listitem>
406 <para>EM (Execution Manager), controls execution of guest code.</para>
407 </listitem>
408
409 <listitem>
410 <para>REM (Recompiled Execution Monitor), provides software emulation
411 of CPU instructions.</para>
412 </listitem>
413
414 <listitem>
415 <para>TRPM (Trap Manager), intercepts and processes guest traps and
416 exceptions.</para>
417 </listitem>
418
419 <listitem>
420 <para>HWACCM (Hardware Acceleration Manager), provides support for
421 VT-x and AMD-V.</para>
422 </listitem>
423
424 <listitem>
425 <para>PDM (Pluggable Device Manager), an abstract interface between
426 the VMM and emulated devices which separates device implementations
427 from VMM internals and makes it easy to add new emulated devices.
428 Through PDM, third-party developers can add new virtual devices to
429 VirtualBox without having to change VirtualBox itself.</para>
430 </listitem>
431
432 <listitem>
433 <para>PGM (Page Manager), a component controlling guest paging.</para>
434 </listitem>
435
436 <listitem>
437 <para>PATM (Patch Manager), patches guest code to improve and speed up
438 software virtualization.</para>
439 </listitem>
440
441 <listitem>
442 <para>TM (Time Manager), handles timers and all aspects of time inside
443 guests.</para>
444 </listitem>
445
446 <listitem>
447 <para>CFGM (Configuration Manager), provides a tree structure which
448 holds configuration settings for the VM and all emulated
449 devices.</para>
450 </listitem>
451
452 <listitem>
453 <para>SSM (Saved State Manager), saves and loads VM state.</para>
454 </listitem>
455
456 <listitem>
457 <para>VUSB (Virtual USB), a USB layer which separates emulated USB
458 controllers from the controllers on the host and from USB devices;
459 this also enables remote USB.</para>
460 </listitem>
461
462 <listitem>
463 <para>DBGF (Debug Facility), a built-in VM debugger.</para>
464 </listitem>
465
466 <listitem>
467 <para>VirtualBox emulates a number of devices to provide the hardware
468 environment that various guests need. Most of these are standard
469 devices found in many PC compatible machines and widely supported by
470 guest operating systems. For network and storage devices in
471 particular, there are several options for the emulated devices to
472 access the underlying hardware. These devices are managed by
473 PDM.</para>
474 </listitem>
475
476 <listitem>
477 <para>Guest Additions for various guest operating systems. This is
478 code that is installed from within a virtual machine; see <xref
479 linkend="guestadditions" />.</para>
480 </listitem>
481
482 <listitem>
483 <para>The "Main" component is special: it ties all the above bits
484 together and is the only public API that VirtualBox provides. All the
485 client processes listed above use only this API and never access the
486 hypervisor components directly. As a result, third-party applications
487 that use the VirtualBox Main API can rely on the fact that it is
488 always well-tested and that all capabilities of VirtualBox are fully
489 exposed. It is this API that is described in the VirtualBox SDK
490 mentioned above (again, see <xref linkend="VirtualBoxAPI" />).</para>
491 </listitem>
492 </itemizedlist>
493 </sect1>
494
495 <sect1 id="hwvirt">
496 <title>Hardware vs. software virtualization</title>
497
498 <para>VirtualBox allows software in the virtual machine to run directly on
499 the processor of the host, but an array of complex techniques is employed
500 to intercept operations that would interfere with your host. Whenever the
501 guest attempts to do something that could be harmful to your computer and
502 its data, VirtualBox steps in and takes action. In particular, for lots of
503 hardware that the guest believes to be accessing, VirtualBox simulates a
504 certain "virtual" environment according to how you have configured a
505 virtual machine. For example, when the guest attempts to access a hard
506 disk, VirtualBox redirects these requests to whatever you have configured
507 to be the virtual machine's virtual hard disk -- normally, an image file
508 on your host.</para>
509
510 <para>Unfortunately, the x86 platform was never designed to be
511 virtualized. Detecting situations in which VirtualBox needs to take
512 control over the guest code that is executing, as described above, is
513 difficult. There are two ways in which to achieve this:<itemizedlist>
514 <listitem>
515 <para>Since 2006, Intel and AMD processors have had support for
516 so-called <emphasis role="bold">"hardware
517 virtualization"</emphasis>. This means that these processors can
518 help VirtualBox to intercept potentially dangerous operations that a
519 guest operating system may be attempting and also makes it easier to
520 present virtual hardware to a virtual machine.</para>
521
522 <para>These hardware features differ between Intel and AMD
523 processors. Intel named its technology <emphasis
524 role="bold">VT-x</emphasis>; AMD calls theirs <emphasis
525 role="bold">AMD-V</emphasis>. The Intel and AMD support for
526 virtualization is very different in detail, but not very different
527 in principle.<note>
528 <para>On many systems, the hardware virtualization features
529 first need to be enabled in the BIOS before VirtualBox can use
530 them.</para>
531 </note></para>
532 </listitem>
533
534 <listitem>
535 <para>As opposed to other virtualization software, for many usage
536 scenarios, VirtualBox does not <emphasis>require</emphasis> hardware
537 virtualization features to be present. Through sophisticated
538 techniques, VirtualBox virtualizes many guest operating systems
539 entirely in <emphasis role="bold">software</emphasis>. This means
540 that you can run virtual machines even on older processors which do
541 not support hardware virtualization.</para>
542 </listitem>
543 </itemizedlist></para>
544
545 <para>Even though VirtualBox does not always require hardware
546 virtualization, enabling it is <emphasis>required</emphasis> in the
547 following scenarios:<itemizedlist>
548 <listitem>
549 <para>Certain rare guest operating systems like OS/2 make use of
550 very esoteric processor instructions that are not supported with our
551 software virtualization. For virtual machines that are configured to
552 contain such an operating system, hardware virtualization is enabled
553 automatically.</para>
554 </listitem>
555
556 <listitem>
557 <para>VirtualBox's 64-bit guest support (added with version 2.0) and
558 multiprocessing (SMP, added with version 3.0) both require hardware
559 virtualization to be enabled. (This is not much of a limitation
560 since the vast majority of today's 64-bit and multicore CPUs ship
561 with hardware virtualization anyway; the exceptions to this rule are
562 e.g. older Intel Celeron and AMD Opteron CPUs.)</para>
563 </listitem>
564 </itemizedlist></para>
565
566 <warning>
567 <para>Do not run other hypervisors (open-source or commercial
568 virtualization products) together with VirtualBox! While several
569 hypervisors can normally be <emphasis>installed</emphasis> in parallel,
570 do not attempt to <emphasis>run</emphasis> several virtual machines from
571 competing hypervisors at the same time. VirtualBox cannot track what
572 another hypervisor is currently attempting to do on the same host, and
573 especially if several products attempt to use hardware virtualization
574 features such as VT-x, this can crash the entire host. Also, within
575 VirtualBox, you can mix software and hardware virtualization when
576 running multiple VMs. In certain cases a small performance penalty will
577 be unavoidable when mixing VT-x and software virtualization VMs. We
578 recommend not mixing virtualization modes if maximum performance and low
579 overhead are essential. This does <emphasis>not</emphasis> apply to
580 AMD-V.</para>
581 </warning>
582 </sect1>
583
584 <sect1>
585 <title>Details about software virtualization</title>
586
587 <para>Implementing virtualization on x86 CPUs with no hardware
588 virtualization support is an extraordinarily complex task because the CPU
589 architecture was not designed to be virtualized. The problems can usually
590 be solved, but at the cost of reduced performance. Thus, there is a
591 constant clash between virtualization performance and accuracy.</para>
592
593 <para>The x86 instruction set was originally designed in the 1970s and
594 underwent significant changes with the addition of protected mode in the
595 1980s with the 286 CPU architecture and then again with the Intel 386 and
596 its 32-bit architecture. Whereas the 386 did have limited virtualization
597 support for real mode operation (V86 mode, as used by the "DOS Box" of
598 Windows 3.x and OS/2 2.x), no support was provided for virtualizing the
599 entire architecture.</para>
600
601 <para>In theory, software virtualization is not overly complex. In
602 addition to the four privilege levels ("rings") provided by the hardware
603 (of which typically only two are used: ring 0 for kernel mode and ring 3
604 for user mode), one needs to differentiate between "host context" and
605 "guest context".</para>
606
607 <para>In "host context", everything is as if no hypervisor was active.
608 This might be the active mode if another application on your host has been
609 scheduled CPU time; in that case, there is a host ring 3 mode and a host
610 ring 0 mode. The hypervisor is not involved.</para>
611
612 <para>In "guest context", however, a virtual machine is active. So long as
613 the guest code is running in ring 3, this is not much of a problem since a
614 hypervisor can set up the page tables properly and run that code natively
615 on the processor. The problems mostly lie in how to intercept what the
616 guest's kernel does.</para>
617
618 <para>There are several possible solutions to these problems. One approach
619 is full software emulation, usually involving recompilation. That is, all
620 code to be run by the guest is analyzed, transformed into a form which
621 will not allow the guest to either modify or see the true state of the
622 CPU, and only then executed. This process is obviously highly complex and
623 costly in terms of performance. (VirtualBox contains a recompiler based on
624 QEMU which can be used for pure software emulation, but the recompiler is
625 only activated in special situations, described below.)</para>
626
627 <para>Another possible solution is paravirtualization, in which only
628 specially modified guest OSes are allowed to run. This way, most of the
629 hardware access is abstracted and any functions which would normally
630 access the hardware or privileged CPU state are passed on to the
631 hypervisor instead. Paravirtualization can achieve good functionality and
632 performance on standard x86 CPUs, but it can only work if the guest OS can
633 actually be modified, which is obviously not always the case.</para>
634
635 <para>VirtualBox chooses a different approach. When starting a virtual
636 machine, through its ring-0 support kernel driver, VirtualBox has set up
637 the host system so that it can run most of the guest code natively, but it
638 has inserted itself at the "bottom" of the picture. It can then assume
639 control when needed -- if a privileged instruction is executed, the guest
640 traps (in particular because an I/O register was accessed and a device
641 needs to be virtualized) or external interrupts occur. VirtualBox may then
642 handle this and either route a request to a virtual device or possibly
643 delegate handling such things to the guest or host OS. In guest context,
644 VirtualBox can therefore be in one of three states:</para>
645
646 <para><itemizedlist>
647 <listitem>
648 <para>Guest ring 3 code is run unmodified, at full speed, as much as
649 possible. The number of faults will generally be low (unless the
650 guest allows port I/O from ring 3, something we cannot do as we
651 don't want the guest to be able to access real ports). This is also
652 referred to as "raw mode", as the guest ring-3 code runs
653 unmodified.</para>
654 </listitem>
655
656 <listitem>
657 <para>For guest code in ring 0, VirtualBox employs a nasty trick: it
658 actually reconfigures the guest so that its ring-0 code is run in
659 ring 1 instead (which is normally not used in x86 operating
660 systems). As a result, when guest ring-0 code (actually running in
661 ring 1) such as a guest device driver attempts to write to an I/O
662 register or execute a privileged instruction, the VirtualBox
663 hypervisor in "real" ring 0 can take over.</para>
664 </listitem>
665
666 <listitem>
667 <para>The hypervisor (VMM) can be active. Every time a fault occurs,
668 VirtualBox looks at the offending instruction and can relegate it to
669 a virtual device or the host OS or the guest OS or run it in the
670 recompiler.</para>
671
672 <para>In particular, the recompiler is used when guest code disables
673 interrupts and VirtualBox cannot figure out when they will be
674 switched back on (in these situations, VirtualBox actually analyzes
675 the guest code using its own disassembler). Also, certain privileged
676 instructions such as LIDT need to be handled specially. Finally, any
677 real-mode or protected-mode code (e.g. BIOS code, a DOS guest, or
678 any operating system startup) is run in the recompiler
679 entirely.</para>
680 </listitem>
681 </itemizedlist></para>
682
683 <para>Unfortunately this only works to a degree. Among others, the
684 following situations require special handling:</para>
685
686 <para><orderedlist>
687 <listitem>
688 <para>Running ring 0 code in ring 1 causes a lot of additional
689 instruction faults, as ring 1 is not allowed to execute any
690 privileged instructions (of which guest's ring-0 contains plenty).
691 With each of these faults, the VMM must step in and emulate the code
692 to achieve the desired behavior. While this works, emulating
693 thousands of these faults is very expensive and severely hurts the
694 performance of the virtualized guest.</para>
695 </listitem>
696
697 <listitem>
698 <para>There are certain flaws in the implementation of ring 1 in the
699 x86 architecture that were never fixed. Certain instructions that
700 <emphasis>should</emphasis> trap in ring 1 don't. This affect for
701 example the LGDT/SGDT, LIDT/SIDT, or POPF/PUSHF instruction pairs.
702 Whereas the "load" operation is privileged and can therefore be
703 trapped, the "store" instruction always succeed. If the guest is
704 allowed to execute these, it will see the true state of the CPU, not
705 the virtualized state. The CPUID instruction also has the same
706 problem.</para>
707 </listitem>
708
709 <listitem>
710 <para>A hypervisor typically needs to reserve some portion of the
711 guest's address space (both linear address space and selectors) for
712 its own use. This is not entirely transparent to the guest OS and
713 may cause clashes.</para>
714 </listitem>
715
716 <listitem>
717 <para>The SYSENTER instruction (used for system calls) executed by
718 an application running in a guest OS always transitions to ring 0.
719 But that is where the hypervisor runs, not the guest OS. In this
720 case, the hypervisor must trap and emulate the instruction even when
721 it is not desirable.</para>
722 </listitem>
723
724 <listitem>
725 <para>The CPU segment registers contain a "hidden" descriptor cache
726 which is not software-accessible. The hypervisor cannot read, save,
727 or restore this state, but the guest OS may use it.</para>
728 </listitem>
729
730 <listitem>
731 <para>Some resources must (and can) be trapped by the hypervisor,
732 but the access is so frequent that this creates a significant
733 performance overhead. An example is the TPR (Task Priority) register
734 in 32-bit mode. Accesses to this register must be trapped by the
735 hypervisor, but certain guest operating systems (notably Windows and
736 Solaris) write this register very often, which adversely affects
737 virtualization performance.</para>
738 </listitem>
739 </orderedlist></para>
740
741 <para>To fix these performance and security issues, VirtualBox contains a
742 Code Scanning and Analysis Manager (CSAM), which disassembles guest code,
743 and the Patch Manager (PATM), which can replace it at runtime.</para>
744
745 <para>Before executing ring 0 code, CSAM scans it recursively to discover
746 problematic instructions. PATM then performs <emphasis>in-situ
747 </emphasis>patching, i.e. it replaces the instruction with a jump to
748 hypervisor memory where an integrated code generator has placed a more
749 suitable implementation. In reality, this is a very complex task as there
750 are lots of odd situations to be discovered and handled correctly. So,
751 with its current complexity, one could argue that PATM is an advanced
752 <emphasis>in-situ</emphasis> recompiler.</para>
753
754 <para>In addition, every time a fault occurs, VirtualBox analyzes the
755 offending code to determine if it is possible to patch it in order to
756 prevent it from causing more faults in the future. This approach works
757 well in practice and dramatically improves software virtualization
758 performance.</para>
759 </sect1>
760
761 <sect1>
762 <title>Details about hardware virtualization</title>
763
764 <para>With Intel VT-x, there are two distinct modes of CPU operation: VMX
765 root mode and non-root mode.<itemizedlist>
766 <listitem>
767 <para>In root mode, the CPU operates much like older generations of
768 processors without VT-x support. There are four privilege levels
769 ("rings"), and the same instruction set is supported, with the
770 addition of several virtualization specific instruction. Root mode
771 is what a host operating system without virtualization uses, and it
772 is also used by a hypervisor when virtualization is active.</para>
773 </listitem>
774
775 <listitem>
776 <para>In non-root mode, CPU operation is significantly different.
777 There are still four privilege rings and the same instruction set,
778 but a new structure called VMCS (Virtual Machine Control Structure)
779 now controls the CPU operation and determines how certain
780 instructions behave. Non-root mode is where guest systems
781 run.</para>
782 </listitem>
783 </itemizedlist></para>
784
785 <para>Switching from root mode to non-root mode is called "VM entry", the
786 switch back is "VM exit". The VMCS includes a guest and host state area
787 which is saved/restored at VM entry and exit. Most importantly, the VMCS
788 controls which guest operations will cause VM exits.</para>
789
790 <para>The VMCS provides fairly fine-grained control over what the guests
791 can and can't do. For example, a hypervisor can allow a guest to write
792 certain bits in shadowed control registers, but not others. This enables
793 efficient virtualization in cases where guests can be allowed to write
794 control bits without disrupting the hypervisor, while preventing them from
795 altering control bits over which the hypervisor needs to retain full
796 control. The VMCS also provides control over interrupt delivery and
797 exceptions.</para>
798
799 <para>Whenever an instruction or event causes a VM exit, the VMCS contains
800 information about the exit reason, often with accompanying detail. For
801 example, if a write to the CR0 register causes an exit, the offending
802 instruction is recorded, along with the fact that a write access to a
803 control register caused the exit, and information about source and
804 destination register. Thus the hypervisor can efficiently handle the
805 condition without needing advanced techniques such as CSAM and PATM
806 described above.</para>
807
808 <para>VT-x inherently avoids several of the problems which software
809 virtualization faces. The guest has its own completely separate address
810 space not shared with the hypervisor, which eliminates potential clashes.
811 Additionally, guest OS kernel code runs at privilege ring 0 in VMX
812 non-root mode, obviating the problems by running ring 0 code at less
813 privileged levels. For example the SYSENTER instruction can transition to
814 ring 0 without causing problems. Naturally, even at ring 0 in VMX non-root
815 mode, any I/O access by guest code still causes a VM exit, allowing for
816 device emulation.</para>
817
818 <para>The biggest difference between VT-x and AMD-V is that AMD-V provides
819 a more complete virtualization environment. VT-x requires the VMX non-root
820 code to run with paging enabled, which precludes hardware virtualization
821 of real-mode code and non-paged protected-mode software. This typically
822 only includes firmware and OS loaders, but nevertheless complicates VT-x
823 hypervisor implementation. AMD-V does not have this restriction.</para>
824
825 <para>Of course hardware virtualization is not perfect. Compared to
826 software virtualization, the overhead of VM exits is relatively high. This
827 causes problems for devices whose emulation requires high number of traps.
828 One example is the VGA device in 16-color modes, where not only every I/O
829 port access but also every access to the framebuffer memory must be
830 trapped.</para>
831 </sect1>
832
833 <sect1 id="nestedpaging">
834 <title>Nested paging and VPIDs</title>
835
836 <para>In addition to "plain" hardware virtualization, your processor may
837 also support additional sophisticated techniques:<footnote>
838 <para>VirtualBox 2.0 added support for AMD's nested paging; support
839 for Intel's EPT and VPIDs was added with version 2.1.</para>
840 </footnote><itemizedlist>
841 <listitem>
842 <para>A newer feature called <emphasis role="bold">"nested
843 paging"</emphasis> implements some memory management in hardware,
844 which can greatly accelerate hardware virtualization since these
845 tasks no longer need to be performed by the virtualization
846 software.</para>
847
848 <para>With nested paging, the hardware provides another level of
849 indirection when translating linear to physical addresses. Page
850 tables function as before, but linear addresses are now translated
851 to "guest physical" addresses first and not physical addresses
852 directly. A new set of paging registers now exists under the
853 traditional paging mechanism and translates from guest physical
854 addresses to host physical addresses, which are used to access
855 memory.</para>
856
857 <para>Nested paging eliminates the overhead caused by VM exits and
858 page table accesses. In essence, with nested page tables the guest
859 can handle paging without intervention from the hypervisor. Nested
860 paging thus significantly improves virtualization
861 performance.</para>
862
863 <para>On AMD processors, nested paging has been available starting
864 with the Barcelona (K10) architecture -- they call it now "rapid
865 virtualization indexing" (RVI). Intel added support for nested
866 paging, which they call "extended page tables" (EPT), with their
867 Core i7 (Nehalem) processors.</para>
868
869 <para>If nested paging is enabled, the VirtualBox hypervisor can
870 also use <emphasis role="bold">large pages</emphasis> to reduce TLB
871 usage and overhead. This can yield a performance improvement of up
872 to 5%. To enable this feature for a VM, you need to use the
873 <computeroutput>VBoxManage modifyvm
874 </computeroutput><computeroutput>--largepages</computeroutput>
875 command; see <xref linkend="vboxmanage-modifyvm" />.</para>
876 </listitem>
877
878 <listitem>
879 <para>On Intel CPUs, another hardware feature called <emphasis
880 role="bold">"Virtual Processor Identifiers" (VPIDs)</emphasis> can
881 greatly accelerate context switching by reducing the need for
882 expensive flushing of the processor's Translation Lookaside Buffers
883 (TLBs).</para>
884
885 <para>To enable these features for a VM, you need to use the
886 <computeroutput>VBoxManage modifyvm --vtxvpid</computeroutput> and
887 <computeroutput>--largepages</computeroutput> commands; see <xref
888 linkend="vboxmanage-modifyvm" />.</para>
889 </listitem>
890 </itemizedlist></para>
891 </sect1>
892</chapter>
Note: See TracBrowser for help on using the repository browser.

© 2024 Oracle Support Privacy / Do Not Sell My Info Terms of Use Trademark Policy Automated Access Etiquette