VirtualBox

source: vbox/trunk/src/VBox/VMM/PGM.cpp@ 16281

Last change on this file since 16281 was 16203, checked in by vboxsync, 16 years ago

Updates in preparation for PGM pool based paging everywhere.

  • Property svn:eol-style set to native
  • Property svn:keywords set to Id
File size: 218.6 KB
Line 
1/* $Id: PGM.cpp 16203 2009-01-23 16:36:23Z vboxsync $ */
2/** @file
3 * PGM - Page Manager and Monitor. (Mixing stuff here, not good?)
4 */
5
6/*
7 * Copyright (C) 2006-2007 Sun Microsystems, Inc.
8 *
9 * This file is part of VirtualBox Open Source Edition (OSE), as
10 * available from http://www.virtualbox.org. This file is free software;
11 * you can redistribute it and/or modify it under the terms of the GNU
12 * General Public License (GPL) as published by the Free Software
13 * Foundation, in version 2 as it comes in the "COPYING" file of the
14 * VirtualBox OSE distribution. VirtualBox OSE is distributed in the
15 * hope that it will be useful, but WITHOUT ANY WARRANTY of any kind.
16 *
17 * Please contact Sun Microsystems, Inc., 4150 Network Circle, Santa
18 * Clara, CA 95054 USA or visit http://www.sun.com if you need
19 * additional information or have any questions.
20 */
21
22
23/** @page pg_pgm PGM - The Page Manager and Monitor
24 *
25 * @see grp_pgm,
26 * @ref pg_pgm_pool,
27 * @ref pg_pgm_phys.
28 *
29 *
30 * @section sec_pgm_modes Paging Modes
31 *
32 * There are three memory contexts: Host Context (HC), Guest Context (GC)
33 * and intermediate context. When talking about paging HC can also be refered to
34 * as "host paging", and GC refered to as "shadow paging".
35 *
36 * We define three basic paging modes: 32-bit, PAE and AMD64. The host paging mode
37 * is defined by the host operating system. The mode used in the shadow paging mode
38 * depends on the host paging mode and what the mode the guest is currently in. The
39 * following relation between the two is defined:
40 *
41 * @verbatim
42 Host > 32-bit | PAE | AMD64 |
43 Guest | | | |
44 ==v================================
45 32-bit 32-bit PAE PAE
46 -------|--------|--------|--------|
47 PAE PAE PAE PAE
48 -------|--------|--------|--------|
49 AMD64 AMD64 AMD64 AMD64
50 -------|--------|--------|--------| @endverbatim
51 *
52 * All configuration except those in the diagonal (upper left) are expected to
53 * require special effort from the switcher (i.e. a bit slower).
54 *
55 *
56 *
57 *
58 * @section sec_pgm_shw The Shadow Memory Context
59 *
60 *
61 * [..]
62 *
63 * Because of guest context mappings requires PDPT and PML4 entries to allow
64 * writing on AMD64, the two upper levels will have fixed flags whatever the
65 * guest is thinking of using there. So, when shadowing the PD level we will
66 * calculate the effective flags of PD and all the higher levels. In legacy
67 * PAE mode this only applies to the PWT and PCD bits (the rest are
68 * ignored/reserved/MBZ). We will ignore those bits for the present.
69 *
70 *
71 *
72 * @section sec_pgm_int The Intermediate Memory Context
73 *
74 * The world switch goes thru an intermediate memory context which purpose it is
75 * to provide different mappings of the switcher code. All guest mappings are also
76 * present in this context.
77 *
78 * The switcher code is mapped at the same location as on the host, at an
79 * identity mapped location (physical equals virtual address), and at the
80 * hypervisor location. The identity mapped location is for when the world
81 * switches that involves disabling paging.
82 *
83 * PGM maintain page tables for 32-bit, PAE and AMD64 paging modes. This
84 * simplifies switching guest CPU mode and consistency at the cost of more
85 * code to do the work. All memory use for those page tables is located below
86 * 4GB (this includes page tables for guest context mappings).
87 *
88 *
89 * @subsection subsec_pgm_int_gc Guest Context Mappings
90 *
91 * During assignment and relocation of a guest context mapping the intermediate
92 * memory context is used to verify the new location.
93 *
94 * Guest context mappings are currently restricted to below 4GB, for reasons
95 * of simplicity. This may change when we implement AMD64 support.
96 *
97 *
98 *
99 *
100 * @section sec_pgm_misc Misc
101 *
102 * @subsection subsec_pgm_misc_diff Differences Between Legacy PAE and Long Mode PAE
103 *
104 * The differences between legacy PAE and long mode PAE are:
105 * -# PDPE bits 1, 2, 5 and 6 are defined differently. In leagcy mode they are
106 * all marked down as must-be-zero, while in long mode 1, 2 and 5 have the
107 * usual meanings while 6 is ignored (AMD). This means that upon switching to
108 * legacy PAE mode we'll have to clear these bits and when going to long mode
109 * they must be set. This applies to both intermediate and shadow contexts,
110 * however we don't need to do it for the intermediate one since we're
111 * executing with CR0.WP at that time.
112 * -# CR3 allows a 32-byte aligned address in legacy mode, while in long mode
113 * a page aligned one is required.
114 *
115 *
116 * @section sec_pgm_handlers Access Handlers
117 *
118 * Placeholder.
119 *
120 *
121 * @subsection sec_pgm_handlers_virt Virtual Access Handlers
122 *
123 * Placeholder.
124 *
125 *
126 * @subsection sec_pgm_handlers_virt Virtual Access Handlers
127 *
128 * We currently implement three types of virtual access handlers: ALL, WRITE
129 * and HYPERVISOR (WRITE). See PGMVIRTHANDLERTYPE for some more details.
130 *
131 * The HYPERVISOR access handlers is kept in a separate tree since it doesn't apply
132 * to physical pages (PGMTREES::HyperVirtHandlers) and only needs to be consulted in
133 * a special \#PF case. The ALL and WRITE are in the PGMTREES::VirtHandlers tree, the
134 * rest of this section is going to be about these handlers.
135 *
136 * We'll go thru the life cycle of a handler and try make sense of it all, don't know
137 * how successfull this is gonna be...
138 *
139 * 1. A handler is registered thru the PGMR3HandlerVirtualRegister and
140 * PGMHandlerVirtualRegisterEx APIs. We check for conflicting virtual handlers
141 * and create a new node that is inserted into the AVL tree (range key). Then
142 * a full PGM resync is flagged (clear pool, sync cr3, update virtual bit of PGMPAGE).
143 *
144 * 2. The following PGMSyncCR3/SyncCR3 operation will first make invoke HandlerVirtualUpdate.
145 *
146 * 2a. HandlerVirtualUpdate will will lookup all the pages covered by virtual handlers
147 * via the current guest CR3 and update the physical page -> virtual handler
148 * translation. Needless to say, this doesn't exactly scale very well. If any changes
149 * are detected, it will flag a virtual bit update just like we did on registration.
150 * PGMPHYS pages with changes will have their virtual handler state reset to NONE.
151 *
152 * 2b. The virtual bit update process will iterate all the pages covered by all the
153 * virtual handlers and update the PGMPAGE virtual handler state to the max of all
154 * virtual handlers on that page.
155 *
156 * 2c. Back in SyncCR3 we will now flush the entire shadow page cache to make sure
157 * we don't miss any alias mappings of the monitored pages.
158 *
159 * 2d. SyncCR3 will then proceed with syncing the CR3 table.
160 *
161 * 3. \#PF(np,read) on a page in the range. This will cause it to be synced
162 * read-only and resumed if it's a WRITE handler. If it's an ALL handler we
163 * will call the handlers like in the next step. If the physical mapping has
164 * changed we will - some time in the future - perform a handler callback
165 * (optional) and update the physical -> virtual handler cache.
166 *
167 * 4. \#PF(,write) on a page in the range. This will cause the handler to
168 * be invoked.
169 *
170 * 5. The guest invalidates the page and changes the physical backing or
171 * unmaps it. This should cause the invalidation callback to be invoked
172 * (it might not yet be 100% perfect). Exactly what happens next... is
173 * this where we mess up and end up out of sync for a while?
174 *
175 * 6. The handler is deregistered by the client via PGMHandlerVirtualDeregister.
176 * We will then set all PGMPAGEs in the physical -> virtual handler cache for
177 * this handler to NONE and trigger a full PGM resync (basically the same
178 * as int step 1). Which means 2 is executed again.
179 *
180 *
181 * @subsubsection sub_sec_pgm_handler_virt_todo TODOs
182 *
183 * There is a bunch of things that needs to be done to make the virtual handlers
184 * work 100% correctly and work more efficiently.
185 *
186 * The first bit hasn't been implemented yet because it's going to slow the
187 * whole mess down even more, and besides it seems to be working reliably for
188 * our current uses. OTOH, some of the optimizations might end up more or less
189 * implementing the missing bits, so we'll see.
190 *
191 * On the optimization side, the first thing to do is to try avoid unnecessary
192 * cache flushing. Then try team up with the shadowing code to track changes
193 * in mappings by means of access to them (shadow in), updates to shadows pages,
194 * invlpg, and shadow PT discarding (perhaps).
195 *
196 * Some idea that have popped up for optimization for current and new features:
197 * - bitmap indicating where there are virtual handlers installed.
198 * (4KB => 2**20 pages, page 2**12 => covers 32-bit address space 1:1!)
199 * - Further optimize this by min/max (needs min/max avl getters).
200 * - Shadow page table entry bit (if any left)?
201 *
202 */
203
204
205/** @page pg_pgm_phys PGM Physical Guest Memory Management
206 *
207 *
208 * Objectives:
209 * - Guest RAM over-commitment using memory ballooning,
210 * zero pages and general page sharing.
211 * - Moving or mirroring a VM onto a different physical machine.
212 *
213 *
214 * @subsection subsec_pgmPhys_Definitions Definitions
215 *
216 * Allocation chunk - A RTR0MemObjAllocPhysNC object and the tracking
217 * machinery assoicated with it.
218 *
219 *
220 *
221 *
222 * @subsection subsec_pgmPhys_AllocPage Allocating a page.
223 *
224 * Initially we map *all* guest memory to the (per VM) zero page, which
225 * means that none of the read functions will cause pages to be allocated.
226 *
227 * Exception, access bit in page tables that have been shared. This must
228 * be handled, but we must also make sure PGMGst*Modify doesn't make
229 * unnecessary modifications.
230 *
231 * Allocation points:
232 * - PGMPhysSimpleWriteGCPhys and PGMPhysWrite.
233 * - Replacing a zero page mapping at \#PF.
234 * - Replacing a shared page mapping at \#PF.
235 * - ROM registration (currently MMR3RomRegister).
236 * - VM restore (pgmR3Load).
237 *
238 * For the first three it would make sense to keep a few pages handy
239 * until we've reached the max memory commitment for the VM.
240 *
241 * For the ROM registration, we know exactly how many pages we need
242 * and will request these from ring-0. For restore, we will save
243 * the number of non-zero pages in the saved state and allocate
244 * them up front. This would allow the ring-0 component to refuse
245 * the request if the isn't sufficient memory available for VM use.
246 *
247 * Btw. for both ROM and restore allocations we won't be requiring
248 * zeroed pages as they are going to be filled instantly.
249 *
250 *
251 * @subsection subsec_pgmPhys_FreePage Freeing a page
252 *
253 * There are a few points where a page can be freed:
254 * - After being replaced by the zero page.
255 * - After being replaced by a shared page.
256 * - After being ballooned by the guest additions.
257 * - At reset.
258 * - At restore.
259 *
260 * When freeing one or more pages they will be returned to the ring-0
261 * component and replaced by the zero page.
262 *
263 * The reasoning for clearing out all the pages on reset is that it will
264 * return us to the exact same state as on power on, and may thereby help
265 * us reduce the memory load on the system. Further it might have a
266 * (temporary) positive influence on memory fragmentation (@see subsec_pgmPhys_Fragmentation).
267 *
268 * On restore, as mention under the allocation topic, pages should be
269 * freed / allocated depending on how many is actually required by the
270 * new VM state. The simplest approach is to do like on reset, and free
271 * all non-ROM pages and then allocate what we need.
272 *
273 * A measure to prevent some fragmentation, would be to let each allocation
274 * chunk have some affinity towards the VM having allocated the most pages
275 * from it. Also, try make sure to allocate from allocation chunks that
276 * are almost full. Admittedly, both these measures might work counter to
277 * our intentions and its probably not worth putting a lot of effort,
278 * cpu time or memory into this.
279 *
280 *
281 * @subsection subsec_pgmPhys_SharePage Sharing a page
282 *
283 * The basic idea is that there there will be a idle priority kernel
284 * thread walking the non-shared VM pages hashing them and looking for
285 * pages with the same checksum. If such pages are found, it will compare
286 * them byte-by-byte to see if they actually are identical. If found to be
287 * identical it will allocate a shared page, copy the content, check that
288 * the page didn't change while doing this, and finally request both the
289 * VMs to use the shared page instead. If the page is all zeros (special
290 * checksum and byte-by-byte check) it will request the VM that owns it
291 * to replace it with the zero page.
292 *
293 * To make this efficient, we will have to make sure not to try share a page
294 * that will change its contents soon. This part requires the most work.
295 * A simple idea would be to request the VM to write monitor the page for
296 * a while to make sure it isn't modified any time soon. Also, it may
297 * make sense to skip pages that are being write monitored since this
298 * information is readily available to the thread if it works on the
299 * per-VM guest memory structures (presently called PGMRAMRANGE).
300 *
301 *
302 * @subsection subsec_pgmPhys_Fragmentation Fragmentation Concerns and Counter Measures
303 *
304 * The pages are organized in allocation chunks in ring-0, this is a necessity
305 * if we wish to have an OS agnostic approach to this whole thing. (On Linux we
306 * could easily work on a page-by-page basis if we liked. Whether this is possible
307 * or efficient on NT I don't quite know.) Fragmentation within these chunks may
308 * become a problem as part of the idea here is that we wish to return memory to
309 * the host system.
310 *
311 * For instance, starting two VMs at the same time, they will both allocate the
312 * guest memory on-demand and if permitted their page allocations will be
313 * intermixed. Shut down one of the two VMs and it will be difficult to return
314 * any memory to the host system because the page allocation for the two VMs are
315 * mixed up in the same allocation chunks.
316 *
317 * To further complicate matters, when pages are freed because they have been
318 * ballooned or become shared/zero the whole idea is that the page is supposed
319 * to be reused by another VM or returned to the host system. This will cause
320 * allocation chunks to contain pages belonging to different VMs and prevent
321 * returning memory to the host when one of those VM shuts down.
322 *
323 * The only way to really deal with this problem is to move pages. This can
324 * either be done at VM shutdown and or by the idle priority worker thread
325 * that will be responsible for finding sharable/zero pages. The mechanisms
326 * involved for coercing a VM to move a page (or to do it for it) will be
327 * the same as when telling it to share/zero a page.
328 *
329 *
330 * @subsection subsec_pgmPhys_Tracking Tracking Structures And Their Cost
331 *
332 * There's a difficult balance between keeping the per-page tracking structures
333 * (global and guest page) easy to use and keeping them from eating too much
334 * memory. We have limited virtual memory resources available when operating in
335 * 32-bit kernel space (on 64-bit there'll it's quite a different story). The
336 * tracking structures will be attemted designed such that we can deal with up
337 * to 32GB of memory on a 32-bit system and essentially unlimited on 64-bit ones.
338 *
339 *
340 * @subsubsection subsubsec_pgmPhys_Tracking_Kernel Kernel Space
341 *
342 * @see pg_GMM
343 *
344 * @subsubsection subsubsec_pgmPhys_Tracking_PerVM Per-VM
345 *
346 * Fixed info is the physical address of the page (HCPhys) and the page id
347 * (described above). Theoretically we'll need 48(-12) bits for the HCPhys part.
348 * Today we've restricting ourselves to 40(-12) bits because this is the current
349 * restrictions of all AMD64 implementations (I think Barcelona will up this
350 * to 48(-12) bits, not that it really matters) and I needed the bits for
351 * tracking mappings of a page. 48-12 = 36. That leaves 28 bits, which means a
352 * decent range for the page id: 2^(28+12) = 1024TB.
353 *
354 * In additions to these, we'll have to keep maintaining the page flags as we
355 * currently do. Although it wouldn't harm to optimize these quite a bit, like
356 * for instance the ROM shouldn't depend on having a write handler installed
357 * in order for it to become read-only. A RO/RW bit should be considered so
358 * that the page syncing code doesn't have to mess about checking multiple
359 * flag combinations (ROM || RW handler || write monitored) in order to
360 * figure out how to setup a shadow PTE. But this of course, is second
361 * priority at present. Current this requires 12 bits, but could probably
362 * be optimized to ~8.
363 *
364 * Then there's the 24 bits used to track which shadow page tables are
365 * currently mapping a page for the purpose of speeding up physical
366 * access handlers, and thereby the page pool cache. More bit for this
367 * purpose wouldn't hurt IIRC.
368 *
369 * Then there is a new bit in which we need to record what kind of page
370 * this is, shared, zero, normal or write-monitored-normal. This'll
371 * require 2 bits. One bit might be needed for indicating whether a
372 * write monitored page has been written to. And yet another one or
373 * two for tracking migration status. 3-4 bits total then.
374 *
375 * Whatever is left will can be used to record the sharabilitiy of a
376 * page. The page checksum will not be stored in the per-VM table as
377 * the idle thread will not be permitted to do modifications to it.
378 * It will instead have to keep its own working set of potentially
379 * shareable pages and their check sums and stuff.
380 *
381 * For the present we'll keep the current packing of the
382 * PGMRAMRANGE::aHCPhys to keep the changes simple, only of course,
383 * we'll have to change it to a struct with a total of 128-bits at
384 * our disposal.
385 *
386 * The initial layout will be like this:
387 * @verbatim
388 RTHCPHYS HCPhys; The current stuff.
389 63:40 Current shadow PT tracking stuff.
390 39:12 The physical page frame number.
391 11:0 The current flags.
392 uint32_t u28PageId : 28; The page id.
393 uint32_t u2State : 2; The page state { zero, shared, normal, write monitored }.
394 uint32_t fWrittenTo : 1; Whether a write monitored page was written to.
395 uint32_t u1Reserved : 1; Reserved for later.
396 uint32_t u32Reserved; Reserved for later, mostly sharing stats.
397 @endverbatim
398 *
399 * The final layout will be something like this:
400 * @verbatim
401 RTHCPHYS HCPhys; The current stuff.
402 63:48 High page id (12+).
403 47:12 The physical page frame number.
404 11:0 Low page id.
405 uint32_t fReadOnly : 1; Whether it's readonly page (rom or monitored in some way).
406 uint32_t u3Type : 3; The page type {RESERVED, MMIO, MMIO2, ROM, shadowed ROM, RAM}.
407 uint32_t u2PhysMon : 2; Physical access handler type {none, read, write, all}.
408 uint32_t u2VirtMon : 2; Virtual access handler type {none, read, write, all}..
409 uint32_t u2State : 2; The page state { zero, shared, normal, write monitored }.
410 uint32_t fWrittenTo : 1; Whether a write monitored page was written to.
411 uint32_t u20Reserved : 20; Reserved for later, mostly sharing stats.
412 uint32_t u32Tracking; The shadow PT tracking stuff, roughly.
413 @endverbatim
414 *
415 * Cost wise, this means we'll double the cost for guest memory. There isn't anyway
416 * around that I'm afraid. It means that the cost of dealing out 32GB of memory
417 * to one or more VMs is: (32GB >> PAGE_SHIFT) * 16 bytes, or 128MBs. Or another
418 * example, the VM heap cost when assigning 1GB to a VM will be: 4MB.
419 *
420 * A couple of cost examples for the total cost per-VM + kernel.
421 * 32-bit Windows and 32-bit linux:
422 * 1GB guest ram, 256K pages: 4MB + 2MB(+) = 6MB
423 * 4GB guest ram, 1M pages: 16MB + 8MB(+) = 24MB
424 * 32GB guest ram, 8M pages: 128MB + 64MB(+) = 192MB
425 * 64-bit Windows and 64-bit linux:
426 * 1GB guest ram, 256K pages: 4MB + 3MB(+) = 7MB
427 * 4GB guest ram, 1M pages: 16MB + 12MB(+) = 28MB
428 * 32GB guest ram, 8M pages: 128MB + 96MB(+) = 224MB
429 *
430 * UPDATE - 2007-09-27:
431 * Will need a ballooned flag/state too because we cannot
432 * trust the guest 100% and reporting the same page as ballooned more
433 * than once will put the GMM off balance.
434 *
435 *
436 * @subsection subsec_pgmPhys_Serializing Serializing Access
437 *
438 * Initially, we'll try a simple scheme:
439 *
440 * - The per-VM RAM tracking structures (PGMRAMRANGE) is only modified
441 * by the EMT thread of that VM while in the pgm critsect.
442 * - Other threads in the VM process that needs to make reliable use of
443 * the per-VM RAM tracking structures will enter the critsect.
444 * - No process external thread or kernel thread will ever try enter
445 * the pgm critical section, as that just won't work.
446 * - The idle thread (and similar threads) doesn't not need 100% reliable
447 * data when performing it tasks as the EMT thread will be the one to
448 * do the actual changes later anyway. So, as long as it only accesses
449 * the main ram range, it can do so by somehow preventing the VM from
450 * being destroyed while it works on it...
451 *
452 * - The over-commitment management, including the allocating/freeing
453 * chunks, is serialized by a ring-0 mutex lock (a fast one since the
454 * more mundane mutex implementation is broken on Linux).
455 * - A separeate mutex is protecting the set of allocation chunks so
456 * that pages can be shared or/and freed up while some other VM is
457 * allocating more chunks. This mutex can be take from under the other
458 * one, but not the otherway around.
459 *
460 *
461 * @subsection subsec_pgmPhys_Request VM Request interface
462 *
463 * When in ring-0 it will become necessary to send requests to a VM so it can
464 * for instance move a page while defragmenting during VM destroy. The idle
465 * thread will make use of this interface to request VMs to setup shared
466 * pages and to perform write monitoring of pages.
467 *
468 * I would propose an interface similar to the current VMReq interface, similar
469 * in that it doesn't require locking and that the one sending the request may
470 * wait for completion if it wishes to. This shouldn't be very difficult to
471 * realize.
472 *
473 * The requests themselves are also pretty simple. They are basically:
474 * -# Check that some precondition is still true.
475 * -# Do the update.
476 * -# Update all shadow page tables involved with the page.
477 *
478 * The 3rd step is identical to what we're already doing when updating a
479 * physical handler, see pgmHandlerPhysicalSetRamFlagsAndFlushShadowPTs.
480 *
481 *
482 *
483 * @section sec_pgmPhys_MappingCaches Mapping Caches
484 *
485 * In order to be able to map in and out memory and to be able to support
486 * guest with more RAM than we've got virtual address space, we'll employing
487 * a mapping cache. There is already a tiny one for GC (see PGMGCDynMapGCPageEx)
488 * and we'll create a similar one for ring-0 unless we decide to setup a dedicate
489 * memory context for the HWACCM execution.
490 *
491 *
492 * @subsection subsec_pgmPhys_MappingCaches_R3 Ring-3
493 *
494 * We've considered implementing the ring-3 mapping cache page based but found
495 * that this was bother some when one had to take into account TLBs+SMP and
496 * portability (missing the necessary APIs on several platforms). There were
497 * also some performance concerns with this approach which hadn't quite been
498 * worked out.
499 *
500 * Instead, we'll be mapping allocation chunks into the VM process. This simplifies
501 * matters greatly quite a bit since we don't need to invent any new ring-0 stuff,
502 * only some minor RTR0MEMOBJ mapping stuff. The main concern here is that mapping
503 * compared to the previous idea is that mapping or unmapping a 1MB chunk is more
504 * costly than a single page, although how much more costly is uncertain. We'll
505 * try address this by using a very big cache, preferably bigger than the actual
506 * VM RAM size if possible. The current VM RAM sizes should give some idea for
507 * 32-bit boxes, while on 64-bit we can probably get away with employing an
508 * unlimited cache.
509 *
510 * The cache have to parts, as already indicated, the ring-3 side and the
511 * ring-0 side.
512 *
513 * The ring-0 will be tied to the page allocator since it will operate on the
514 * memory objects it contains. It will therefore require the first ring-0 mutex
515 * discussed in @ref subsec_pgmPhys_Serializing. We
516 * some double house keeping wrt to who has mapped what I think, since both
517 * VMMR0.r0 and RTR0MemObj will keep track of mapping relataions
518 *
519 * The ring-3 part will be protected by the pgm critsect. For simplicity, we'll
520 * require anyone that desires to do changes to the mapping cache to do that
521 * from within this critsect. Alternatively, we could employ a separate critsect
522 * for serializing changes to the mapping cache as this would reduce potential
523 * contention with other threads accessing mappings unrelated to the changes
524 * that are in process. We can see about this later, contention will show
525 * up in the statistics anyway, so it'll be simple to tell.
526 *
527 * The organization of the ring-3 part will be very much like how the allocation
528 * chunks are organized in ring-0, that is in an AVL tree by chunk id. To avoid
529 * having to walk the tree all the time, we'll have a couple of lookaside entries
530 * like in we do for I/O ports and MMIO in IOM.
531 *
532 * The simplified flow of a PGMPhysRead/Write function:
533 * -# Enter the PGM critsect.
534 * -# Lookup GCPhys in the ram ranges and get the Page ID.
535 * -# Calc the Allocation Chunk ID from the Page ID.
536 * -# Check the lookaside entries and then the AVL tree for the Chunk ID.
537 * If not found in cache:
538 * -# Call ring-0 and request it to be mapped and supply
539 * a chunk to be unmapped if the cache is maxed out already.
540 * -# Insert the new mapping into the AVL tree (id + R3 address).
541 * -# Update the relevant lookaside entry and return the mapping address.
542 * -# Do the read/write according to monitoring flags and everything.
543 * -# Leave the critsect.
544 *
545 *
546 * @section sec_pgmPhys_Fallback Fallback
547 *
548 * Current all the "second tier" hosts will not support the RTR0MemObjAllocPhysNC
549 * API and thus require a fallback.
550 *
551 * So, when RTR0MemObjAllocPhysNC returns VERR_NOT_SUPPORTED the page allocator
552 * will return to the ring-3 caller (and later ring-0) and asking it to seed
553 * the page allocator with some fresh pages (VERR_GMM_SEED_ME). Ring-3 will
554 * then perform an SUPPageAlloc(cbChunk >> PAGE_SHIFT) call and make a
555 * "SeededAllocPages" call to ring-0.
556 *
557 * The first time ring-0 sees the VERR_NOT_SUPPORTED failure it will disable
558 * all page sharing (zero page detection will continue). It will also force
559 * all allocations to come from the VM which seeded the page. Both these
560 * measures are taken to make sure that there will never be any need for
561 * mapping anything into ring-3 - everything will be mapped already.
562 *
563 * Whether we'll continue to use the current MM locked memory management
564 * for this I don't quite know (I'd prefer not to and just ditch that all
565 * togther), we'll see what's simplest to do.
566 *
567 *
568 *
569 * @section sec_pgmPhys_Changes Changes
570 *
571 * Breakdown of the changes involved?
572 */
573
574
575/** Saved state data unit version. */
576#define PGM_SAVED_STATE_VERSION 6
577
578/*******************************************************************************
579* Header Files *
580*******************************************************************************/
581#define LOG_GROUP LOG_GROUP_PGM
582#include <VBox/dbgf.h>
583#include <VBox/pgm.h>
584#include <VBox/cpum.h>
585#include <VBox/iom.h>
586#include <VBox/sup.h>
587#include <VBox/mm.h>
588#include <VBox/em.h>
589#include <VBox/stam.h>
590#include <VBox/rem.h>
591#include <VBox/dbgf.h>
592#include <VBox/rem.h>
593#include <VBox/selm.h>
594#include <VBox/ssm.h>
595#include "PGMInternal.h"
596#include <VBox/vm.h>
597#include <VBox/dbg.h>
598#include <VBox/hwaccm.h>
599
600#include <iprt/assert.h>
601#include <iprt/alloc.h>
602#include <iprt/asm.h>
603#include <iprt/thread.h>
604#include <iprt/string.h>
605#ifdef DEBUG_bird
606# include <iprt/env.h>
607#endif
608#include <VBox/param.h>
609#include <VBox/err.h>
610
611
612
613/*******************************************************************************
614* Internal Functions *
615*******************************************************************************/
616static int pgmR3InitPaging(PVM pVM);
617static DECLCALLBACK(void) pgmR3PhysInfo(PVM pVM, PCDBGFINFOHLP pHlp, const char *pszArgs);
618static DECLCALLBACK(void) pgmR3InfoMode(PVM pVM, PCDBGFINFOHLP pHlp, const char *pszArgs);
619static DECLCALLBACK(void) pgmR3InfoCr3(PVM pVM, PCDBGFINFOHLP pHlp, const char *pszArgs);
620static DECLCALLBACK(int) pgmR3RelocatePhysHandler(PAVLROGCPHYSNODECORE pNode, void *pvUser);
621static DECLCALLBACK(int) pgmR3RelocateVirtHandler(PAVLROGCPTRNODECORE pNode, void *pvUser);
622static DECLCALLBACK(int) pgmR3RelocateHyperVirtHandler(PAVLROGCPTRNODECORE pNode, void *pvUser);
623#ifdef VBOX_STRICT
624static DECLCALLBACK(void) pgmR3ResetNoMorePhysWritesFlag(PVM pVM, VMSTATE enmState, VMSTATE enmOldState, void *pvUser);
625#endif
626static DECLCALLBACK(int) pgmR3Save(PVM pVM, PSSMHANDLE pSSM);
627static DECLCALLBACK(int) pgmR3Load(PVM pVM, PSSMHANDLE pSSM, uint32_t u32Version);
628static int pgmR3ModeDataInit(PVM pVM, bool fResolveGCAndR0);
629static void pgmR3ModeDataSwitch(PVM pVM, PGMMODE enmShw, PGMMODE enmGst);
630static PGMMODE pgmR3CalcShadowMode(PVM pVM, PGMMODE enmGuestMode, SUPPAGINGMODE enmHostMode, PGMMODE enmShadowMode, VMMSWITCHER *penmSwitcher);
631
632#ifdef VBOX_WITH_STATISTICS
633static void pgmR3InitStats(PVM pVM);
634#endif
635
636#ifdef VBOX_WITH_DEBUGGER
637/** @todo all but the two last commands must be converted to 'info'. */
638static DECLCALLBACK(int) pgmR3CmdRam(PCDBGCCMD pCmd, PDBGCCMDHLP pCmdHlp, PVM pVM, PCDBGCVAR paArgs, unsigned cArgs, PDBGCVAR pResult);
639static DECLCALLBACK(int) pgmR3CmdMap(PCDBGCCMD pCmd, PDBGCCMDHLP pCmdHlp, PVM pVM, PCDBGCVAR paArgs, unsigned cArgs, PDBGCVAR pResult);
640static DECLCALLBACK(int) pgmR3CmdSync(PCDBGCCMD pCmd, PDBGCCMDHLP pCmdHlp, PVM pVM, PCDBGCVAR paArgs, unsigned cArgs, PDBGCVAR pResult);
641static DECLCALLBACK(int) pgmR3CmdSyncAlways(PCDBGCCMD pCmd, PDBGCCMDHLP pCmdHlp, PVM pVM, PCDBGCVAR paArgs, unsigned cArgs, PDBGCVAR pResult);
642# ifdef VBOX_STRICT
643static DECLCALLBACK(int) pgmR3CmdAssertCR3(PCDBGCCMD pCmd, PDBGCCMDHLP pCmdHlp, PVM pVM, PCDBGCVAR paArgs, unsigned cArgs, PDBGCVAR pResult);
644# endif
645#endif
646
647
648/*******************************************************************************
649* Global Variables *
650*******************************************************************************/
651#ifdef VBOX_WITH_DEBUGGER
652/** Command descriptors. */
653static const DBGCCMD g_aCmds[] =
654{
655 /* pszCmd, cArgsMin, cArgsMax, paArgDesc, cArgDescs, pResultDesc, fFlags, pfnHandler pszSyntax, ....pszDescription */
656 { "pgmram", 0, 0, NULL, 0, NULL, 0, pgmR3CmdRam, "", "Display the ram ranges." },
657 { "pgmmap", 0, 0, NULL, 0, NULL, 0, pgmR3CmdMap, "", "Display the mapping ranges." },
658 { "pgmsync", 0, 0, NULL, 0, NULL, 0, pgmR3CmdSync, "", "Sync the CR3 page." },
659#ifdef VBOX_STRICT
660 { "pgmassertcr3", 0, 0, NULL, 0, NULL, 0, pgmR3CmdAssertCR3, "", "Check the shadow CR3 mapping." },
661#endif
662 { "pgmsyncalways", 0, 0, NULL, 0, NULL, 0, pgmR3CmdSyncAlways, "", "Toggle permanent CR3 syncing." },
663};
664#endif
665
666
667
668
669/*
670 * Shadow - 32-bit mode
671 */
672#define PGM_SHW_TYPE PGM_TYPE_32BIT
673#define PGM_SHW_NAME(name) PGM_SHW_NAME_32BIT(name)
674#define PGM_SHW_NAME_RC_STR(name) PGM_SHW_NAME_RC_32BIT_STR(name)
675#define PGM_SHW_NAME_R0_STR(name) PGM_SHW_NAME_R0_32BIT_STR(name)
676#include "PGMShw.h"
677
678/* Guest - real mode */
679#define PGM_GST_TYPE PGM_TYPE_REAL
680#define PGM_GST_NAME(name) PGM_GST_NAME_REAL(name)
681#define PGM_GST_NAME_RC_STR(name) PGM_GST_NAME_RC_REAL_STR(name)
682#define PGM_GST_NAME_R0_STR(name) PGM_GST_NAME_R0_REAL_STR(name)
683#define PGM_BTH_NAME(name) PGM_BTH_NAME_32BIT_REAL(name)
684#define PGM_BTH_NAME_RC_STR(name) PGM_BTH_NAME_RC_32BIT_REAL_STR(name)
685#define PGM_BTH_NAME_R0_STR(name) PGM_BTH_NAME_R0_32BIT_REAL_STR(name)
686#define BTH_PGMPOOLKIND_PT_FOR_PT PGMPOOLKIND_32BIT_PT_FOR_PHYS
687#include "PGMGst.h"
688#include "PGMBth.h"
689#undef BTH_PGMPOOLKIND_PT_FOR_PT
690#undef PGM_BTH_NAME
691#undef PGM_BTH_NAME_RC_STR
692#undef PGM_BTH_NAME_R0_STR
693#undef PGM_GST_TYPE
694#undef PGM_GST_NAME
695#undef PGM_GST_NAME_RC_STR
696#undef PGM_GST_NAME_R0_STR
697
698/* Guest - protected mode */
699#define PGM_GST_TYPE PGM_TYPE_PROT
700#define PGM_GST_NAME(name) PGM_GST_NAME_PROT(name)
701#define PGM_GST_NAME_RC_STR(name) PGM_GST_NAME_RC_PROT_STR(name)
702#define PGM_GST_NAME_R0_STR(name) PGM_GST_NAME_R0_PROT_STR(name)
703#define PGM_BTH_NAME(name) PGM_BTH_NAME_32BIT_PROT(name)
704#define PGM_BTH_NAME_RC_STR(name) PGM_BTH_NAME_RC_32BIT_PROT_STR(name)
705#define PGM_BTH_NAME_R0_STR(name) PGM_BTH_NAME_R0_32BIT_PROT_STR(name)
706#define BTH_PGMPOOLKIND_PT_FOR_PT PGMPOOLKIND_32BIT_PT_FOR_PHYS
707#include "PGMGst.h"
708#include "PGMBth.h"
709#undef BTH_PGMPOOLKIND_PT_FOR_PT
710#undef PGM_BTH_NAME
711#undef PGM_BTH_NAME_RC_STR
712#undef PGM_BTH_NAME_R0_STR
713#undef PGM_GST_TYPE
714#undef PGM_GST_NAME
715#undef PGM_GST_NAME_RC_STR
716#undef PGM_GST_NAME_R0_STR
717
718/* Guest - 32-bit mode */
719#define PGM_GST_TYPE PGM_TYPE_32BIT
720#define PGM_GST_NAME(name) PGM_GST_NAME_32BIT(name)
721#define PGM_GST_NAME_RC_STR(name) PGM_GST_NAME_RC_32BIT_STR(name)
722#define PGM_GST_NAME_R0_STR(name) PGM_GST_NAME_R0_32BIT_STR(name)
723#define PGM_BTH_NAME(name) PGM_BTH_NAME_32BIT_32BIT(name)
724#define PGM_BTH_NAME_RC_STR(name) PGM_BTH_NAME_RC_32BIT_32BIT_STR(name)
725#define PGM_BTH_NAME_R0_STR(name) PGM_BTH_NAME_R0_32BIT_32BIT_STR(name)
726#define BTH_PGMPOOLKIND_PT_FOR_PT PGMPOOLKIND_32BIT_PT_FOR_32BIT_PT
727#define BTH_PGMPOOLKIND_PT_FOR_BIG PGMPOOLKIND_32BIT_PT_FOR_32BIT_4MB
728#include "PGMGst.h"
729#include "PGMBth.h"
730#undef BTH_PGMPOOLKIND_PT_FOR_BIG
731#undef BTH_PGMPOOLKIND_PT_FOR_PT
732#undef PGM_BTH_NAME
733#undef PGM_BTH_NAME_RC_STR
734#undef PGM_BTH_NAME_R0_STR
735#undef PGM_GST_TYPE
736#undef PGM_GST_NAME
737#undef PGM_GST_NAME_RC_STR
738#undef PGM_GST_NAME_R0_STR
739
740#undef PGM_SHW_TYPE
741#undef PGM_SHW_NAME
742#undef PGM_SHW_NAME_RC_STR
743#undef PGM_SHW_NAME_R0_STR
744
745
746/*
747 * Shadow - PAE mode
748 */
749#define PGM_SHW_TYPE PGM_TYPE_PAE
750#define PGM_SHW_NAME(name) PGM_SHW_NAME_PAE(name)
751#define PGM_SHW_NAME_RC_STR(name) PGM_SHW_NAME_RC_PAE_STR(name)
752#define PGM_SHW_NAME_R0_STR(name) PGM_SHW_NAME_R0_PAE_STR(name)
753#define PGM_BTH_NAME(name) PGM_BTH_NAME_PAE_REAL(name)
754#include "PGMShw.h"
755
756/* Guest - real mode */
757#define PGM_GST_TYPE PGM_TYPE_REAL
758#define PGM_GST_NAME(name) PGM_GST_NAME_REAL(name)
759#define PGM_GST_NAME_RC_STR(name) PGM_GST_NAME_RC_REAL_STR(name)
760#define PGM_GST_NAME_R0_STR(name) PGM_GST_NAME_R0_REAL_STR(name)
761#define PGM_BTH_NAME(name) PGM_BTH_NAME_PAE_REAL(name)
762#define PGM_BTH_NAME_RC_STR(name) PGM_BTH_NAME_RC_PAE_REAL_STR(name)
763#define PGM_BTH_NAME_R0_STR(name) PGM_BTH_NAME_R0_PAE_REAL_STR(name)
764#define BTH_PGMPOOLKIND_PT_FOR_PT PGMPOOLKIND_PAE_PT_FOR_PHYS
765#include "PGMBth.h"
766#undef BTH_PGMPOOLKIND_PT_FOR_PT
767#undef PGM_BTH_NAME
768#undef PGM_BTH_NAME_RC_STR
769#undef PGM_BTH_NAME_R0_STR
770#undef PGM_GST_TYPE
771#undef PGM_GST_NAME
772#undef PGM_GST_NAME_RC_STR
773#undef PGM_GST_NAME_R0_STR
774
775/* Guest - protected mode */
776#define PGM_GST_TYPE PGM_TYPE_PROT
777#define PGM_GST_NAME(name) PGM_GST_NAME_PROT(name)
778#define PGM_GST_NAME_RC_STR(name) PGM_GST_NAME_RC_PROT_STR(name)
779#define PGM_GST_NAME_R0_STR(name) PGM_GST_NAME_R0_PROT_STR(name)
780#define PGM_BTH_NAME(name) PGM_BTH_NAME_PAE_PROT(name)
781#define PGM_BTH_NAME_RC_STR(name) PGM_BTH_NAME_RC_PAE_PROT_STR(name)
782#define PGM_BTH_NAME_R0_STR(name) PGM_BTH_NAME_R0_PAE_PROT_STR(name)
783#define BTH_PGMPOOLKIND_PT_FOR_PT PGMPOOLKIND_PAE_PT_FOR_PHYS
784#include "PGMBth.h"
785#undef BTH_PGMPOOLKIND_PT_FOR_PT
786#undef PGM_BTH_NAME
787#undef PGM_BTH_NAME_RC_STR
788#undef PGM_BTH_NAME_R0_STR
789#undef PGM_GST_TYPE
790#undef PGM_GST_NAME
791#undef PGM_GST_NAME_RC_STR
792#undef PGM_GST_NAME_R0_STR
793
794/* Guest - 32-bit mode */
795#define PGM_GST_TYPE PGM_TYPE_32BIT
796#define PGM_GST_NAME(name) PGM_GST_NAME_32BIT(name)
797#define PGM_GST_NAME_RC_STR(name) PGM_GST_NAME_RC_32BIT_STR(name)
798#define PGM_GST_NAME_R0_STR(name) PGM_GST_NAME_R0_32BIT_STR(name)
799#define PGM_BTH_NAME(name) PGM_BTH_NAME_PAE_32BIT(name)
800#define PGM_BTH_NAME_RC_STR(name) PGM_BTH_NAME_RC_PAE_32BIT_STR(name)
801#define PGM_BTH_NAME_R0_STR(name) PGM_BTH_NAME_R0_PAE_32BIT_STR(name)
802#define BTH_PGMPOOLKIND_PT_FOR_PT PGMPOOLKIND_PAE_PT_FOR_32BIT_PT
803#define BTH_PGMPOOLKIND_PT_FOR_BIG PGMPOOLKIND_PAE_PT_FOR_32BIT_4MB
804#include "PGMBth.h"
805#undef BTH_PGMPOOLKIND_PT_FOR_BIG
806#undef BTH_PGMPOOLKIND_PT_FOR_PT
807#undef PGM_BTH_NAME
808#undef PGM_BTH_NAME_RC_STR
809#undef PGM_BTH_NAME_R0_STR
810#undef PGM_GST_TYPE
811#undef PGM_GST_NAME
812#undef PGM_GST_NAME_RC_STR
813#undef PGM_GST_NAME_R0_STR
814
815/* Guest - PAE mode */
816#define PGM_GST_TYPE PGM_TYPE_PAE
817#define PGM_GST_NAME(name) PGM_GST_NAME_PAE(name)
818#define PGM_GST_NAME_RC_STR(name) PGM_GST_NAME_RC_PAE_STR(name)
819#define PGM_GST_NAME_R0_STR(name) PGM_GST_NAME_R0_PAE_STR(name)
820#define PGM_BTH_NAME(name) PGM_BTH_NAME_PAE_PAE(name)
821#define PGM_BTH_NAME_RC_STR(name) PGM_BTH_NAME_RC_PAE_PAE_STR(name)
822#define PGM_BTH_NAME_R0_STR(name) PGM_BTH_NAME_R0_PAE_PAE_STR(name)
823#define BTH_PGMPOOLKIND_PT_FOR_PT PGMPOOLKIND_PAE_PT_FOR_PAE_PT
824#define BTH_PGMPOOLKIND_PT_FOR_BIG PGMPOOLKIND_PAE_PT_FOR_PAE_2MB
825#include "PGMGst.h"
826#include "PGMBth.h"
827#undef BTH_PGMPOOLKIND_PT_FOR_BIG
828#undef BTH_PGMPOOLKIND_PT_FOR_PT
829#undef PGM_BTH_NAME
830#undef PGM_BTH_NAME_RC_STR
831#undef PGM_BTH_NAME_R0_STR
832#undef PGM_GST_TYPE
833#undef PGM_GST_NAME
834#undef PGM_GST_NAME_RC_STR
835#undef PGM_GST_NAME_R0_STR
836
837#undef PGM_SHW_TYPE
838#undef PGM_SHW_NAME
839#undef PGM_SHW_NAME_RC_STR
840#undef PGM_SHW_NAME_R0_STR
841
842
843/*
844 * Shadow - AMD64 mode
845 */
846#define PGM_SHW_TYPE PGM_TYPE_AMD64
847#define PGM_SHW_NAME(name) PGM_SHW_NAME_AMD64(name)
848#define PGM_SHW_NAME_RC_STR(name) PGM_SHW_NAME_RC_AMD64_STR(name)
849#define PGM_SHW_NAME_R0_STR(name) PGM_SHW_NAME_R0_AMD64_STR(name)
850#include "PGMShw.h"
851
852#ifdef VBOX_WITH_64_BITS_GUESTS
853/* Guest - AMD64 mode */
854# define PGM_GST_TYPE PGM_TYPE_AMD64
855# define PGM_GST_NAME(name) PGM_GST_NAME_AMD64(name)
856# define PGM_GST_NAME_RC_STR(name) PGM_GST_NAME_RC_AMD64_STR(name)
857# define PGM_GST_NAME_R0_STR(name) PGM_GST_NAME_R0_AMD64_STR(name)
858# define PGM_BTH_NAME(name) PGM_BTH_NAME_AMD64_AMD64(name)
859# define PGM_BTH_NAME_RC_STR(name) PGM_BTH_NAME_RC_AMD64_AMD64_STR(name)
860# define PGM_BTH_NAME_R0_STR(name) PGM_BTH_NAME_R0_AMD64_AMD64_STR(name)
861# define BTH_PGMPOOLKIND_PT_FOR_PT PGMPOOLKIND_PAE_PT_FOR_PAE_PT
862# define BTH_PGMPOOLKIND_PT_FOR_BIG PGMPOOLKIND_PAE_PT_FOR_PAE_2MB
863# include "PGMGst.h"
864# include "PGMBth.h"
865# undef BTH_PGMPOOLKIND_PT_FOR_BIG
866# undef BTH_PGMPOOLKIND_PT_FOR_PT
867# undef PGM_BTH_NAME
868# undef PGM_BTH_NAME_RC_STR
869# undef PGM_BTH_NAME_R0_STR
870# undef PGM_GST_TYPE
871# undef PGM_GST_NAME
872# undef PGM_GST_NAME_RC_STR
873# undef PGM_GST_NAME_R0_STR
874#endif /* VBOX_WITH_64_BITS_GUESTS */
875
876#undef PGM_SHW_TYPE
877#undef PGM_SHW_NAME
878#undef PGM_SHW_NAME_RC_STR
879#undef PGM_SHW_NAME_R0_STR
880
881
882/*
883 * Shadow - Nested paging mode
884 */
885#define PGM_SHW_TYPE PGM_TYPE_NESTED
886#define PGM_SHW_NAME(name) PGM_SHW_NAME_NESTED(name)
887#define PGM_SHW_NAME_RC_STR(name) PGM_SHW_NAME_RC_NESTED_STR(name)
888#define PGM_SHW_NAME_R0_STR(name) PGM_SHW_NAME_R0_NESTED_STR(name)
889#include "PGMShw.h"
890
891/* Guest - real mode */
892#define PGM_GST_TYPE PGM_TYPE_REAL
893#define PGM_GST_NAME(name) PGM_GST_NAME_REAL(name)
894#define PGM_GST_NAME_RC_STR(name) PGM_GST_NAME_RC_REAL_STR(name)
895#define PGM_GST_NAME_R0_STR(name) PGM_GST_NAME_R0_REAL_STR(name)
896#define PGM_BTH_NAME(name) PGM_BTH_NAME_NESTED_REAL(name)
897#define PGM_BTH_NAME_RC_STR(name) PGM_BTH_NAME_RC_NESTED_REAL_STR(name)
898#define PGM_BTH_NAME_R0_STR(name) PGM_BTH_NAME_R0_NESTED_REAL_STR(name)
899#define BTH_PGMPOOLKIND_PT_FOR_PT PGMPOOLKIND_PAE_PT_FOR_PHYS
900#include "PGMBth.h"
901#undef BTH_PGMPOOLKIND_PT_FOR_PT
902#undef PGM_BTH_NAME
903#undef PGM_BTH_NAME_RC_STR
904#undef PGM_BTH_NAME_R0_STR
905#undef PGM_GST_TYPE
906#undef PGM_GST_NAME
907#undef PGM_GST_NAME_RC_STR
908#undef PGM_GST_NAME_R0_STR
909
910/* Guest - protected mode */
911#define PGM_GST_TYPE PGM_TYPE_PROT
912#define PGM_GST_NAME(name) PGM_GST_NAME_PROT(name)
913#define PGM_GST_NAME_RC_STR(name) PGM_GST_NAME_RC_PROT_STR(name)
914#define PGM_GST_NAME_R0_STR(name) PGM_GST_NAME_R0_PROT_STR(name)
915#define PGM_BTH_NAME(name) PGM_BTH_NAME_NESTED_PROT(name)
916#define PGM_BTH_NAME_RC_STR(name) PGM_BTH_NAME_RC_NESTED_PROT_STR(name)
917#define PGM_BTH_NAME_R0_STR(name) PGM_BTH_NAME_R0_NESTED_PROT_STR(name)
918#define BTH_PGMPOOLKIND_PT_FOR_PT PGMPOOLKIND_PAE_PT_FOR_PHYS
919#include "PGMBth.h"
920#undef BTH_PGMPOOLKIND_PT_FOR_PT
921#undef PGM_BTH_NAME
922#undef PGM_BTH_NAME_RC_STR
923#undef PGM_BTH_NAME_R0_STR
924#undef PGM_GST_TYPE
925#undef PGM_GST_NAME
926#undef PGM_GST_NAME_RC_STR
927#undef PGM_GST_NAME_R0_STR
928
929/* Guest - 32-bit mode */
930#define PGM_GST_TYPE PGM_TYPE_32BIT
931#define PGM_GST_NAME(name) PGM_GST_NAME_32BIT(name)
932#define PGM_GST_NAME_RC_STR(name) PGM_GST_NAME_RC_32BIT_STR(name)
933#define PGM_GST_NAME_R0_STR(name) PGM_GST_NAME_R0_32BIT_STR(name)
934#define PGM_BTH_NAME(name) PGM_BTH_NAME_NESTED_32BIT(name)
935#define PGM_BTH_NAME_RC_STR(name) PGM_BTH_NAME_RC_NESTED_32BIT_STR(name)
936#define PGM_BTH_NAME_R0_STR(name) PGM_BTH_NAME_R0_NESTED_32BIT_STR(name)
937#define BTH_PGMPOOLKIND_PT_FOR_PT PGMPOOLKIND_PAE_PT_FOR_32BIT_PT
938#define BTH_PGMPOOLKIND_PT_FOR_BIG PGMPOOLKIND_PAE_PT_FOR_32BIT_4MB
939#include "PGMBth.h"
940#undef BTH_PGMPOOLKIND_PT_FOR_BIG
941#undef BTH_PGMPOOLKIND_PT_FOR_PT
942#undef PGM_BTH_NAME
943#undef PGM_BTH_NAME_RC_STR
944#undef PGM_BTH_NAME_R0_STR
945#undef PGM_GST_TYPE
946#undef PGM_GST_NAME
947#undef PGM_GST_NAME_RC_STR
948#undef PGM_GST_NAME_R0_STR
949
950/* Guest - PAE mode */
951#define PGM_GST_TYPE PGM_TYPE_PAE
952#define PGM_GST_NAME(name) PGM_GST_NAME_PAE(name)
953#define PGM_GST_NAME_RC_STR(name) PGM_GST_NAME_RC_PAE_STR(name)
954#define PGM_GST_NAME_R0_STR(name) PGM_GST_NAME_R0_PAE_STR(name)
955#define PGM_BTH_NAME(name) PGM_BTH_NAME_NESTED_PAE(name)
956#define PGM_BTH_NAME_RC_STR(name) PGM_BTH_NAME_RC_NESTED_PAE_STR(name)
957#define PGM_BTH_NAME_R0_STR(name) PGM_BTH_NAME_R0_NESTED_PAE_STR(name)
958#define BTH_PGMPOOLKIND_PT_FOR_PT PGMPOOLKIND_PAE_PT_FOR_PAE_PT
959#define BTH_PGMPOOLKIND_PT_FOR_BIG PGMPOOLKIND_PAE_PT_FOR_PAE_2MB
960#include "PGMBth.h"
961#undef BTH_PGMPOOLKIND_PT_FOR_BIG
962#undef BTH_PGMPOOLKIND_PT_FOR_PT
963#undef PGM_BTH_NAME
964#undef PGM_BTH_NAME_RC_STR
965#undef PGM_BTH_NAME_R0_STR
966#undef PGM_GST_TYPE
967#undef PGM_GST_NAME
968#undef PGM_GST_NAME_RC_STR
969#undef PGM_GST_NAME_R0_STR
970
971#ifdef VBOX_WITH_64_BITS_GUESTS
972/* Guest - AMD64 mode */
973# define PGM_GST_TYPE PGM_TYPE_AMD64
974# define PGM_GST_NAME(name) PGM_GST_NAME_AMD64(name)
975# define PGM_GST_NAME_RC_STR(name) PGM_GST_NAME_RC_AMD64_STR(name)
976# define PGM_GST_NAME_R0_STR(name) PGM_GST_NAME_R0_AMD64_STR(name)
977# define PGM_BTH_NAME(name) PGM_BTH_NAME_NESTED_AMD64(name)
978# define PGM_BTH_NAME_RC_STR(name) PGM_BTH_NAME_RC_NESTED_AMD64_STR(name)
979# define PGM_BTH_NAME_R0_STR(name) PGM_BTH_NAME_R0_NESTED_AMD64_STR(name)
980# define BTH_PGMPOOLKIND_PT_FOR_PT PGMPOOLKIND_PAE_PT_FOR_PAE_PT
981# define BTH_PGMPOOLKIND_PT_FOR_BIG PGMPOOLKIND_PAE_PT_FOR_PAE_2MB
982# include "PGMBth.h"
983# undef BTH_PGMPOOLKIND_PT_FOR_BIG
984# undef BTH_PGMPOOLKIND_PT_FOR_PT
985# undef PGM_BTH_NAME
986# undef PGM_BTH_NAME_RC_STR
987# undef PGM_BTH_NAME_R0_STR
988# undef PGM_GST_TYPE
989# undef PGM_GST_NAME
990# undef PGM_GST_NAME_RC_STR
991# undef PGM_GST_NAME_R0_STR
992#endif /* VBOX_WITH_64_BITS_GUESTS */
993
994#undef PGM_SHW_TYPE
995#undef PGM_SHW_NAME
996#undef PGM_SHW_NAME_RC_STR
997#undef PGM_SHW_NAME_R0_STR
998
999
1000/*
1001 * Shadow - EPT
1002 */
1003#define PGM_SHW_TYPE PGM_TYPE_EPT
1004#define PGM_SHW_NAME(name) PGM_SHW_NAME_EPT(name)
1005#define PGM_SHW_NAME_RC_STR(name) PGM_SHW_NAME_RC_EPT_STR(name)
1006#define PGM_SHW_NAME_R0_STR(name) PGM_SHW_NAME_R0_EPT_STR(name)
1007#include "PGMShw.h"
1008
1009/* Guest - real mode */
1010#define PGM_GST_TYPE PGM_TYPE_REAL
1011#define PGM_GST_NAME(name) PGM_GST_NAME_REAL(name)
1012#define PGM_GST_NAME_RC_STR(name) PGM_GST_NAME_RC_REAL_STR(name)
1013#define PGM_GST_NAME_R0_STR(name) PGM_GST_NAME_R0_REAL_STR(name)
1014#define PGM_BTH_NAME(name) PGM_BTH_NAME_EPT_REAL(name)
1015#define PGM_BTH_NAME_RC_STR(name) PGM_BTH_NAME_RC_EPT_REAL_STR(name)
1016#define PGM_BTH_NAME_R0_STR(name) PGM_BTH_NAME_R0_EPT_REAL_STR(name)
1017#define BTH_PGMPOOLKIND_PT_FOR_PT PGMPOOLKIND_PAE_PT_FOR_PHYS
1018#include "PGMBth.h"
1019#undef BTH_PGMPOOLKIND_PT_FOR_PT
1020#undef PGM_BTH_NAME
1021#undef PGM_BTH_NAME_RC_STR
1022#undef PGM_BTH_NAME_R0_STR
1023#undef PGM_GST_TYPE
1024#undef PGM_GST_NAME
1025#undef PGM_GST_NAME_RC_STR
1026#undef PGM_GST_NAME_R0_STR
1027
1028/* Guest - protected mode */
1029#define PGM_GST_TYPE PGM_TYPE_PROT
1030#define PGM_GST_NAME(name) PGM_GST_NAME_PROT(name)
1031#define PGM_GST_NAME_RC_STR(name) PGM_GST_NAME_RC_PROT_STR(name)
1032#define PGM_GST_NAME_R0_STR(name) PGM_GST_NAME_R0_PROT_STR(name)
1033#define PGM_BTH_NAME(name) PGM_BTH_NAME_EPT_PROT(name)
1034#define PGM_BTH_NAME_RC_STR(name) PGM_BTH_NAME_RC_EPT_PROT_STR(name)
1035#define PGM_BTH_NAME_R0_STR(name) PGM_BTH_NAME_R0_EPT_PROT_STR(name)
1036#define BTH_PGMPOOLKIND_PT_FOR_PT PGMPOOLKIND_PAE_PT_FOR_PHYS
1037#include "PGMBth.h"
1038#undef BTH_PGMPOOLKIND_PT_FOR_PT
1039#undef PGM_BTH_NAME
1040#undef PGM_BTH_NAME_RC_STR
1041#undef PGM_BTH_NAME_R0_STR
1042#undef PGM_GST_TYPE
1043#undef PGM_GST_NAME
1044#undef PGM_GST_NAME_RC_STR
1045#undef PGM_GST_NAME_R0_STR
1046
1047/* Guest - 32-bit mode */
1048#define PGM_GST_TYPE PGM_TYPE_32BIT
1049#define PGM_GST_NAME(name) PGM_GST_NAME_32BIT(name)
1050#define PGM_GST_NAME_RC_STR(name) PGM_GST_NAME_RC_32BIT_STR(name)
1051#define PGM_GST_NAME_R0_STR(name) PGM_GST_NAME_R0_32BIT_STR(name)
1052#define PGM_BTH_NAME(name) PGM_BTH_NAME_EPT_32BIT(name)
1053#define PGM_BTH_NAME_RC_STR(name) PGM_BTH_NAME_RC_EPT_32BIT_STR(name)
1054#define PGM_BTH_NAME_R0_STR(name) PGM_BTH_NAME_R0_EPT_32BIT_STR(name)
1055#define BTH_PGMPOOLKIND_PT_FOR_PT PGMPOOLKIND_PAE_PT_FOR_32BIT_PT
1056#define BTH_PGMPOOLKIND_PT_FOR_BIG PGMPOOLKIND_PAE_PT_FOR_32BIT_4MB
1057#include "PGMBth.h"
1058#undef BTH_PGMPOOLKIND_PT_FOR_BIG
1059#undef BTH_PGMPOOLKIND_PT_FOR_PT
1060#undef PGM_BTH_NAME
1061#undef PGM_BTH_NAME_RC_STR
1062#undef PGM_BTH_NAME_R0_STR
1063#undef PGM_GST_TYPE
1064#undef PGM_GST_NAME
1065#undef PGM_GST_NAME_RC_STR
1066#undef PGM_GST_NAME_R0_STR
1067
1068/* Guest - PAE mode */
1069#define PGM_GST_TYPE PGM_TYPE_PAE
1070#define PGM_GST_NAME(name) PGM_GST_NAME_PAE(name)
1071#define PGM_GST_NAME_RC_STR(name) PGM_GST_NAME_RC_PAE_STR(name)
1072#define PGM_GST_NAME_R0_STR(name) PGM_GST_NAME_R0_PAE_STR(name)
1073#define PGM_BTH_NAME(name) PGM_BTH_NAME_EPT_PAE(name)
1074#define PGM_BTH_NAME_RC_STR(name) PGM_BTH_NAME_RC_EPT_PAE_STR(name)
1075#define PGM_BTH_NAME_R0_STR(name) PGM_BTH_NAME_R0_EPT_PAE_STR(name)
1076#define BTH_PGMPOOLKIND_PT_FOR_PT PGMPOOLKIND_PAE_PT_FOR_PAE_PT
1077#define BTH_PGMPOOLKIND_PT_FOR_BIG PGMPOOLKIND_PAE_PT_FOR_PAE_2MB
1078#include "PGMBth.h"
1079#undef BTH_PGMPOOLKIND_PT_FOR_BIG
1080#undef BTH_PGMPOOLKIND_PT_FOR_PT
1081#undef PGM_BTH_NAME
1082#undef PGM_BTH_NAME_RC_STR
1083#undef PGM_BTH_NAME_R0_STR
1084#undef PGM_GST_TYPE
1085#undef PGM_GST_NAME
1086#undef PGM_GST_NAME_RC_STR
1087#undef PGM_GST_NAME_R0_STR
1088
1089#ifdef VBOX_WITH_64_BITS_GUESTS
1090/* Guest - AMD64 mode */
1091# define PGM_GST_TYPE PGM_TYPE_AMD64
1092# define PGM_GST_NAME(name) PGM_GST_NAME_AMD64(name)
1093# define PGM_GST_NAME_RC_STR(name) PGM_GST_NAME_RC_AMD64_STR(name)
1094# define PGM_GST_NAME_R0_STR(name) PGM_GST_NAME_R0_AMD64_STR(name)
1095# define PGM_BTH_NAME(name) PGM_BTH_NAME_EPT_AMD64(name)
1096# define PGM_BTH_NAME_RC_STR(name) PGM_BTH_NAME_RC_EPT_AMD64_STR(name)
1097# define PGM_BTH_NAME_R0_STR(name) PGM_BTH_NAME_R0_EPT_AMD64_STR(name)
1098# define BTH_PGMPOOLKIND_PT_FOR_PT PGMPOOLKIND_PAE_PT_FOR_PAE_PT
1099# define BTH_PGMPOOLKIND_PT_FOR_BIG PGMPOOLKIND_PAE_PT_FOR_PAE_2MB
1100# include "PGMBth.h"
1101# undef BTH_PGMPOOLKIND_PT_FOR_BIG
1102# undef BTH_PGMPOOLKIND_PT_FOR_PT
1103# undef PGM_BTH_NAME
1104# undef PGM_BTH_NAME_RC_STR
1105# undef PGM_BTH_NAME_R0_STR
1106# undef PGM_GST_TYPE
1107# undef PGM_GST_NAME
1108# undef PGM_GST_NAME_RC_STR
1109# undef PGM_GST_NAME_R0_STR
1110#endif /* VBOX_WITH_64_BITS_GUESTS */
1111
1112#undef PGM_SHW_TYPE
1113#undef PGM_SHW_NAME
1114#undef PGM_SHW_NAME_RC_STR
1115#undef PGM_SHW_NAME_R0_STR
1116
1117
1118
1119/**
1120 * Initiates the paging of VM.
1121 *
1122 * @returns VBox status code.
1123 * @param pVM Pointer to VM structure.
1124 */
1125VMMR3DECL(int) PGMR3Init(PVM pVM)
1126{
1127 LogFlow(("PGMR3Init:\n"));
1128
1129 /*
1130 * Assert alignment and sizes.
1131 */
1132 AssertRelease(sizeof(pVM->pgm.s) <= sizeof(pVM->pgm.padding));
1133
1134 /*
1135 * Init the structure.
1136 */
1137 pVM->pgm.s.offVM = RT_OFFSETOF(VM, pgm.s);
1138 pVM->pgm.s.offVCpu = RT_OFFSETOF(VMCPU, pgm.s);
1139 pVM->pgm.s.enmShadowMode = PGMMODE_INVALID;
1140 pVM->pgm.s.enmGuestMode = PGMMODE_INVALID;
1141 pVM->pgm.s.enmHostMode = SUPPAGINGMODE_INVALID;
1142 pVM->pgm.s.GCPhysCR3 = NIL_RTGCPHYS;
1143 pVM->pgm.s.GCPhysGstCR3Monitored = NIL_RTGCPHYS;
1144 pVM->pgm.s.fA20Enabled = true;
1145 pVM->pgm.s.GCPhys4MBPSEMask = RT_BIT_64(32) - 1; /* default; checked later */
1146 pVM->pgm.s.pGstPaePdptR3 = NULL;
1147#ifndef VBOX_WITH_2X_4GB_ADDR_SPACE
1148 pVM->pgm.s.pGstPaePdptR0 = NIL_RTR0PTR;
1149#endif
1150 pVM->pgm.s.pGstPaePdptRC = NIL_RTRCPTR;
1151 for (unsigned i = 0; i < RT_ELEMENTS(pVM->pgm.s.apGstPaePDsR3); i++)
1152 {
1153 pVM->pgm.s.apGstPaePDsR3[i] = NULL;
1154#ifndef VBOX_WITH_2X_4GB_ADDR_SPACE
1155 pVM->pgm.s.apGstPaePDsR0[i] = NIL_RTR0PTR;
1156#endif
1157 pVM->pgm.s.apGstPaePDsRC[i] = NIL_RTRCPTR;
1158 pVM->pgm.s.aGCPhysGstPaePDs[i] = NIL_RTGCPHYS;
1159 pVM->pgm.s.aGCPhysGstPaePDsMonitored[i] = NIL_RTGCPHYS;
1160 }
1161
1162#ifdef VBOX_STRICT
1163 VMR3AtStateRegister(pVM, pgmR3ResetNoMorePhysWritesFlag, NULL);
1164#endif
1165
1166 /*
1167 * Get the configured RAM size - to estimate saved state size.
1168 */
1169 uint64_t cbRam;
1170 int rc = CFGMR3QueryU64(CFGMR3GetRoot(pVM), "RamSize", &cbRam);
1171 if (rc == VERR_CFGM_VALUE_NOT_FOUND)
1172 cbRam = pVM->pgm.s.cbRamSize = 0;
1173 else if (RT_SUCCESS(rc))
1174 {
1175 if (cbRam < PAGE_SIZE)
1176 cbRam = 0;
1177 cbRam = RT_ALIGN_64(cbRam, PAGE_SIZE);
1178 pVM->pgm.s.cbRamSize = (RTUINT)cbRam;
1179 }
1180 else
1181 {
1182 AssertMsgFailed(("Configuration error: Failed to query integer \"RamSize\", rc=%Rrc.\n", rc));
1183 return rc;
1184 }
1185
1186 /*
1187 * Register saved state data unit.
1188 */
1189 rc = SSMR3RegisterInternal(pVM, "pgm", 1, PGM_SAVED_STATE_VERSION, (size_t)cbRam + sizeof(PGM),
1190 NULL, pgmR3Save, NULL,
1191 NULL, pgmR3Load, NULL);
1192 if (RT_FAILURE(rc))
1193 return rc;
1194
1195 /*
1196 * Initialize the PGM critical section and flush the phys TLBs
1197 */
1198 rc = PDMR3CritSectInit(pVM, &pVM->pgm.s.CritSect, "PGM");
1199 AssertRCReturn(rc, rc);
1200
1201 PGMR3PhysChunkInvalidateTLB(pVM);
1202 PGMPhysInvalidatePageR3MapTLB(pVM);
1203 PGMPhysInvalidatePageR0MapTLB(pVM);
1204 PGMPhysInvalidatePageGCMapTLB(pVM);
1205
1206 /*
1207 * Trees
1208 */
1209 rc = MMHyperAlloc(pVM, sizeof(PGMTREES), 0, MM_TAG_PGM, (void **)&pVM->pgm.s.pTreesR3);
1210 if (RT_SUCCESS(rc))
1211 {
1212 pVM->pgm.s.pTreesR0 = MMHyperR3ToR0(pVM, pVM->pgm.s.pTreesR3);
1213 pVM->pgm.s.pTreesRC = MMHyperR3ToRC(pVM, pVM->pgm.s.pTreesR3);
1214
1215 /*
1216 * Alocate the zero page.
1217 */
1218 rc = MMHyperAlloc(pVM, PAGE_SIZE, PAGE_SIZE, MM_TAG_PGM, &pVM->pgm.s.pvZeroPgR3);
1219 }
1220 if (RT_SUCCESS(rc))
1221 {
1222 pVM->pgm.s.pvZeroPgGC = MMHyperR3ToRC(pVM, pVM->pgm.s.pvZeroPgR3);
1223 pVM->pgm.s.pvZeroPgR0 = MMHyperR3ToR0(pVM, pVM->pgm.s.pvZeroPgR3);
1224 AssertRelease(pVM->pgm.s.pvZeroPgR0 != NIL_RTHCPHYS);
1225 pVM->pgm.s.HCPhysZeroPg = MMR3HyperHCVirt2HCPhys(pVM, pVM->pgm.s.pvZeroPgR3);
1226 AssertRelease(pVM->pgm.s.HCPhysZeroPg != NIL_RTHCPHYS);
1227
1228 /*
1229 * Init the paging.
1230 */
1231 rc = pgmR3InitPaging(pVM);
1232 }
1233 if (RT_SUCCESS(rc))
1234 {
1235 /*
1236 * Init the page pool.
1237 */
1238 rc = pgmR3PoolInit(pVM);
1239 }
1240 if (RT_SUCCESS(rc))
1241 {
1242 /*
1243 * Info & statistics
1244 */
1245 DBGFR3InfoRegisterInternal(pVM, "mode",
1246 "Shows the current paging mode. "
1247 "Recognizes 'all', 'guest', 'shadow' and 'host' as arguments, defaulting to 'all' if nothing's given.",
1248 pgmR3InfoMode);
1249 DBGFR3InfoRegisterInternal(pVM, "pgmcr3",
1250 "Dumps all the entries in the top level paging table. No arguments.",
1251 pgmR3InfoCr3);
1252 DBGFR3InfoRegisterInternal(pVM, "phys",
1253 "Dumps all the physical address ranges. No arguments.",
1254 pgmR3PhysInfo);
1255 DBGFR3InfoRegisterInternal(pVM, "handlers",
1256 "Dumps physical, virtual and hyper virtual handlers. "
1257 "Pass 'phys', 'virt', 'hyper' as argument if only one kind is wanted."
1258 "Add 'nost' if the statistics are unwanted, use together with 'all' or explicit selection.",
1259 pgmR3InfoHandlers);
1260 DBGFR3InfoRegisterInternal(pVM, "mappings",
1261 "Dumps guest mappings.",
1262 pgmR3MapInfo);
1263
1264 STAM_REL_REG(pVM, &pVM->pgm.s.cGuestModeChanges, STAMTYPE_COUNTER, "/PGM/cGuestModeChanges", STAMUNIT_OCCURENCES, "Number of guest mode changes.");
1265#ifdef VBOX_WITH_STATISTICS
1266 pgmR3InitStats(pVM);
1267#endif
1268#ifdef VBOX_WITH_DEBUGGER
1269 /*
1270 * Debugger commands.
1271 */
1272 static bool fRegisteredCmds = false;
1273 if (!fRegisteredCmds)
1274 {
1275 int rc = DBGCRegisterCommands(&g_aCmds[0], RT_ELEMENTS(g_aCmds));
1276 if (RT_SUCCESS(rc))
1277 fRegisteredCmds = true;
1278 }
1279#endif
1280 return VINF_SUCCESS;
1281 }
1282
1283 /* Almost no cleanup necessary, MM frees all memory. */
1284 PDMR3CritSectDelete(&pVM->pgm.s.CritSect);
1285
1286 return rc;
1287}
1288
1289
1290/**
1291 * Initializes the per-VCPU PGM.
1292 *
1293 * @returns VBox status code.
1294 * @param pVM The VM to operate on.
1295 */
1296VMMR3DECL(int) PGMR3InitCPU(PVM pVM)
1297{
1298 LogFlow(("PGMR3InitCPU\n"));
1299 return VINF_SUCCESS;
1300}
1301
1302
1303/**
1304 * Init paging.
1305 *
1306 * Since we need to check what mode the host is operating in before we can choose
1307 * the right paging functions for the host we have to delay this until R0 has
1308 * been initialized.
1309 *
1310 * @returns VBox status code.
1311 * @param pVM VM handle.
1312 */
1313static int pgmR3InitPaging(PVM pVM)
1314{
1315 /*
1316 * Force a recalculation of modes and switcher so everyone gets notified.
1317 */
1318 pVM->pgm.s.enmShadowMode = PGMMODE_INVALID;
1319 pVM->pgm.s.enmGuestMode = PGMMODE_INVALID;
1320 pVM->pgm.s.enmHostMode = SUPPAGINGMODE_INVALID;
1321
1322 /*
1323 * Allocate static mapping space for whatever the cr3 register
1324 * points to and in the case of PAE mode to the 4 PDs.
1325 */
1326 int rc = MMR3HyperReserve(pVM, PAGE_SIZE * 5, "CR3 mapping", &pVM->pgm.s.GCPtrCR3Mapping);
1327 if (RT_FAILURE(rc))
1328 {
1329 AssertMsgFailed(("Failed to reserve two pages for cr mapping in HMA, rc=%Rrc\n", rc));
1330 return rc;
1331 }
1332 MMR3HyperReserve(pVM, PAGE_SIZE, "fence", NULL);
1333
1334 /*
1335 * Allocate pages for the three possible intermediate contexts
1336 * (AMD64, PAE and plain 32-Bit). We maintain all three contexts
1337 * for the sake of simplicity. The AMD64 uses the PAE for the
1338 * lower levels, making the total number of pages 11 (3 + 7 + 1).
1339 *
1340 * We assume that two page tables will be enought for the core code
1341 * mappings (HC virtual and identity).
1342 */
1343 pVM->pgm.s.pInterPD = (PX86PD)MMR3PageAllocLow(pVM);
1344 pVM->pgm.s.apInterPTs[0] = (PX86PT)MMR3PageAllocLow(pVM);
1345 pVM->pgm.s.apInterPTs[1] = (PX86PT)MMR3PageAllocLow(pVM);
1346 pVM->pgm.s.apInterPaePTs[0] = (PX86PTPAE)MMR3PageAlloc(pVM);
1347 pVM->pgm.s.apInterPaePTs[1] = (PX86PTPAE)MMR3PageAlloc(pVM);
1348 pVM->pgm.s.apInterPaePDs[0] = (PX86PDPAE)MMR3PageAlloc(pVM);
1349 pVM->pgm.s.apInterPaePDs[1] = (PX86PDPAE)MMR3PageAlloc(pVM);
1350 pVM->pgm.s.apInterPaePDs[2] = (PX86PDPAE)MMR3PageAlloc(pVM);
1351 pVM->pgm.s.apInterPaePDs[3] = (PX86PDPAE)MMR3PageAlloc(pVM);
1352 pVM->pgm.s.pInterPaePDPT = (PX86PDPT)MMR3PageAllocLow(pVM);
1353 pVM->pgm.s.pInterPaePDPT64 = (PX86PDPT)MMR3PageAllocLow(pVM);
1354 pVM->pgm.s.pInterPaePML4 = (PX86PML4)MMR3PageAllocLow(pVM);
1355 if ( !pVM->pgm.s.pInterPD
1356 || !pVM->pgm.s.apInterPTs[0]
1357 || !pVM->pgm.s.apInterPTs[1]
1358 || !pVM->pgm.s.apInterPaePTs[0]
1359 || !pVM->pgm.s.apInterPaePTs[1]
1360 || !pVM->pgm.s.apInterPaePDs[0]
1361 || !pVM->pgm.s.apInterPaePDs[1]
1362 || !pVM->pgm.s.apInterPaePDs[2]
1363 || !pVM->pgm.s.apInterPaePDs[3]
1364 || !pVM->pgm.s.pInterPaePDPT
1365 || !pVM->pgm.s.pInterPaePDPT64
1366 || !pVM->pgm.s.pInterPaePML4)
1367 {
1368 AssertMsgFailed(("Failed to allocate pages for the intermediate context!\n"));
1369 return VERR_NO_PAGE_MEMORY;
1370 }
1371
1372 pVM->pgm.s.HCPhysInterPD = MMPage2Phys(pVM, pVM->pgm.s.pInterPD);
1373 AssertRelease(pVM->pgm.s.HCPhysInterPD != NIL_RTHCPHYS && !(pVM->pgm.s.HCPhysInterPD & PAGE_OFFSET_MASK));
1374 pVM->pgm.s.HCPhysInterPaePDPT = MMPage2Phys(pVM, pVM->pgm.s.pInterPaePDPT);
1375 AssertRelease(pVM->pgm.s.HCPhysInterPaePDPT != NIL_RTHCPHYS && !(pVM->pgm.s.HCPhysInterPaePDPT & PAGE_OFFSET_MASK));
1376 pVM->pgm.s.HCPhysInterPaePML4 = MMPage2Phys(pVM, pVM->pgm.s.pInterPaePML4);
1377 AssertRelease(pVM->pgm.s.HCPhysInterPaePML4 != NIL_RTHCPHYS && !(pVM->pgm.s.HCPhysInterPaePML4 & PAGE_OFFSET_MASK) && pVM->pgm.s.HCPhysInterPaePML4 < 0xffffffff);
1378
1379 /*
1380 * Initialize the pages, setting up the PML4 and PDPT for repetitive 4GB action.
1381 */
1382 ASMMemZeroPage(pVM->pgm.s.pInterPD);
1383 ASMMemZeroPage(pVM->pgm.s.apInterPTs[0]);
1384 ASMMemZeroPage(pVM->pgm.s.apInterPTs[1]);
1385
1386 ASMMemZeroPage(pVM->pgm.s.apInterPaePTs[0]);
1387 ASMMemZeroPage(pVM->pgm.s.apInterPaePTs[1]);
1388
1389 ASMMemZeroPage(pVM->pgm.s.pInterPaePDPT);
1390 for (unsigned i = 0; i < RT_ELEMENTS(pVM->pgm.s.apInterPaePDs); i++)
1391 {
1392 ASMMemZeroPage(pVM->pgm.s.apInterPaePDs[i]);
1393 pVM->pgm.s.pInterPaePDPT->a[i].u = X86_PDPE_P | PGM_PLXFLAGS_PERMANENT
1394 | MMPage2Phys(pVM, pVM->pgm.s.apInterPaePDs[i]);
1395 }
1396
1397 for (unsigned i = 0; i < RT_ELEMENTS(pVM->pgm.s.pInterPaePDPT64->a); i++)
1398 {
1399 const unsigned iPD = i % RT_ELEMENTS(pVM->pgm.s.apInterPaePDs);
1400 pVM->pgm.s.pInterPaePDPT64->a[i].u = X86_PDPE_P | X86_PDPE_RW | X86_PDPE_US | X86_PDPE_A | PGM_PLXFLAGS_PERMANENT
1401 | MMPage2Phys(pVM, pVM->pgm.s.apInterPaePDs[iPD]);
1402 }
1403
1404 RTHCPHYS HCPhysInterPaePDPT64 = MMPage2Phys(pVM, pVM->pgm.s.pInterPaePDPT64);
1405 for (unsigned i = 0; i < RT_ELEMENTS(pVM->pgm.s.pInterPaePML4->a); i++)
1406 pVM->pgm.s.pInterPaePML4->a[i].u = X86_PML4E_P | X86_PML4E_RW | X86_PML4E_US | X86_PML4E_A | PGM_PLXFLAGS_PERMANENT
1407 | HCPhysInterPaePDPT64;
1408
1409 /*
1410 * Allocate pages for the three possible guest contexts (AMD64, PAE and plain 32-Bit).
1411 * We allocate pages for all three posibilities in order to simplify mappings and
1412 * avoid resource failure during mode switches. So, we need to cover all levels of the
1413 * of the first 4GB down to PD level.
1414 * As with the intermediate context, AMD64 uses the PAE PDPT and PDs.
1415 */
1416#ifndef VBOX_WITH_PGMPOOL_PAGING_ONLY
1417 pVM->pgm.s.pShw32BitPdR3 = (PX86PD)MMR3PageAllocLow(pVM);
1418# ifndef VBOX_WITH_2X_4GB_ADDR_SPACE
1419 pVM->pgm.s.pShw32BitPdR0 = (uintptr_t)pVM->pgm.s.pShw32BitPdR3;
1420# endif
1421 pVM->pgm.s.apShwPaePDsR3[0] = (PX86PDPAE)MMR3PageAlloc(pVM);
1422 pVM->pgm.s.apShwPaePDsR3[1] = (PX86PDPAE)MMR3PageAlloc(pVM);
1423 AssertRelease((uintptr_t)pVM->pgm.s.apShwPaePDsR3[0] + PAGE_SIZE == (uintptr_t)pVM->pgm.s.apShwPaePDsR3[1]);
1424 pVM->pgm.s.apShwPaePDsR3[2] = (PX86PDPAE)MMR3PageAlloc(pVM);
1425 AssertRelease((uintptr_t)pVM->pgm.s.apShwPaePDsR3[1] + PAGE_SIZE == (uintptr_t)pVM->pgm.s.apShwPaePDsR3[2]);
1426 pVM->pgm.s.apShwPaePDsR3[3] = (PX86PDPAE)MMR3PageAlloc(pVM);
1427 AssertRelease((uintptr_t)pVM->pgm.s.apShwPaePDsR3[2] + PAGE_SIZE == (uintptr_t)pVM->pgm.s.apShwPaePDsR3[3]);
1428# ifndef VBOX_WITH_2X_4GB_ADDR_SPACE
1429 pVM->pgm.s.apShwPaePDsR0[0] = (uintptr_t)pVM->pgm.s.apShwPaePDsR3[0];
1430 pVM->pgm.s.apShwPaePDsR0[1] = (uintptr_t)pVM->pgm.s.apShwPaePDsR3[1];
1431 pVM->pgm.s.apShwPaePDsR0[2] = (uintptr_t)pVM->pgm.s.apShwPaePDsR3[2];
1432 pVM->pgm.s.apShwPaePDsR0[3] = (uintptr_t)pVM->pgm.s.apShwPaePDsR3[3];
1433# endif
1434 pVM->pgm.s.pShwPaePdptR3 = (PX86PDPT)MMR3PageAllocLow(pVM);
1435# ifndef VBOX_WITH_2X_4GB_ADDR_SPACE
1436 pVM->pgm.s.pShwPaePdptR0 = (uintptr_t)pVM->pgm.s.pShwPaePdptR3;
1437# endif
1438#endif /* VBOX_WITH_PGMPOOL_PAGING_ONLY */
1439 pVM->pgm.s.pShwNestedRootR3 = MMR3PageAllocLow(pVM);
1440#ifndef VBOX_WITH_2X_4GB_ADDR_SPACE
1441 pVM->pgm.s.pShwNestedRootR0 = (uintptr_t)pVM->pgm.s.pShwNestedRootR3;
1442#endif
1443
1444#ifdef VBOX_WITH_PGMPOOL_PAGING_ONLY
1445 if (!pVM->pgm.s.pShwNestedRootR3)
1446#else
1447 if ( !pVM->pgm.s.pShw32BitPdR3
1448 || !pVM->pgm.s.apShwPaePDsR3[0]
1449 || !pVM->pgm.s.apShwPaePDsR3[1]
1450 || !pVM->pgm.s.apShwPaePDsR3[2]
1451 || !pVM->pgm.s.apShwPaePDsR3[3]
1452 || !pVM->pgm.s.pShwPaePdptR3
1453 || !pVM->pgm.s.pShwNestedRootR3)
1454#endif
1455 {
1456 AssertMsgFailed(("Failed to allocate pages for the intermediate context!\n"));
1457 return VERR_NO_PAGE_MEMORY;
1458 }
1459
1460 /* get physical addresses. */
1461#ifndef VBOX_WITH_PGMPOOL_PAGING_ONLY
1462 pVM->pgm.s.HCPhysShw32BitPD = MMPage2Phys(pVM, pVM->pgm.s.pShw32BitPdR3);
1463 Assert(MMPagePhys2Page(pVM, pVM->pgm.s.HCPhysShw32BitPD) == pVM->pgm.s.pShw32BitPdR3);
1464 pVM->pgm.s.aHCPhysPaePDs[0] = MMPage2Phys(pVM, pVM->pgm.s.apShwPaePDsR3[0]);
1465 pVM->pgm.s.aHCPhysPaePDs[1] = MMPage2Phys(pVM, pVM->pgm.s.apShwPaePDsR3[1]);
1466 pVM->pgm.s.aHCPhysPaePDs[2] = MMPage2Phys(pVM, pVM->pgm.s.apShwPaePDsR3[2]);
1467 pVM->pgm.s.aHCPhysPaePDs[3] = MMPage2Phys(pVM, pVM->pgm.s.apShwPaePDsR3[3]);
1468 pVM->pgm.s.HCPhysShwPaePdpt = MMPage2Phys(pVM, pVM->pgm.s.pShwPaePdptR3);
1469#endif
1470 pVM->pgm.s.HCPhysShwNestedRoot = MMPage2Phys(pVM, pVM->pgm.s.pShwNestedRootR3);
1471
1472 /*
1473 * Initialize the pages, setting up the PML4 and PDPT for action below 4GB.
1474 */
1475#ifndef VBOX_WITH_PGMPOOL_PAGING_ONLY
1476 ASMMemZero32(pVM->pgm.s.pShw32BitPdR3, PAGE_SIZE);
1477 ASMMemZero32(pVM->pgm.s.pShwPaePdptR3, PAGE_SIZE);
1478#endif
1479 ASMMemZero32(pVM->pgm.s.pShwNestedRootR3, PAGE_SIZE);
1480#ifndef VBOX_WITH_PGMPOOL_PAGING_ONLY
1481 for (unsigned i = 0; i < RT_ELEMENTS(pVM->pgm.s.apShwPaePDsR3); i++)
1482 {
1483 ASMMemZero32(pVM->pgm.s.apShwPaePDsR3[i], PAGE_SIZE);
1484 pVM->pgm.s.pShwPaePdptR3->a[i].u = X86_PDPE_P | PGM_PLXFLAGS_PERMANENT | pVM->pgm.s.aHCPhysPaePDs[i];
1485 /* The flags will be corrected when entering and leaving long mode. */
1486 }
1487
1488 CPUMSetHyperCR3(pVM, (uint32_t)pVM->pgm.s.HCPhysShw32BitPD);
1489#endif
1490
1491 /*
1492 * Initialize paging workers and mode from current host mode
1493 * and the guest running in real mode.
1494 */
1495 pVM->pgm.s.enmHostMode = SUPGetPagingMode();
1496 switch (pVM->pgm.s.enmHostMode)
1497 {
1498 case SUPPAGINGMODE_32_BIT:
1499 case SUPPAGINGMODE_32_BIT_GLOBAL:
1500 case SUPPAGINGMODE_PAE:
1501 case SUPPAGINGMODE_PAE_GLOBAL:
1502 case SUPPAGINGMODE_PAE_NX:
1503 case SUPPAGINGMODE_PAE_GLOBAL_NX:
1504 break;
1505
1506 case SUPPAGINGMODE_AMD64:
1507 case SUPPAGINGMODE_AMD64_GLOBAL:
1508 case SUPPAGINGMODE_AMD64_NX:
1509 case SUPPAGINGMODE_AMD64_GLOBAL_NX:
1510#ifndef VBOX_WITH_HYBRID_32BIT_KERNEL
1511 if (ARCH_BITS != 64)
1512 {
1513 AssertMsgFailed(("Host mode %d (64-bit) is not supported by non-64bit builds\n", pVM->pgm.s.enmHostMode));
1514 LogRel(("Host mode %d (64-bit) is not supported by non-64bit builds\n", pVM->pgm.s.enmHostMode));
1515 return VERR_PGM_UNSUPPORTED_HOST_PAGING_MODE;
1516 }
1517#endif
1518 break;
1519 default:
1520 AssertMsgFailed(("Host mode %d is not supported\n", pVM->pgm.s.enmHostMode));
1521 return VERR_PGM_UNSUPPORTED_HOST_PAGING_MODE;
1522 }
1523 rc = pgmR3ModeDataInit(pVM, false /* don't resolve GC and R0 syms yet */);
1524 if (RT_SUCCESS(rc))
1525 rc = PGMR3ChangeMode(pVM, PGMMODE_REAL);
1526 if (RT_SUCCESS(rc))
1527 {
1528 LogFlow(("pgmR3InitPaging: returns successfully\n"));
1529#if HC_ARCH_BITS == 64
1530# ifndef VBOX_WITH_PGMPOOL_PAGING_ONLY
1531 LogRel(("Debug: HCPhysShw32BitPD=%RHp aHCPhysPaePDs={%RHp,%RHp,%RHp,%RHp} HCPhysShwPaePdpt=%RHp\n",
1532 pVM->pgm.s.HCPhysShw32BitPD,
1533 pVM->pgm.s.aHCPhysPaePDs[0], pVM->pgm.s.aHCPhysPaePDs[1], pVM->pgm.s.aHCPhysPaePDs[2], pVM->pgm.s.aHCPhysPaePDs[3],
1534 pVM->pgm.s.HCPhysShwPaePdpt));
1535# endif
1536 LogRel(("Debug: HCPhysInterPD=%RHp HCPhysInterPaePDPT=%RHp HCPhysInterPaePML4=%RHp\n",
1537 pVM->pgm.s.HCPhysInterPD, pVM->pgm.s.HCPhysInterPaePDPT, pVM->pgm.s.HCPhysInterPaePML4));
1538 LogRel(("Debug: apInterPTs={%RHp,%RHp} apInterPaePTs={%RHp,%RHp} apInterPaePDs={%RHp,%RHp,%RHp,%RHp} pInterPaePDPT64=%RHp\n",
1539 MMPage2Phys(pVM, pVM->pgm.s.apInterPTs[0]), MMPage2Phys(pVM, pVM->pgm.s.apInterPTs[1]),
1540 MMPage2Phys(pVM, pVM->pgm.s.apInterPaePTs[0]), MMPage2Phys(pVM, pVM->pgm.s.apInterPaePTs[1]),
1541 MMPage2Phys(pVM, pVM->pgm.s.apInterPaePDs[0]), MMPage2Phys(pVM, pVM->pgm.s.apInterPaePDs[1]), MMPage2Phys(pVM, pVM->pgm.s.apInterPaePDs[2]), MMPage2Phys(pVM, pVM->pgm.s.apInterPaePDs[3]),
1542 MMPage2Phys(pVM, pVM->pgm.s.pInterPaePDPT64)));
1543#endif
1544
1545 return VINF_SUCCESS;
1546 }
1547
1548 LogFlow(("pgmR3InitPaging: returns %Rrc\n", rc));
1549 return rc;
1550}
1551
1552
1553#ifdef VBOX_WITH_STATISTICS
1554/**
1555 * Init statistics
1556 */
1557static void pgmR3InitStats(PVM pVM)
1558{
1559 PPGM pPGM = &pVM->pgm.s;
1560 unsigned i;
1561
1562 /*
1563 * Note! The layout of this function matches the member layout exactly!
1564 */
1565
1566 /* Common - misc variables */
1567 STAM_REG(pVM, &pPGM->cAllPages, STAMTYPE_U32, "/PGM/Page/cAllPages", STAMUNIT_OCCURENCES, "The total number of pages.");
1568 STAM_REG(pVM, &pPGM->cPrivatePages, STAMTYPE_U32, "/PGM/Page/cPrivatePages", STAMUNIT_OCCURENCES, "The number of private pages.");
1569 STAM_REG(pVM, &pPGM->cSharedPages, STAMTYPE_U32, "/PGM/Page/cSharedPages", STAMUNIT_OCCURENCES, "The number of shared pages.");
1570 STAM_REG(pVM, &pPGM->cZeroPages, STAMTYPE_U32, "/PGM/Page/cZeroPages", STAMUNIT_OCCURENCES, "The number of zero backed pages.");
1571 STAM_REG(pVM, &pPGM->ChunkR3Map.c, STAMTYPE_U32, "/PGM/ChunkR3Map/c", STAMUNIT_OCCURENCES, "Number of mapped chunks.");
1572 STAM_REG(pVM, &pPGM->ChunkR3Map.cMax, STAMTYPE_U32, "/PGM/ChunkR3Map/cMax", STAMUNIT_OCCURENCES, "Maximum number of mapped chunks.");
1573
1574 /* Common - stats */
1575#ifdef PGMPOOL_WITH_GCPHYS_TRACKING
1576 STAM_REG(pVM, &pPGM->StatTrackVirgin, STAMTYPE_COUNTER, "/PGM/Track/Virgin", STAMUNIT_OCCURENCES, "The number of first time shadowings");
1577 STAM_REG(pVM, &pPGM->StatTrackAliased, STAMTYPE_COUNTER, "/PGM/Track/Aliased", STAMUNIT_OCCURENCES, "The number of times switching to cRef2, i.e. the page is being shadowed by two PTs.");
1578 STAM_REG(pVM, &pPGM->StatTrackAliasedMany, STAMTYPE_COUNTER, "/PGM/Track/AliasedMany", STAMUNIT_OCCURENCES, "The number of times we're tracking using cRef2.");
1579 STAM_REG(pVM, &pPGM->StatTrackAliasedLots, STAMTYPE_COUNTER, "/PGM/Track/AliasedLots", STAMUNIT_OCCURENCES, "The number of times we're hitting pages which has overflowed cRef2");
1580 STAM_REG(pVM, &pPGM->StatTrackOverflows, STAMTYPE_COUNTER, "/PGM/Track/Overflows", STAMUNIT_OCCURENCES, "The number of times the extent list grows to long.");
1581 STAM_REG(pVM, &pPGM->StatTrackDeref, STAMTYPE_PROFILE, "/PGM/Track/Deref", STAMUNIT_OCCURENCES, "Profiling of SyncPageWorkerTrackDeref (expensive).");
1582#endif
1583 for (i = 0; i < RT_ELEMENTS(pPGM->StatSyncPtPD); i++)
1584 STAMR3RegisterF(pVM, &pPGM->StatSyncPtPD[i], STAMTYPE_COUNTER, STAMVISIBILITY_USED, STAMUNIT_OCCURENCES,
1585 "The number of SyncPT per PD n.", "/PGM/PDSyncPT/%04X", i);
1586 for (i = 0; i < RT_ELEMENTS(pPGM->StatSyncPagePD); i++)
1587 STAMR3RegisterF(pVM, &pPGM->StatSyncPagePD[i], STAMTYPE_COUNTER, STAMVISIBILITY_USED, STAMUNIT_OCCURENCES,
1588 "The number of SyncPage per PD n.", "/PGM/PDSyncPage/%04X", i);
1589
1590 /* R3 only: */
1591 STAM_REG(pVM, &pPGM->StatR3DetectedConflicts, STAMTYPE_COUNTER, "/PGM/R3/DetectedConflicts", STAMUNIT_OCCURENCES, "The number of times PGMR3CheckMappingConflicts() detected a conflict.");
1592 STAM_REG(pVM, &pPGM->StatR3ResolveConflict, STAMTYPE_PROFILE, "/PGM/R3/ResolveConflict", STAMUNIT_TICKS_PER_CALL, "pgmR3SyncPTResolveConflict() profiling (includes the entire relocation).");
1593 STAM_REG(pVM, &pPGM->StatR3GuestPDWrite, STAMTYPE_COUNTER, "/PGM/R3/PDWrite", STAMUNIT_OCCURENCES, "The total number of times pgmHCGuestPDWriteHandler() was called.");
1594 STAM_REG(pVM, &pPGM->StatR3GuestPDWriteConflict, STAMTYPE_COUNTER, "/PGM/R3/PDWriteConflict", STAMUNIT_OCCURENCES, "The number of times pgmHCGuestPDWriteHandler() detected a conflict.");
1595 STAM_REG(pVM, &pPGM->StatR3DynRamTotal, STAMTYPE_COUNTER, "/PGM/DynAlloc/TotalAlloc", STAMUNIT_MEGABYTES, "Allocated MBs of guest ram.");
1596 STAM_REG(pVM, &pPGM->StatR3DynRamGrow, STAMTYPE_COUNTER, "/PGM/DynAlloc/Grow", STAMUNIT_OCCURENCES, "Nr of pgmr3PhysGrowRange calls.");
1597
1598 /* R0 only: */
1599 STAM_REG(pVM, &pPGM->StatR0DynMapMigrateInvlPg, STAMTYPE_COUNTER, "/PGM/R0/DynMapMigrateInvlPg", STAMUNIT_OCCURENCES, "invlpg count in PGMDynMapMigrateAutoSet.");
1600 STAM_REG(pVM, &pPGM->StatR0DynMapGCPageInl, STAMTYPE_PROFILE, "/PGM/R0/DynMapPageGCPageInl", STAMUNIT_TICKS_PER_CALL, "Calls to pgmR0DynMapGCPageInlined.");
1601 STAM_REG(pVM, &pPGM->StatR0DynMapGCPageInlHits, STAMTYPE_COUNTER, "/PGM/R0/DynMapPageGCPageInl/Hits", STAMUNIT_OCCURENCES, "Hash table lookup hits.");
1602 STAM_REG(pVM, &pPGM->StatR0DynMapGCPageInlMisses, STAMTYPE_COUNTER, "/PGM/R0/DynMapPageGCPageInl/Misses", STAMUNIT_OCCURENCES, "Misses that falls back to code common with PGMDynMapHCPage.");
1603 STAM_REG(pVM, &pPGM->StatR0DynMapGCPageInlRamHits, STAMTYPE_COUNTER, "/PGM/R0/DynMapPageGCPageInl/RamHits", STAMUNIT_OCCURENCES, "1st ram range hits.");
1604 STAM_REG(pVM, &pPGM->StatR0DynMapGCPageInlRamMisses, STAMTYPE_COUNTER, "/PGM/R0/DynMapPageGCPageInl/RamMisses", STAMUNIT_OCCURENCES, "1st ram range misses, takes slow path.");
1605 STAM_REG(pVM, &pPGM->StatR0DynMapHCPageInl, STAMTYPE_PROFILE, "/PGM/R0/DynMapPageHCPageInl", STAMUNIT_TICKS_PER_CALL, "Calls to pgmR0DynMapHCPageInlined.");
1606 STAM_REG(pVM, &pPGM->StatR0DynMapHCPageInlHits, STAMTYPE_COUNTER, "/PGM/R0/DynMapPageHCPageInl/Hits", STAMUNIT_OCCURENCES, "Hash table lookup hits.");
1607 STAM_REG(pVM, &pPGM->StatR0DynMapHCPageInlMisses, STAMTYPE_COUNTER, "/PGM/R0/DynMapPageHCPageInl/Misses", STAMUNIT_OCCURENCES, "Misses that falls back to code common with PGMDynMapHCPage.");
1608 STAM_REG(pVM, &pPGM->StatR0DynMapPage, STAMTYPE_COUNTER, "/PGM/R0/DynMapPage", STAMUNIT_OCCURENCES, "Calls to pgmR0DynMapPage");
1609 STAM_REG(pVM, &pPGM->StatR0DynMapSetOptimize, STAMTYPE_COUNTER, "/PGM/R0/DynMapPage/SetOptimize", STAMUNIT_OCCURENCES, "Calls to pgmDynMapOptimizeAutoSet.");
1610 STAM_REG(pVM, &pPGM->StatR0DynMapSetSearchFlushes, STAMTYPE_COUNTER, "/PGM/R0/DynMapPage/SetSearchFlushes",STAMUNIT_OCCURENCES, "Set search restorting to subset flushes.");
1611 STAM_REG(pVM, &pPGM->StatR0DynMapSetSearchHits, STAMTYPE_COUNTER, "/PGM/R0/DynMapPage/SetSearchHits", STAMUNIT_OCCURENCES, "Set search hits.");
1612 STAM_REG(pVM, &pPGM->StatR0DynMapSetSearchMisses, STAMTYPE_COUNTER, "/PGM/R0/DynMapPage/SetSearchMisses", STAMUNIT_OCCURENCES, "Set search misses.");
1613 STAM_REG(pVM, &pPGM->StatR0DynMapHCPage, STAMTYPE_PROFILE, "/PGM/R0/DynMapPage/HCPage", STAMUNIT_TICKS_PER_CALL, "Calls to PGMDynMapHCPage (ring-0).");
1614 STAM_REG(pVM, &pPGM->StatR0DynMapPageHits0, STAMTYPE_COUNTER, "/PGM/R0/DynMapPage/Hits0", STAMUNIT_OCCURENCES, "Hits at iPage+0");
1615 STAM_REG(pVM, &pPGM->StatR0DynMapPageHits1, STAMTYPE_COUNTER, "/PGM/R0/DynMapPage/Hits1", STAMUNIT_OCCURENCES, "Hits at iPage+1");
1616 STAM_REG(pVM, &pPGM->StatR0DynMapPageHits2, STAMTYPE_COUNTER, "/PGM/R0/DynMapPage/Hits2", STAMUNIT_OCCURENCES, "Hits at iPage+2");
1617 STAM_REG(pVM, &pPGM->StatR0DynMapPageInvlPg, STAMTYPE_COUNTER, "/PGM/R0/DynMapPage/InvlPg", STAMUNIT_OCCURENCES, "invlpg count in pgmR0DynMapPageSlow.");
1618 STAM_REG(pVM, &pPGM->StatR0DynMapPageSlow, STAMTYPE_COUNTER, "/PGM/R0/DynMapPage/Slow", STAMUNIT_OCCURENCES, "Calls to pgmR0DynMapPageSlow - subtract this from pgmR0DynMapPage to get 1st level hits.");
1619 STAM_REG(pVM, &pPGM->StatR0DynMapPageSlowLoopHits, STAMTYPE_COUNTER, "/PGM/R0/DynMapPage/SlowLoopHits" , STAMUNIT_OCCURENCES, "Hits in the loop path.");
1620 STAM_REG(pVM, &pPGM->StatR0DynMapPageSlowLoopMisses, STAMTYPE_COUNTER, "/PGM/R0/DynMapPage/SlowLoopMisses", STAMUNIT_OCCURENCES, "Misses in the loop path. NonLoopMisses = Slow - SlowLoopHit - SlowLoopMisses");
1621 //STAM_REG(pVM, &pPGM->StatR0DynMapPageSlowLostHits, STAMTYPE_COUNTER, "/PGM/R0/DynMapPage/SlowLostHits", STAMUNIT_OCCURENCES, "Lost hits.");
1622 STAM_REG(pVM, &pPGM->StatR0DynMapSubsets, STAMTYPE_COUNTER, "/PGM/R0/Subsets", STAMUNIT_OCCURENCES, "Times PGMDynMapPushAutoSubset was called.");
1623 STAM_REG(pVM, &pPGM->StatR0DynMapPopFlushes, STAMTYPE_COUNTER, "/PGM/R0/SubsetPopFlushes", STAMUNIT_OCCURENCES, "Times PGMDynMapPopAutoSubset flushes the subset.");
1624 STAM_REG(pVM, &pPGM->aStatR0DynMapSetSize[0], STAMTYPE_COUNTER, "/PGM/R0/SetSize000..09", STAMUNIT_OCCURENCES, "00-09% filled");
1625 STAM_REG(pVM, &pPGM->aStatR0DynMapSetSize[1], STAMTYPE_COUNTER, "/PGM/R0/SetSize010..19", STAMUNIT_OCCURENCES, "10-19% filled");
1626 STAM_REG(pVM, &pPGM->aStatR0DynMapSetSize[2], STAMTYPE_COUNTER, "/PGM/R0/SetSize020..29", STAMUNIT_OCCURENCES, "20-29% filled");
1627 STAM_REG(pVM, &pPGM->aStatR0DynMapSetSize[3], STAMTYPE_COUNTER, "/PGM/R0/SetSize030..39", STAMUNIT_OCCURENCES, "30-39% filled");
1628 STAM_REG(pVM, &pPGM->aStatR0DynMapSetSize[4], STAMTYPE_COUNTER, "/PGM/R0/SetSize040..49", STAMUNIT_OCCURENCES, "40-49% filled");
1629 STAM_REG(pVM, &pPGM->aStatR0DynMapSetSize[5], STAMTYPE_COUNTER, "/PGM/R0/SetSize050..59", STAMUNIT_OCCURENCES, "50-59% filled");
1630 STAM_REG(pVM, &pPGM->aStatR0DynMapSetSize[6], STAMTYPE_COUNTER, "/PGM/R0/SetSize060..69", STAMUNIT_OCCURENCES, "60-69% filled");
1631 STAM_REG(pVM, &pPGM->aStatR0DynMapSetSize[7], STAMTYPE_COUNTER, "/PGM/R0/SetSize070..79", STAMUNIT_OCCURENCES, "70-79% filled");
1632 STAM_REG(pVM, &pPGM->aStatR0DynMapSetSize[8], STAMTYPE_COUNTER, "/PGM/R0/SetSize080..89", STAMUNIT_OCCURENCES, "80-89% filled");
1633 STAM_REG(pVM, &pPGM->aStatR0DynMapSetSize[9], STAMTYPE_COUNTER, "/PGM/R0/SetSize090..99", STAMUNIT_OCCURENCES, "90-99% filled");
1634 STAM_REG(pVM, &pPGM->aStatR0DynMapSetSize[10], STAMTYPE_COUNTER, "/PGM/R0/SetSize100", STAMUNIT_OCCURENCES, "100% filled");
1635
1636 /* GC only: */
1637 STAM_REG(pVM, &pPGM->StatRCDynMapCacheHits, STAMTYPE_COUNTER, "/PGM/RC/DynMapCache/Hits" , STAMUNIT_OCCURENCES, "Number of dynamic page mapping cache hits.");
1638 STAM_REG(pVM, &pPGM->StatRCDynMapCacheMisses, STAMTYPE_COUNTER, "/PGM/RC/DynMapCache/Misses" , STAMUNIT_OCCURENCES, "Number of dynamic page mapping cache misses.");
1639 STAM_REG(pVM, &pPGM->StatRCInvlPgConflict, STAMTYPE_COUNTER, "/PGM/RC/InvlPgConflict", STAMUNIT_OCCURENCES, "Number of times PGMInvalidatePage() detected a mapping conflict.");
1640 STAM_REG(pVM, &pPGM->StatRCInvlPgSyncMonCR3, STAMTYPE_COUNTER, "/PGM/RC/InvlPgSyncMonitorCR3", STAMUNIT_OCCURENCES, "Number of times PGMInvalidatePage() ran into PGM_SYNC_MONITOR_CR3.");
1641
1642 /* RZ only: */
1643 STAM_REG(pVM, &pPGM->StatRZTrap0e, STAMTYPE_PROFILE, "/PGM/RZ/Trap0e", STAMUNIT_TICKS_PER_CALL, "Profiling of the PGMTrap0eHandler() body.");
1644 STAM_REG(pVM, &pPGM->StatRZTrap0eTimeCheckPageFault, STAMTYPE_PROFILE, "/PGM/RZ/Trap0e/Time/CheckPageFault", STAMUNIT_TICKS_PER_CALL, "Profiling of checking for dirty/access emulation faults.");
1645 STAM_REG(pVM, &pPGM->StatRZTrap0eTimeSyncPT, STAMTYPE_PROFILE, "/PGM/RZ/Trap0e/Time/SyncPT", STAMUNIT_TICKS_PER_CALL, "Profiling of lazy page table syncing.");
1646 STAM_REG(pVM, &pPGM->StatRZTrap0eTimeMapping, STAMTYPE_PROFILE, "/PGM/RZ/Trap0e/Time/Mapping", STAMUNIT_TICKS_PER_CALL, "Profiling of checking virtual mappings.");
1647 STAM_REG(pVM, &pPGM->StatRZTrap0eTimeOutOfSync, STAMTYPE_PROFILE, "/PGM/RZ/Trap0e/Time/OutOfSync", STAMUNIT_TICKS_PER_CALL, "Profiling of out of sync page handling.");
1648 STAM_REG(pVM, &pPGM->StatRZTrap0eTimeHandlers, STAMTYPE_PROFILE, "/PGM/RZ/Trap0e/Time/Handlers", STAMUNIT_TICKS_PER_CALL, "Profiling of checking handlers.");
1649 STAM_REG(pVM, &pPGM->StatRZTrap0eTime2CSAM, STAMTYPE_PROFILE, "/PGM/RZ/Trap0e/Time2/CSAM", STAMUNIT_TICKS_PER_CALL, "Profiling of the Trap0eHandler body when the cause is CSAM.");
1650 STAM_REG(pVM, &pPGM->StatRZTrap0eTime2DirtyAndAccessed, STAMTYPE_PROFILE, "/PGM/RZ/Trap0e/Time2/DirtyAndAccessedBits", STAMUNIT_TICKS_PER_CALL, "Profiling of the Trap0eHandler body when the cause is dirty and/or accessed bit emulation.");
1651 STAM_REG(pVM, &pPGM->StatRZTrap0eTime2GuestTrap, STAMTYPE_PROFILE, "/PGM/RZ/Trap0e/Time2/GuestTrap", STAMUNIT_TICKS_PER_CALL, "Profiling of the Trap0eHandler body when the cause is a guest trap.");
1652 STAM_REG(pVM, &pPGM->StatRZTrap0eTime2HndPhys, STAMTYPE_PROFILE, "/PGM/RZ/Trap0e/Time2/HandlerPhysical", STAMUNIT_TICKS_PER_CALL, "Profiling of the Trap0eHandler body when the cause is a physical handler.");
1653 STAM_REG(pVM, &pPGM->StatRZTrap0eTime2HndVirt, STAMTYPE_PROFILE, "/PGM/RZ/Trap0e/Time2/HandlerVirtual", STAMUNIT_TICKS_PER_CALL, "Profiling of the Trap0eHandler body when the cause is a virtual handler.");
1654 STAM_REG(pVM, &pPGM->StatRZTrap0eTime2HndUnhandled, STAMTYPE_PROFILE, "/PGM/RZ/Trap0e/Time2/HandlerUnhandled", STAMUNIT_TICKS_PER_CALL, "Profiling of the Trap0eHandler body when the cause is access outside the monitored areas of a monitored page.");
1655 STAM_REG(pVM, &pPGM->StatRZTrap0eTime2Misc, STAMTYPE_PROFILE, "/PGM/RZ/Trap0e/Time2/Misc", STAMUNIT_TICKS_PER_CALL, "Profiling of the Trap0eHandler body when the cause is not known.");
1656 STAM_REG(pVM, &pPGM->StatRZTrap0eTime2OutOfSync, STAMTYPE_PROFILE, "/PGM/RZ/Trap0e/Time2/OutOfSync", STAMUNIT_TICKS_PER_CALL, "Profiling of the Trap0eHandler body when the cause is an out-of-sync page.");
1657 STAM_REG(pVM, &pPGM->StatRZTrap0eTime2OutOfSyncHndPhys, STAMTYPE_PROFILE, "/PGM/RZ/Trap0e/Time2/OutOfSyncHndPhys", STAMUNIT_TICKS_PER_CALL, "Profiling of the Trap0eHandler body when the cause is an out-of-sync physical handler page.");
1658 STAM_REG(pVM, &pPGM->StatRZTrap0eTime2OutOfSyncHndVirt, STAMTYPE_PROFILE, "/PGM/RZ/Trap0e/Time2/OutOfSyncHndVirt", STAMUNIT_TICKS_PER_CALL, "Profiling of the Trap0eHandler body when the cause is an out-of-sync virtual handler page.");
1659 STAM_REG(pVM, &pPGM->StatRZTrap0eTime2OutOfSyncHndObs, STAMTYPE_PROFILE, "/PGM/RZ/Trap0e/Time2/OutOfSyncObsHnd", STAMUNIT_TICKS_PER_CALL, "Profiling of the Trap0eHandler body when the cause is an obsolete handler page.");
1660 STAM_REG(pVM, &pPGM->StatRZTrap0eTime2SyncPT, STAMTYPE_PROFILE, "/PGM/RZ/Trap0e/Time2/SyncPT", STAMUNIT_TICKS_PER_CALL, "Profiling of the Trap0eHandler body when the cause is lazy syncing of a PT.");
1661 STAM_REG(pVM, &pPGM->StatRZTrap0eConflicts, STAMTYPE_COUNTER, "/PGM/RZ/Trap0e/Conflicts", STAMUNIT_OCCURENCES, "The number of times #PF was caused by an undetected conflict.");
1662 STAM_REG(pVM, &pPGM->StatRZTrap0eHandlersMapping, STAMTYPE_COUNTER, "/PGM/RZ/Trap0e/Handlers/Mapping", STAMUNIT_OCCURENCES, "Number of traps due to access handlers in mappings.");
1663 STAM_REG(pVM, &pPGM->StatRZTrap0eHandlersOutOfSync, STAMTYPE_COUNTER, "/PGM/RZ/Trap0e/Handlers/OutOfSync", STAMUNIT_OCCURENCES, "Number of traps due to out-of-sync handled pages.");
1664 STAM_REG(pVM, &pPGM->StatRZTrap0eHandlersPhysical, STAMTYPE_COUNTER, "/PGM/RZ/Trap0e/Handlers/Physical", STAMUNIT_OCCURENCES, "Number of traps due to physical access handlers.");
1665 STAM_REG(pVM, &pPGM->StatRZTrap0eHandlersVirtual, STAMTYPE_COUNTER, "/PGM/RZ/Trap0e/Handlers/Virtual", STAMUNIT_OCCURENCES, "Number of traps due to virtual access handlers.");
1666 STAM_REG(pVM, &pPGM->StatRZTrap0eHandlersVirtualByPhys, STAMTYPE_COUNTER, "/PGM/RZ/Trap0e/Handlers/VirtualByPhys", STAMUNIT_OCCURENCES, "Number of traps due to virtual access handlers by physical address.");
1667 STAM_REG(pVM, &pPGM->StatRZTrap0eHandlersVirtualUnmarked,STAMTYPE_COUNTER,"/PGM/RZ/Trap0e/Handlers/VirtualUnmarked",STAMUNIT_OCCURENCES, "Number of traps due to virtual access handlers by virtual address (without proper physical flags).");
1668 STAM_REG(pVM, &pPGM->StatRZTrap0eHandlersUnhandled, STAMTYPE_COUNTER, "/PGM/RZ/Trap0e/Handlers/Unhandled", STAMUNIT_OCCURENCES, "Number of traps due to access outside range of monitored page(s).");
1669 STAM_REG(pVM, &pPGM->StatRZTrap0eHandlersInvalid, STAMTYPE_COUNTER, "/PGM/RZ/Trap0e/Handlers/Invalid", STAMUNIT_OCCURENCES, "Number of traps due to access to invalid physical memory.");
1670 STAM_REG(pVM, &pPGM->StatRZTrap0eUSNotPresentRead, STAMTYPE_COUNTER, "/PGM/RZ/Trap0e/Err/User/NPRead", STAMUNIT_OCCURENCES, "Number of user mode not present read page faults.");
1671 STAM_REG(pVM, &pPGM->StatRZTrap0eUSNotPresentWrite, STAMTYPE_COUNTER, "/PGM/RZ/Trap0e/Err/User/NPWrite", STAMUNIT_OCCURENCES, "Number of user mode not present write page faults.");
1672 STAM_REG(pVM, &pPGM->StatRZTrap0eUSWrite, STAMTYPE_COUNTER, "/PGM/RZ/Trap0e/Err/User/Write", STAMUNIT_OCCURENCES, "Number of user mode write page faults.");
1673 STAM_REG(pVM, &pPGM->StatRZTrap0eUSReserved, STAMTYPE_COUNTER, "/PGM/RZ/Trap0e/Err/User/Reserved", STAMUNIT_OCCURENCES, "Number of user mode reserved bit page faults.");
1674 STAM_REG(pVM, &pPGM->StatRZTrap0eUSNXE, STAMTYPE_COUNTER, "/PGM/RZ/Trap0e/Err/User/NXE", STAMUNIT_OCCURENCES, "Number of user mode NXE page faults.");
1675 STAM_REG(pVM, &pPGM->StatRZTrap0eUSRead, STAMTYPE_COUNTER, "/PGM/RZ/Trap0e/Err/User/Read", STAMUNIT_OCCURENCES, "Number of user mode read page faults.");
1676 STAM_REG(pVM, &pPGM->StatRZTrap0eSVNotPresentRead, STAMTYPE_COUNTER, "/PGM/RZ/Trap0e/Err/Supervisor/NPRead", STAMUNIT_OCCURENCES, "Number of supervisor mode not present read page faults.");
1677 STAM_REG(pVM, &pPGM->StatRZTrap0eSVNotPresentWrite, STAMTYPE_COUNTER, "/PGM/RZ/Trap0e/Err/Supervisor/NPWrite", STAMUNIT_OCCURENCES, "Number of supervisor mode not present write page faults.");
1678 STAM_REG(pVM, &pPGM->StatRZTrap0eSVWrite, STAMTYPE_COUNTER, "/PGM/RZ/Trap0e/Err/Supervisor/Write", STAMUNIT_OCCURENCES, "Number of supervisor mode write page faults.");
1679 STAM_REG(pVM, &pPGM->StatRZTrap0eSVReserved, STAMTYPE_COUNTER, "/PGM/RZ/Trap0e/Err/Supervisor/Reserved", STAMUNIT_OCCURENCES, "Number of supervisor mode reserved bit page faults.");
1680 STAM_REG(pVM, &pPGM->StatRZTrap0eSNXE, STAMTYPE_COUNTER, "/PGM/RZ/Trap0e/Err/Supervisor/NXE", STAMUNIT_OCCURENCES, "Number of supervisor mode NXE page faults.");
1681 STAM_REG(pVM, &pPGM->StatRZTrap0eGuestPF, STAMTYPE_COUNTER, "/PGM/RZ/Trap0e/GuestPF", STAMUNIT_OCCURENCES, "Number of real guest page faults.");
1682 STAM_REG(pVM, &pPGM->StatRZTrap0eGuestPFUnh, STAMTYPE_COUNTER, "/PGM/RZ/Trap0e/GuestPF/Unhandled", STAMUNIT_OCCURENCES, "Number of real guest page faults from the 'unhandled' case.");
1683 STAM_REG(pVM, &pPGM->StatRZTrap0eGuestPFMapping, STAMTYPE_COUNTER, "/PGM/RZ/Trap0e/GuestPF/InMapping", STAMUNIT_OCCURENCES, "Number of real guest page faults in a mapping.");
1684 STAM_REG(pVM, &pPGM->StatRZTrap0eWPEmulInRZ, STAMTYPE_COUNTER, "/PGM/RZ/Trap0e/WP/InRZ", STAMUNIT_OCCURENCES, "Number of guest page faults due to X86_CR0_WP emulation.");
1685 STAM_REG(pVM, &pPGM->StatRZTrap0eWPEmulToR3, STAMTYPE_COUNTER, "/PGM/RZ/Trap0e/WP/ToR3", STAMUNIT_OCCURENCES, "Number of guest page faults due to X86_CR0_WP emulation (forward to R3 for emulation).");
1686 for (i = 0; i < RT_ELEMENTS(pPGM->StatRZTrap0ePD); i++)
1687 STAMR3RegisterF(pVM, &pPGM->StatRZTrap0ePD[i], STAMTYPE_COUNTER, STAMVISIBILITY_USED, STAMUNIT_OCCURENCES,
1688 "The number of traps in page directory n.", "/PGM/RZ/Trap0e/PD/%04X", i);
1689 STAM_REG(pVM, &pPGM->StatRZGuestCR3WriteHandled, STAMTYPE_COUNTER, "/PGM/RZ/CR3WriteHandled", STAMUNIT_OCCURENCES, "The number of times the Guest CR3 change was successfully handled.");
1690 STAM_REG(pVM, &pPGM->StatRZGuestCR3WriteUnhandled, STAMTYPE_COUNTER, "/PGM/RZ/CR3WriteUnhandled", STAMUNIT_OCCURENCES, "The number of times the Guest CR3 change was passed back to the recompiler.");
1691 STAM_REG(pVM, &pPGM->StatRZGuestCR3WriteConflict, STAMTYPE_COUNTER, "/PGM/RZ/CR3WriteConflict", STAMUNIT_OCCURENCES, "The number of times the Guest CR3 monitoring detected a conflict.");
1692 STAM_REG(pVM, &pPGM->StatRZGuestROMWriteHandled, STAMTYPE_COUNTER, "/PGM/RZ/ROMWriteHandled", STAMUNIT_OCCURENCES, "The number of times the Guest ROM change was successfully handled.");
1693 STAM_REG(pVM, &pPGM->StatRZGuestROMWriteUnhandled, STAMTYPE_COUNTER, "/PGM/RZ/ROMWriteUnhandled", STAMUNIT_OCCURENCES, "The number of times the Guest ROM change was passed back to the recompiler.");
1694
1695 /* HC only: */
1696
1697 /* RZ & R3: */
1698 STAM_REG(pVM, &pPGM->StatRZSyncCR3, STAMTYPE_PROFILE, "/PGM/RZ/SyncCR3", STAMUNIT_TICKS_PER_CALL, "Profiling of the PGMSyncCR3() body.");
1699 STAM_REG(pVM, &pPGM->StatRZSyncCR3Handlers, STAMTYPE_PROFILE, "/PGM/RZ/SyncCR3/Handlers", STAMUNIT_TICKS_PER_CALL, "Profiling of the PGMSyncCR3() update handler section.");
1700 STAM_REG(pVM, &pPGM->StatRZSyncCR3HandlerVirtualUpdate, STAMTYPE_PROFILE, "/PGM/RZ/SyncCR3/Handlers/VirtualUpdate", STAMUNIT_TICKS_PER_CALL, "Profiling of the virtual handler updates.");
1701 STAM_REG(pVM, &pPGM->StatRZSyncCR3HandlerVirtualReset, STAMTYPE_PROFILE, "/PGM/RZ/SyncCR3/Handlers/VirtualReset", STAMUNIT_TICKS_PER_CALL, "Profiling of the virtual handler resets.");
1702 STAM_REG(pVM, &pPGM->StatRZSyncCR3Global, STAMTYPE_COUNTER, "/PGM/RZ/SyncCR3/Global", STAMUNIT_OCCURENCES, "The number of global CR3 syncs.");
1703 STAM_REG(pVM, &pPGM->StatRZSyncCR3NotGlobal, STAMTYPE_COUNTER, "/PGM/RZ/SyncCR3/NotGlobal", STAMUNIT_OCCURENCES, "The number of non-global CR3 syncs.");
1704 STAM_REG(pVM, &pPGM->StatRZSyncCR3DstCacheHit, STAMTYPE_COUNTER, "/PGM/RZ/SyncCR3/DstChacheHit", STAMUNIT_OCCURENCES, "The number of times we got some kind of a cache hit.");
1705 STAM_REG(pVM, &pPGM->StatRZSyncCR3DstFreed, STAMTYPE_COUNTER, "/PGM/RZ/SyncCR3/DstFreed", STAMUNIT_OCCURENCES, "The number of times we've had to free a shadow entry.");
1706 STAM_REG(pVM, &pPGM->StatRZSyncCR3DstFreedSrcNP, STAMTYPE_COUNTER, "/PGM/RZ/SyncCR3/DstFreedSrcNP", STAMUNIT_OCCURENCES, "The number of times we've had to free a shadow entry for which the source entry was not present.");
1707 STAM_REG(pVM, &pPGM->StatRZSyncCR3DstNotPresent, STAMTYPE_COUNTER, "/PGM/RZ/SyncCR3/DstNotPresent", STAMUNIT_OCCURENCES, "The number of times we've encountered a not present shadow entry for a present guest entry.");
1708 STAM_REG(pVM, &pPGM->StatRZSyncCR3DstSkippedGlobalPD, STAMTYPE_COUNTER, "/PGM/RZ/SyncCR3/DstSkippedGlobalPD", STAMUNIT_OCCURENCES, "The number of times a global page directory wasn't flushed.");
1709 STAM_REG(pVM, &pPGM->StatRZSyncCR3DstSkippedGlobalPT, STAMTYPE_COUNTER, "/PGM/RZ/SyncCR3/DstSkippedGlobalPT", STAMUNIT_OCCURENCES, "The number of times a page table with only global entries wasn't flushed.");
1710 STAM_REG(pVM, &pPGM->StatRZSyncPT, STAMTYPE_PROFILE, "/PGM/RZ/SyncPT", STAMUNIT_TICKS_PER_CALL, "Profiling of the pfnSyncPT() body.");
1711 STAM_REG(pVM, &pPGM->StatRZSyncPTFailed, STAMTYPE_COUNTER, "/PGM/RZ/SyncPT/Failed", STAMUNIT_OCCURENCES, "The number of times pfnSyncPT() failed.");
1712 STAM_REG(pVM, &pPGM->StatRZSyncPT4K, STAMTYPE_COUNTER, "/PGM/RZ/SyncPT/4K", STAMUNIT_OCCURENCES, "Nr of 4K PT syncs");
1713 STAM_REG(pVM, &pPGM->StatRZSyncPT4M, STAMTYPE_COUNTER, "/PGM/RZ/SyncPT/4M", STAMUNIT_OCCURENCES, "Nr of 4M PT syncs");
1714 STAM_REG(pVM, &pPGM->StatRZSyncPagePDNAs, STAMTYPE_COUNTER, "/PGM/RZ/SyncPagePDNAs", STAMUNIT_OCCURENCES, "The number of time we've marked a PD not present from SyncPage to virtualize the accessed bit.");
1715 STAM_REG(pVM, &pPGM->StatRZSyncPagePDOutOfSync, STAMTYPE_COUNTER, "/PGM/RZ/SyncPagePDOutOfSync", STAMUNIT_OCCURENCES, "The number of time we've encountered an out-of-sync PD in SyncPage.");
1716 STAM_REG(pVM, &pPGM->StatRZAccessedPage, STAMTYPE_COUNTER, "/PGM/RZ/AccessedPage", STAMUNIT_OCCURENCES, "The number of pages marked not present for accessed bit emulation.");
1717 STAM_REG(pVM, &pPGM->StatRZDirtyBitTracking, STAMTYPE_PROFILE, "/PGM/RZ/DirtyPage", STAMUNIT_TICKS_PER_CALL, "Profiling the dirty bit tracking in CheckPageFault().");
1718 STAM_REG(pVM, &pPGM->StatRZDirtyPage, STAMTYPE_COUNTER, "/PGM/RZ/DirtyPage/Mark", STAMUNIT_OCCURENCES, "The number of pages marked read-only for dirty bit tracking.");
1719 STAM_REG(pVM, &pPGM->StatRZDirtyPageBig, STAMTYPE_COUNTER, "/PGM/RZ/DirtyPage/MarkBig", STAMUNIT_OCCURENCES, "The number of 4MB pages marked read-only for dirty bit tracking.");
1720 STAM_REG(pVM, &pPGM->StatRZDirtyPageSkipped, STAMTYPE_COUNTER, "/PGM/RZ/DirtyPage/Skipped", STAMUNIT_OCCURENCES, "The number of pages already dirty or readonly.");
1721 STAM_REG(pVM, &pPGM->StatRZDirtyPageTrap, STAMTYPE_COUNTER, "/PGM/RZ/DirtyPage/Trap", STAMUNIT_OCCURENCES, "The number of traps generated for dirty bit tracking.");
1722 STAM_REG(pVM, &pPGM->StatRZDirtiedPage, STAMTYPE_COUNTER, "/PGM/RZ/DirtyPage/SetDirty", STAMUNIT_OCCURENCES, "The number of pages marked dirty because of write accesses.");
1723 STAM_REG(pVM, &pPGM->StatRZDirtyTrackRealPF, STAMTYPE_COUNTER, "/PGM/RZ/DirtyPage/RealPF", STAMUNIT_OCCURENCES, "The number of real pages faults during dirty bit tracking.");
1724 STAM_REG(pVM, &pPGM->StatRZPageAlreadyDirty, STAMTYPE_COUNTER, "/PGM/RZ/DirtyPage/AlreadySet", STAMUNIT_OCCURENCES, "The number of pages already marked dirty because of write accesses.");
1725 STAM_REG(pVM, &pPGM->StatRZInvalidatePage, STAMTYPE_PROFILE, "/PGM/RZ/InvalidatePage", STAMUNIT_TICKS_PER_CALL, "PGMInvalidatePage() profiling.");
1726 STAM_REG(pVM, &pPGM->StatRZInvalidatePage4KBPages, STAMTYPE_COUNTER, "/PGM/RZ/InvalidatePage/4KBPages", STAMUNIT_OCCURENCES, "The number of times PGMInvalidatePage() was called for a 4KB page.");
1727 STAM_REG(pVM, &pPGM->StatRZInvalidatePage4MBPages, STAMTYPE_COUNTER, "/PGM/RZ/InvalidatePage/4MBPages", STAMUNIT_OCCURENCES, "The number of times PGMInvalidatePage() was called for a 4MB page.");
1728 STAM_REG(pVM, &pPGM->StatRZInvalidatePage4MBPagesSkip, STAMTYPE_COUNTER, "/PGM/RZ/InvalidatePage/4MBPagesSkip",STAMUNIT_OCCURENCES, "The number of times PGMInvalidatePage() skipped a 4MB page.");
1729 STAM_REG(pVM, &pPGM->StatRZInvalidatePagePDMappings, STAMTYPE_COUNTER, "/PGM/RZ/InvalidatePage/PDMappings", STAMUNIT_OCCURENCES, "The number of times PGMInvalidatePage() was called for a page directory containing mappings (no conflict).");
1730 STAM_REG(pVM, &pPGM->StatRZInvalidatePagePDNAs, STAMTYPE_COUNTER, "/PGM/RZ/InvalidatePage/PDNAs", STAMUNIT_OCCURENCES, "The number of times PGMInvalidatePage() was called for a not accessed page directory.");
1731 STAM_REG(pVM, &pPGM->StatRZInvalidatePagePDNPs, STAMTYPE_COUNTER, "/PGM/RZ/InvalidatePage/PDNPs", STAMUNIT_OCCURENCES, "The number of times PGMInvalidatePage() was called for a not present page directory.");
1732 STAM_REG(pVM, &pPGM->StatRZInvalidatePagePDOutOfSync, STAMTYPE_COUNTER, "/PGM/RZ/InvalidatePage/PDOutOfSync", STAMUNIT_OCCURENCES, "The number of times PGMInvalidatePage() was called for an out of sync page directory.");
1733 STAM_REG(pVM, &pPGM->StatRZInvalidatePageSkipped, STAMTYPE_COUNTER, "/PGM/RZ/InvalidatePage/Skipped", STAMUNIT_OCCURENCES, "The number of times PGMInvalidatePage() was skipped due to not present shw or pending pending SyncCR3.");
1734 STAM_REG(pVM, &pPGM->StatRZVirtHandlerSearchByPhys, STAMTYPE_PROFILE, "/PGM/RZ/VirtHandlerSearchByPhys", STAMUNIT_TICKS_PER_CALL, "Profiling of pgmHandlerVirtualFindByPhysAddr.");
1735 STAM_REG(pVM, &pPGM->StatRZPhysHandlerReset, STAMTYPE_COUNTER, "/PGM/RZ/PhysHandlerReset", STAMUNIT_OCCURENCES, "The number of times PGMHandlerPhysicalReset is called.");
1736 STAM_REG(pVM, &pPGM->StatRZPageOutOfSyncSupervisor, STAMTYPE_COUNTER, "/PGM/RZ/OutOfSync/SuperVisor", STAMUNIT_OCCURENCES, "Number of traps due to pages out of sync and times VerifyAccessSyncPage calls SyncPage.");
1737 STAM_REG(pVM, &pPGM->StatRZPageOutOfSyncUser, STAMTYPE_COUNTER, "/PGM/RZ/OutOfSync/User", STAMUNIT_OCCURENCES, "Number of traps due to pages out of sync and times VerifyAccessSyncPage calls SyncPage.");
1738 STAM_REG(pVM, &pPGM->StatRZPrefetch, STAMTYPE_PROFILE, "/PGM/RZ/Prefetch", STAMUNIT_TICKS_PER_CALL, "PGMPrefetchPage profiling.");
1739 STAM_REG(pVM, &pPGM->StatRZChunkR3MapTlbHits, STAMTYPE_COUNTER, "/PGM/ChunkR3Map/TlbHitsRZ", STAMUNIT_OCCURENCES, "TLB hits.");
1740 STAM_REG(pVM, &pPGM->StatRZChunkR3MapTlbMisses, STAMTYPE_COUNTER, "/PGM/ChunkR3Map/TlbMissesRZ", STAMUNIT_OCCURENCES, "TLB misses.");
1741 STAM_REG(pVM, &pPGM->StatRZPageMapTlbHits, STAMTYPE_COUNTER, "/PGM/RZ/Page/MapTlbHits", STAMUNIT_OCCURENCES, "TLB hits.");
1742 STAM_REG(pVM, &pPGM->StatRZPageMapTlbMisses, STAMTYPE_COUNTER, "/PGM/RZ/Page/MapTlbMisses", STAMUNIT_OCCURENCES, "TLB misses.");
1743 STAM_REG(pVM, &pPGM->StatRZPageReplaceShared, STAMTYPE_COUNTER, "/PGM/RZ/Page/ReplacedShared", STAMUNIT_OCCURENCES, "Times a shared page was replaced.");
1744 STAM_REG(pVM, &pPGM->StatRZPageReplaceZero, STAMTYPE_COUNTER, "/PGM/RZ/Page/ReplacedZero", STAMUNIT_OCCURENCES, "Times the zero page was replaced.");
1745/// @todo STAM_REG(pVM, &pPGM->StatRZPageHandyAllocs, STAMTYPE_COUNTER, "/PGM/RZ/Page/HandyAllocs", STAMUNIT_OCCURENCES, "Number of times we've allocated more handy pages.");
1746 STAM_REG(pVM, &pPGM->StatRZFlushTLB, STAMTYPE_PROFILE, "/PGM/RZ/FlushTLB", STAMUNIT_OCCURENCES, "Profiling of the PGMFlushTLB() body.");
1747 STAM_REG(pVM, &pPGM->StatRZFlushTLBNewCR3, STAMTYPE_COUNTER, "/PGM/RZ/FlushTLB/NewCR3", STAMUNIT_OCCURENCES, "The number of times PGMFlushTLB was called with a new CR3, non-global. (switch)");
1748 STAM_REG(pVM, &pPGM->StatRZFlushTLBNewCR3Global, STAMTYPE_COUNTER, "/PGM/RZ/FlushTLB/NewCR3Global", STAMUNIT_OCCURENCES, "The number of times PGMFlushTLB was called with a new CR3, global. (switch)");
1749 STAM_REG(pVM, &pPGM->StatRZFlushTLBSameCR3, STAMTYPE_COUNTER, "/PGM/RZ/FlushTLB/SameCR3", STAMUNIT_OCCURENCES, "The number of times PGMFlushTLB was called with the same CR3, non-global. (flush)");
1750 STAM_REG(pVM, &pPGM->StatRZFlushTLBSameCR3Global, STAMTYPE_COUNTER, "/PGM/RZ/FlushTLB/SameCR3Global", STAMUNIT_OCCURENCES, "The number of times PGMFlushTLB was called with the same CR3, global. (flush)");
1751 STAM_REG(pVM, &pPGM->StatRZGstModifyPage, STAMTYPE_PROFILE, "/PGM/RZ/GstModifyPage", STAMUNIT_TICKS_PER_CALL, "Profiling of the PGMGstModifyPage() body.");
1752
1753 STAM_REG(pVM, &pPGM->StatR3SyncCR3, STAMTYPE_PROFILE, "/PGM/R3/SyncCR3", STAMUNIT_TICKS_PER_CALL, "Profiling of the PGMSyncCR3() body.");
1754 STAM_REG(pVM, &pPGM->StatR3SyncCR3Handlers, STAMTYPE_PROFILE, "/PGM/R3/SyncCR3/Handlers", STAMUNIT_TICKS_PER_CALL, "Profiling of the PGMSyncCR3() update handler section.");
1755 STAM_REG(pVM, &pPGM->StatR3SyncCR3HandlerVirtualUpdate, STAMTYPE_PROFILE, "/PGM/R3/SyncCR3/Handlers/VirtualUpdate", STAMUNIT_TICKS_PER_CALL, "Profiling of the virtual handler updates.");
1756 STAM_REG(pVM, &pPGM->StatR3SyncCR3HandlerVirtualReset, STAMTYPE_PROFILE, "/PGM/R3/SyncCR3/Handlers/VirtualReset", STAMUNIT_TICKS_PER_CALL, "Profiling of the virtual handler resets.");
1757 STAM_REG(pVM, &pPGM->StatR3SyncCR3Global, STAMTYPE_COUNTER, "/PGM/R3/SyncCR3/Global", STAMUNIT_OCCURENCES, "The number of global CR3 syncs.");
1758 STAM_REG(pVM, &pPGM->StatR3SyncCR3NotGlobal, STAMTYPE_COUNTER, "/PGM/R3/SyncCR3/NotGlobal", STAMUNIT_OCCURENCES, "The number of non-global CR3 syncs.");
1759 STAM_REG(pVM, &pPGM->StatR3SyncCR3DstCacheHit, STAMTYPE_COUNTER, "/PGM/R3/SyncCR3/DstChacheHit", STAMUNIT_OCCURENCES, "The number of times we got some kind of a cache hit.");
1760 STAM_REG(pVM, &pPGM->StatR3SyncCR3DstFreed, STAMTYPE_COUNTER, "/PGM/R3/SyncCR3/DstFreed", STAMUNIT_OCCURENCES, "The number of times we've had to free a shadow entry.");
1761 STAM_REG(pVM, &pPGM->StatR3SyncCR3DstFreedSrcNP, STAMTYPE_COUNTER, "/PGM/R3/SyncCR3/DstFreedSrcNP", STAMUNIT_OCCURENCES, "The number of times we've had to free a shadow entry for which the source entry was not present.");
1762 STAM_REG(pVM, &pPGM->StatR3SyncCR3DstNotPresent, STAMTYPE_COUNTER, "/PGM/R3/SyncCR3/DstNotPresent", STAMUNIT_OCCURENCES, "The number of times we've encountered a not present shadow entry for a present guest entry.");
1763 STAM_REG(pVM, &pPGM->StatR3SyncCR3DstSkippedGlobalPD, STAMTYPE_COUNTER, "/PGM/R3/SyncCR3/DstSkippedGlobalPD", STAMUNIT_OCCURENCES, "The number of times a global page directory wasn't flushed.");
1764 STAM_REG(pVM, &pPGM->StatR3SyncCR3DstSkippedGlobalPT, STAMTYPE_COUNTER, "/PGM/R3/SyncCR3/DstSkippedGlobalPT", STAMUNIT_OCCURENCES, "The number of times a page table with only global entries wasn't flushed.");
1765 STAM_REG(pVM, &pPGM->StatR3SyncPT, STAMTYPE_PROFILE, "/PGM/R3/SyncPT", STAMUNIT_TICKS_PER_CALL, "Profiling of the pfnSyncPT() body.");
1766 STAM_REG(pVM, &pPGM->StatR3SyncPTFailed, STAMTYPE_COUNTER, "/PGM/R3/SyncPT/Failed", STAMUNIT_OCCURENCES, "The number of times pfnSyncPT() failed.");
1767 STAM_REG(pVM, &pPGM->StatR3SyncPT4K, STAMTYPE_COUNTER, "/PGM/R3/SyncPT/4K", STAMUNIT_OCCURENCES, "Nr of 4K PT syncs");
1768 STAM_REG(pVM, &pPGM->StatR3SyncPT4M, STAMTYPE_COUNTER, "/PGM/R3/SyncPT/4M", STAMUNIT_OCCURENCES, "Nr of 4M PT syncs");
1769 STAM_REG(pVM, &pPGM->StatR3SyncPagePDNAs, STAMTYPE_COUNTER, "/PGM/R3/SyncPagePDNAs", STAMUNIT_OCCURENCES, "The number of time we've marked a PD not present from SyncPage to virtualize the accessed bit.");
1770 STAM_REG(pVM, &pPGM->StatR3SyncPagePDOutOfSync, STAMTYPE_COUNTER, "/PGM/R3/SyncPagePDOutOfSync", STAMUNIT_OCCURENCES, "The number of time we've encountered an out-of-sync PD in SyncPage.");
1771 STAM_REG(pVM, &pPGM->StatR3AccessedPage, STAMTYPE_COUNTER, "/PGM/R3/AccessedPage", STAMUNIT_OCCURENCES, "The number of pages marked not present for accessed bit emulation.");
1772 STAM_REG(pVM, &pPGM->StatR3DirtyBitTracking, STAMTYPE_PROFILE, "/PGM/R3/DirtyPage", STAMUNIT_TICKS_PER_CALL, "Profiling the dirty bit tracking in CheckPageFault().");
1773 STAM_REG(pVM, &pPGM->StatR3DirtyPage, STAMTYPE_COUNTER, "/PGM/R3/DirtyPage/Mark", STAMUNIT_OCCURENCES, "The number of pages marked read-only for dirty bit tracking.");
1774 STAM_REG(pVM, &pPGM->StatR3DirtyPageBig, STAMTYPE_COUNTER, "/PGM/R3/DirtyPage/MarkBig", STAMUNIT_OCCURENCES, "The number of 4MB pages marked read-only for dirty bit tracking.");
1775 STAM_REG(pVM, &pPGM->StatR3DirtyPageSkipped, STAMTYPE_COUNTER, "/PGM/R3/DirtyPage/Skipped", STAMUNIT_OCCURENCES, "The number of pages already dirty or readonly.");
1776 STAM_REG(pVM, &pPGM->StatR3DirtyPageTrap, STAMTYPE_COUNTER, "/PGM/R3/DirtyPage/Trap", STAMUNIT_OCCURENCES, "The number of traps generated for dirty bit tracking.");
1777 STAM_REG(pVM, &pPGM->StatR3DirtiedPage, STAMTYPE_COUNTER, "/PGM/R3/DirtyPage/SetDirty", STAMUNIT_OCCURENCES, "The number of pages marked dirty because of write accesses.");
1778 STAM_REG(pVM, &pPGM->StatR3DirtyTrackRealPF, STAMTYPE_COUNTER, "/PGM/R3/DirtyPage/RealPF", STAMUNIT_OCCURENCES, "The number of real pages faults during dirty bit tracking.");
1779 STAM_REG(pVM, &pPGM->StatR3PageAlreadyDirty, STAMTYPE_COUNTER, "/PGM/R3/DirtyPage/AlreadySet", STAMUNIT_OCCURENCES, "The number of pages already marked dirty because of write accesses.");
1780 STAM_REG(pVM, &pPGM->StatR3InvalidatePage, STAMTYPE_PROFILE, "/PGM/R3/InvalidatePage", STAMUNIT_TICKS_PER_CALL, "PGMInvalidatePage() profiling.");
1781 STAM_REG(pVM, &pPGM->StatR3InvalidatePage4KBPages, STAMTYPE_COUNTER, "/PGM/R3/InvalidatePage/4KBPages", STAMUNIT_OCCURENCES, "The number of times PGMInvalidatePage() was called for a 4KB page.");
1782 STAM_REG(pVM, &pPGM->StatR3InvalidatePage4MBPages, STAMTYPE_COUNTER, "/PGM/R3/InvalidatePage/4MBPages", STAMUNIT_OCCURENCES, "The number of times PGMInvalidatePage() was called for a 4MB page.");
1783 STAM_REG(pVM, &pPGM->StatR3InvalidatePage4MBPagesSkip, STAMTYPE_COUNTER, "/PGM/R3/InvalidatePage/4MBPagesSkip",STAMUNIT_OCCURENCES, "The number of times PGMInvalidatePage() skipped a 4MB page.");
1784 STAM_REG(pVM, &pPGM->StatR3InvalidatePagePDMappings, STAMTYPE_COUNTER, "/PGM/R3/InvalidatePage/PDMappings", STAMUNIT_OCCURENCES, "The number of times PGMInvalidatePage() was called for a page directory containing mappings (no conflict).");
1785 STAM_REG(pVM, &pPGM->StatR3InvalidatePagePDNAs, STAMTYPE_COUNTER, "/PGM/R3/InvalidatePage/PDNAs", STAMUNIT_OCCURENCES, "The number of times PGMInvalidatePage() was called for a not accessed page directory.");
1786 STAM_REG(pVM, &pPGM->StatR3InvalidatePagePDNPs, STAMTYPE_COUNTER, "/PGM/R3/InvalidatePage/PDNPs", STAMUNIT_OCCURENCES, "The number of times PGMInvalidatePage() was called for a not present page directory.");
1787 STAM_REG(pVM, &pPGM->StatR3InvalidatePagePDOutOfSync, STAMTYPE_COUNTER, "/PGM/R3/InvalidatePage/PDOutOfSync", STAMUNIT_OCCURENCES, "The number of times PGMInvalidatePage() was called for an out of sync page directory.");
1788 STAM_REG(pVM, &pPGM->StatR3InvalidatePageSkipped, STAMTYPE_COUNTER, "/PGM/R3/InvalidatePage/Skipped", STAMUNIT_OCCURENCES, "The number of times PGMInvalidatePage() was skipped due to not present shw or pending pending SyncCR3.");
1789 STAM_REG(pVM, &pPGM->StatR3VirtHandlerSearchByPhys, STAMTYPE_PROFILE, "/PGM/R3/VirtHandlerSearchByPhys", STAMUNIT_TICKS_PER_CALL, "Profiling of pgmHandlerVirtualFindByPhysAddr.");
1790 STAM_REG(pVM, &pPGM->StatR3PhysHandlerReset, STAMTYPE_COUNTER, "/PGM/R3/PhysHandlerReset", STAMUNIT_OCCURENCES, "The number of times PGMHandlerPhysicalReset is called.");
1791 STAM_REG(pVM, &pPGM->StatR3PageOutOfSyncSupervisor, STAMTYPE_COUNTER, "/PGM/R3/OutOfSync/SuperVisor", STAMUNIT_OCCURENCES, "Number of traps due to pages out of sync and times VerifyAccessSyncPage calls SyncPage.");
1792 STAM_REG(pVM, &pPGM->StatR3PageOutOfSyncUser, STAMTYPE_COUNTER, "/PGM/R3/OutOfSync/User", STAMUNIT_OCCURENCES, "Number of traps due to pages out of sync and times VerifyAccessSyncPage calls SyncPage.");
1793 STAM_REG(pVM, &pPGM->StatR3Prefetch, STAMTYPE_PROFILE, "/PGM/R3/Prefetch", STAMUNIT_TICKS_PER_CALL, "PGMPrefetchPage profiling.");
1794 STAM_REG(pVM, &pPGM->StatR3ChunkR3MapTlbHits, STAMTYPE_COUNTER, "/PGM/ChunkR3Map/TlbHitsR3", STAMUNIT_OCCURENCES, "TLB hits.");
1795 STAM_REG(pVM, &pPGM->StatR3ChunkR3MapTlbMisses, STAMTYPE_COUNTER, "/PGM/ChunkR3Map/TlbMissesR3", STAMUNIT_OCCURENCES, "TLB misses.");
1796 STAM_REG(pVM, &pPGM->StatR3PageMapTlbHits, STAMTYPE_COUNTER, "/PGM/R3/Page/MapTlbHits", STAMUNIT_OCCURENCES, "TLB hits.");
1797 STAM_REG(pVM, &pPGM->StatR3PageMapTlbMisses, STAMTYPE_COUNTER, "/PGM/R3/Page/MapTlbMisses", STAMUNIT_OCCURENCES, "TLB misses.");
1798 STAM_REG(pVM, &pPGM->StatR3PageReplaceShared, STAMTYPE_COUNTER, "/PGM/R3/Page/ReplacedShared", STAMUNIT_OCCURENCES, "Times a shared page was replaced.");
1799 STAM_REG(pVM, &pPGM->StatR3PageReplaceZero, STAMTYPE_COUNTER, "/PGM/R3/Page/ReplacedZero", STAMUNIT_OCCURENCES, "Times the zero page was replaced.");
1800/// @todo STAM_REG(pVM, &pPGM->StatR3PageHandyAllocs, STAMTYPE_COUNTER, "/PGM/R3/Page/HandyAllocs", STAMUNIT_OCCURENCES, "Number of times we've allocated more handy pages.");
1801 STAM_REG(pVM, &pPGM->StatR3FlushTLB, STAMTYPE_PROFILE, "/PGM/R3/FlushTLB", STAMUNIT_OCCURENCES, "Profiling of the PGMFlushTLB() body.");
1802 STAM_REG(pVM, &pPGM->StatR3FlushTLBNewCR3, STAMTYPE_COUNTER, "/PGM/R3/FlushTLB/NewCR3", STAMUNIT_OCCURENCES, "The number of times PGMFlushTLB was called with a new CR3, non-global. (switch)");
1803 STAM_REG(pVM, &pPGM->StatR3FlushTLBNewCR3Global, STAMTYPE_COUNTER, "/PGM/R3/FlushTLB/NewCR3Global", STAMUNIT_OCCURENCES, "The number of times PGMFlushTLB was called with a new CR3, global. (switch)");
1804 STAM_REG(pVM, &pPGM->StatR3FlushTLBSameCR3, STAMTYPE_COUNTER, "/PGM/R3/FlushTLB/SameCR3", STAMUNIT_OCCURENCES, "The number of times PGMFlushTLB was called with the same CR3, non-global. (flush)");
1805 STAM_REG(pVM, &pPGM->StatR3FlushTLBSameCR3Global, STAMTYPE_COUNTER, "/PGM/R3/FlushTLB/SameCR3Global", STAMUNIT_OCCURENCES, "The number of times PGMFlushTLB was called with the same CR3, global. (flush)");
1806 STAM_REG(pVM, &pPGM->StatR3GstModifyPage, STAMTYPE_PROFILE, "/PGM/R3/GstModifyPage", STAMUNIT_TICKS_PER_CALL, "Profiling of the PGMGstModifyPage() body.");
1807
1808}
1809#endif /* VBOX_WITH_STATISTICS */
1810
1811
1812/**
1813 * Init the PGM bits that rely on VMMR0 and MM to be fully initialized.
1814 *
1815 * The dynamic mapping area will also be allocated and initialized at this
1816 * time. We could allocate it during PGMR3Init of course, but the mapping
1817 * wouldn't be allocated at that time preventing us from setting up the
1818 * page table entries with the dummy page.
1819 *
1820 * @returns VBox status code.
1821 * @param pVM VM handle.
1822 */
1823VMMR3DECL(int) PGMR3InitDynMap(PVM pVM)
1824{
1825 RTGCPTR GCPtr;
1826 int rc;
1827
1828#ifndef VBOX_WITH_PGMPOOL_PAGING_ONLY
1829 /*
1830 * Reserve space for mapping the paging pages into guest context.
1831 */
1832 rc = MMR3HyperReserve(pVM, PAGE_SIZE * (2 + RT_ELEMENTS(pVM->pgm.s.apShwPaePDsR3) + 1 + 2 + 2), "Paging", &GCPtr);
1833 AssertRCReturn(rc, rc);
1834 pVM->pgm.s.pShw32BitPdRC = GCPtr;
1835 MMR3HyperReserve(pVM, PAGE_SIZE, "fence", NULL);
1836#endif
1837
1838 /*
1839 * Reserve space for the dynamic mappings.
1840 */
1841 rc = MMR3HyperReserve(pVM, MM_HYPER_DYNAMIC_SIZE, "Dynamic mapping", &GCPtr);
1842 if (RT_SUCCESS(rc))
1843 pVM->pgm.s.pbDynPageMapBaseGC = GCPtr;
1844
1845 if ( RT_SUCCESS(rc)
1846 && (pVM->pgm.s.pbDynPageMapBaseGC >> X86_PD_PAE_SHIFT) != ((pVM->pgm.s.pbDynPageMapBaseGC + MM_HYPER_DYNAMIC_SIZE - 1) >> X86_PD_PAE_SHIFT))
1847 {
1848 rc = MMR3HyperReserve(pVM, MM_HYPER_DYNAMIC_SIZE, "Dynamic mapping not crossing", &GCPtr);
1849 if (RT_SUCCESS(rc))
1850 pVM->pgm.s.pbDynPageMapBaseGC = GCPtr;
1851 }
1852 if (RT_SUCCESS(rc))
1853 {
1854 AssertRelease((pVM->pgm.s.pbDynPageMapBaseGC >> X86_PD_PAE_SHIFT) == ((pVM->pgm.s.pbDynPageMapBaseGC + MM_HYPER_DYNAMIC_SIZE - 1) >> X86_PD_PAE_SHIFT));
1855 MMR3HyperReserve(pVM, PAGE_SIZE, "fence", NULL);
1856 }
1857 return rc;
1858}
1859
1860
1861/**
1862 * Ring-3 init finalizing.
1863 *
1864 * @returns VBox status code.
1865 * @param pVM The VM handle.
1866 */
1867VMMR3DECL(int) PGMR3InitFinalize(PVM pVM)
1868{
1869 int rc;
1870
1871#ifndef VBOX_WITH_PGMPOOL_PAGING_ONLY
1872 /*
1873 * Map the paging pages into the guest context.
1874 */
1875 RTGCPTR GCPtr = pVM->pgm.s.pShw32BitPdRC;
1876 AssertReleaseReturn(GCPtr, VERR_INTERNAL_ERROR);
1877
1878 rc = PGMMap(pVM, GCPtr, pVM->pgm.s.HCPhysShw32BitPD, PAGE_SIZE, 0);
1879 AssertRCReturn(rc, rc);
1880 pVM->pgm.s.pShw32BitPdRC = GCPtr;
1881 GCPtr += PAGE_SIZE;
1882 GCPtr += PAGE_SIZE; /* reserved page */
1883
1884 for (unsigned i = 0; i < RT_ELEMENTS(pVM->pgm.s.apShwPaePDsR3); i++)
1885 {
1886 rc = PGMMap(pVM, GCPtr, pVM->pgm.s.aHCPhysPaePDs[i], PAGE_SIZE, 0);
1887 AssertRCReturn(rc, rc);
1888 pVM->pgm.s.apShwPaePDsRC[i] = GCPtr;
1889 GCPtr += PAGE_SIZE;
1890 }
1891 /* A bit of paranoia is justified. */
1892 AssertRelease(pVM->pgm.s.apShwPaePDsRC[0] + PAGE_SIZE == pVM->pgm.s.apShwPaePDsRC[1]);
1893 AssertRelease(pVM->pgm.s.apShwPaePDsRC[1] + PAGE_SIZE == pVM->pgm.s.apShwPaePDsRC[2]);
1894 AssertRelease(pVM->pgm.s.apShwPaePDsRC[2] + PAGE_SIZE == pVM->pgm.s.apShwPaePDsRC[3]);
1895 GCPtr += PAGE_SIZE; /* reserved page */
1896
1897 rc = PGMMap(pVM, GCPtr, pVM->pgm.s.HCPhysShwPaePdpt, PAGE_SIZE, 0);
1898 AssertRCReturn(rc, rc);
1899 pVM->pgm.s.pShwPaePdptRC = GCPtr;
1900 GCPtr += PAGE_SIZE;
1901 GCPtr += PAGE_SIZE; /* reserved page */
1902#endif
1903
1904 /*
1905 * Reserve space for the dynamic mappings.
1906 * Initialize the dynamic mapping pages with dummy pages to simply the cache.
1907 */
1908 /* get the pointer to the page table entries. */
1909 PPGMMAPPING pMapping = pgmGetMapping(pVM, pVM->pgm.s.pbDynPageMapBaseGC);
1910 AssertRelease(pMapping);
1911 const uintptr_t off = pVM->pgm.s.pbDynPageMapBaseGC - pMapping->GCPtr;
1912 const unsigned iPT = off >> X86_PD_SHIFT;
1913 const unsigned iPG = (off >> X86_PT_SHIFT) & X86_PT_MASK;
1914 pVM->pgm.s.paDynPageMap32BitPTEsGC = pMapping->aPTs[iPT].pPTRC + iPG * sizeof(pMapping->aPTs[0].pPTR3->a[0]);
1915 pVM->pgm.s.paDynPageMapPaePTEsGC = pMapping->aPTs[iPT].paPaePTsRC + iPG * sizeof(pMapping->aPTs[0].paPaePTsR3->a[0]);
1916
1917 /* init cache */
1918 RTHCPHYS HCPhysDummy = MMR3PageDummyHCPhys(pVM);
1919 for (unsigned i = 0; i < RT_ELEMENTS(pVM->pgm.s.aHCPhysDynPageMapCache); i++)
1920 pVM->pgm.s.aHCPhysDynPageMapCache[i] = HCPhysDummy;
1921
1922 for (unsigned i = 0; i < MM_HYPER_DYNAMIC_SIZE; i += PAGE_SIZE)
1923 {
1924 rc = PGMMap(pVM, pVM->pgm.s.pbDynPageMapBaseGC + i, HCPhysDummy, PAGE_SIZE, 0);
1925 AssertRCReturn(rc, rc);
1926 }
1927
1928 /*
1929 * Note that AMD uses all the 8 reserved bits for the address (so 40 bits in total);
1930 * Intel only goes up to 36 bits, so we stick to 36 as well.
1931 */
1932 /** @todo How to test for the 40 bits support? Long mode seems to be the test criterium. */
1933 uint32_t u32Dummy, u32Features;
1934 CPUMGetGuestCpuId(pVM, 1, &u32Dummy, &u32Dummy, &u32Dummy, &u32Features);
1935
1936 if (u32Features & X86_CPUID_FEATURE_EDX_PSE36)
1937 pVM->pgm.s.GCPhys4MBPSEMask = RT_BIT_64(36) - 1;
1938 else
1939 pVM->pgm.s.GCPhys4MBPSEMask = RT_BIT_64(32) - 1;
1940
1941 LogRel(("PGMR3InitFinalize: 4 MB PSE mask %RGp\n", pVM->pgm.s.GCPhys4MBPSEMask));
1942
1943 return rc;
1944}
1945
1946
1947/**
1948 * Applies relocations to data and code managed by this component.
1949 *
1950 * This function will be called at init and whenever the VMM need to relocate it
1951 * self inside the GC.
1952 *
1953 * @param pVM The VM.
1954 * @param offDelta Relocation delta relative to old location.
1955 */
1956VMMR3DECL(void) PGMR3Relocate(PVM pVM, RTGCINTPTR offDelta)
1957{
1958 LogFlow(("PGMR3Relocate\n"));
1959
1960 /*
1961 * Paging stuff.
1962 */
1963 pVM->pgm.s.GCPtrCR3Mapping += offDelta;
1964 /** @todo move this into shadow and guest specific relocation functions. */
1965#ifdef VBOX_WITH_PGMPOOL_PAGING_ONLY
1966 AssertMsg(pVM->pgm.s.pShwNestedRootR3, ("Init order, no relocation before paging is initialized!\n"));
1967#else
1968 AssertMsg(pVM->pgm.s.pShw32BitPdR3, ("Init order, no relocation before paging is initialized!\n"));
1969 pVM->pgm.s.pShw32BitPdRC += offDelta;
1970#endif
1971 pVM->pgm.s.pGst32BitPdRC += offDelta;
1972 for (unsigned i = 0; i < RT_ELEMENTS(pVM->pgm.s.apGstPaePDsRC); i++)
1973 {
1974#ifndef VBOX_WITH_PGMPOOL_PAGING_ONLY
1975 AssertCompile(RT_ELEMENTS(pVM->pgm.s.apShwPaePDsRC) == RT_ELEMENTS(pVM->pgm.s.apGstPaePDsRC));
1976 pVM->pgm.s.apShwPaePDsRC[i] += offDelta;
1977#endif
1978 pVM->pgm.s.apGstPaePDsRC[i] += offDelta;
1979 }
1980 pVM->pgm.s.pGstPaePdptRC += offDelta;
1981#ifndef VBOX_WITH_PGMPOOL_PAGING_ONLY
1982 pVM->pgm.s.pShwPaePdptRC += offDelta;
1983#endif
1984
1985 pgmR3ModeDataInit(pVM, true /* resolve GC/R0 symbols */);
1986 pgmR3ModeDataSwitch(pVM, pVM->pgm.s.enmShadowMode, pVM->pgm.s.enmGuestMode);
1987
1988 PGM_SHW_PFN(Relocate, pVM)(pVM, offDelta);
1989 PGM_GST_PFN(Relocate, pVM)(pVM, offDelta);
1990 PGM_BTH_PFN(Relocate, pVM)(pVM, offDelta);
1991
1992 /*
1993 * Trees.
1994 */
1995 pVM->pgm.s.pTreesRC = MMHyperR3ToRC(pVM, pVM->pgm.s.pTreesR3);
1996
1997 /*
1998 * Ram ranges.
1999 */
2000 if (pVM->pgm.s.pRamRangesR3)
2001 {
2002 pVM->pgm.s.pRamRangesRC = MMHyperR3ToRC(pVM, pVM->pgm.s.pRamRangesR3);
2003 for (PPGMRAMRANGE pCur = pVM->pgm.s.pRamRangesR3; pCur->pNextR3; pCur = pCur->pNextR3)
2004 pCur->pNextRC = MMHyperR3ToRC(pVM, pCur->pNextR3);
2005 }
2006
2007 /*
2008 * Update the two page directories with all page table mappings.
2009 * (One or more of them have changed, that's why we're here.)
2010 */
2011 pVM->pgm.s.pMappingsRC = MMHyperR3ToRC(pVM, pVM->pgm.s.pMappingsR3);
2012 for (PPGMMAPPING pCur = pVM->pgm.s.pMappingsR3; pCur->pNextR3; pCur = pCur->pNextR3)
2013 pCur->pNextRC = MMHyperR3ToRC(pVM, pCur->pNextR3);
2014
2015 /* Relocate GC addresses of Page Tables. */
2016 for (PPGMMAPPING pCur = pVM->pgm.s.pMappingsR3; pCur; pCur = pCur->pNextR3)
2017 {
2018 for (RTHCUINT i = 0; i < pCur->cPTs; i++)
2019 {
2020 pCur->aPTs[i].pPTRC = MMHyperR3ToRC(pVM, pCur->aPTs[i].pPTR3);
2021 pCur->aPTs[i].paPaePTsRC = MMHyperR3ToRC(pVM, pCur->aPTs[i].paPaePTsR3);
2022 }
2023 }
2024
2025 /*
2026 * Dynamic page mapping area.
2027 */
2028 pVM->pgm.s.paDynPageMap32BitPTEsGC += offDelta;
2029 pVM->pgm.s.paDynPageMapPaePTEsGC += offDelta;
2030 pVM->pgm.s.pbDynPageMapBaseGC += offDelta;
2031
2032 /*
2033 * The Zero page.
2034 */
2035 pVM->pgm.s.pvZeroPgR0 = MMHyperR3ToR0(pVM, pVM->pgm.s.pvZeroPgR3);
2036#ifdef VBOX_WITH_2X_4GB_ADDR_SPACE
2037 AssertRelease(pVM->pgm.s.pvZeroPgR0 != NIL_RTR0PTR || !VMMIsHwVirtExtForced(pVM));
2038#else
2039 AssertRelease(pVM->pgm.s.pvZeroPgR0 != NIL_RTR0PTR);
2040#endif
2041
2042 /*
2043 * Physical and virtual handlers.
2044 */
2045 RTAvlroGCPhysDoWithAll(&pVM->pgm.s.pTreesR3->PhysHandlers, true, pgmR3RelocatePhysHandler, &offDelta);
2046 RTAvlroGCPtrDoWithAll(&pVM->pgm.s.pTreesR3->VirtHandlers, true, pgmR3RelocateVirtHandler, &offDelta);
2047 RTAvlroGCPtrDoWithAll(&pVM->pgm.s.pTreesR3->HyperVirtHandlers, true, pgmR3RelocateHyperVirtHandler, &offDelta);
2048
2049 /*
2050 * The page pool.
2051 */
2052 pgmR3PoolRelocate(pVM);
2053}
2054
2055
2056/**
2057 * Callback function for relocating a physical access handler.
2058 *
2059 * @returns 0 (continue enum)
2060 * @param pNode Pointer to a PGMPHYSHANDLER node.
2061 * @param pvUser Pointer to the offDelta. This is a pointer to the delta since we're
2062 * not certain the delta will fit in a void pointer for all possible configs.
2063 */
2064static DECLCALLBACK(int) pgmR3RelocatePhysHandler(PAVLROGCPHYSNODECORE pNode, void *pvUser)
2065{
2066 PPGMPHYSHANDLER pHandler = (PPGMPHYSHANDLER)pNode;
2067 RTGCINTPTR offDelta = *(PRTGCINTPTR)pvUser;
2068 if (pHandler->pfnHandlerRC)
2069 pHandler->pfnHandlerRC += offDelta;
2070 if (pHandler->pvUserRC >= 0x10000)
2071 pHandler->pvUserRC += offDelta;
2072 return 0;
2073}
2074
2075
2076/**
2077 * Callback function for relocating a virtual access handler.
2078 *
2079 * @returns 0 (continue enum)
2080 * @param pNode Pointer to a PGMVIRTHANDLER node.
2081 * @param pvUser Pointer to the offDelta. This is a pointer to the delta since we're
2082 * not certain the delta will fit in a void pointer for all possible configs.
2083 */
2084static DECLCALLBACK(int) pgmR3RelocateVirtHandler(PAVLROGCPTRNODECORE pNode, void *pvUser)
2085{
2086 PPGMVIRTHANDLER pHandler = (PPGMVIRTHANDLER)pNode;
2087 RTGCINTPTR offDelta = *(PRTGCINTPTR)pvUser;
2088 Assert( pHandler->enmType == PGMVIRTHANDLERTYPE_ALL
2089 || pHandler->enmType == PGMVIRTHANDLERTYPE_WRITE);
2090 Assert(pHandler->pfnHandlerRC);
2091 pHandler->pfnHandlerRC += offDelta;
2092 return 0;
2093}
2094
2095
2096/**
2097 * Callback function for relocating a virtual access handler for the hypervisor mapping.
2098 *
2099 * @returns 0 (continue enum)
2100 * @param pNode Pointer to a PGMVIRTHANDLER node.
2101 * @param pvUser Pointer to the offDelta. This is a pointer to the delta since we're
2102 * not certain the delta will fit in a void pointer for all possible configs.
2103 */
2104static DECLCALLBACK(int) pgmR3RelocateHyperVirtHandler(PAVLROGCPTRNODECORE pNode, void *pvUser)
2105{
2106 PPGMVIRTHANDLER pHandler = (PPGMVIRTHANDLER)pNode;
2107 RTGCINTPTR offDelta = *(PRTGCINTPTR)pvUser;
2108 Assert(pHandler->enmType == PGMVIRTHANDLERTYPE_HYPERVISOR);
2109 Assert(pHandler->pfnHandlerRC);
2110 pHandler->pfnHandlerRC += offDelta;
2111 return 0;
2112}
2113
2114
2115/**
2116 * The VM is being reset.
2117 *
2118 * For the PGM component this means that any PD write monitors
2119 * needs to be removed.
2120 *
2121 * @param pVM VM handle.
2122 */
2123VMMR3DECL(void) PGMR3Reset(PVM pVM)
2124{
2125 LogFlow(("PGMR3Reset:\n"));
2126 VM_ASSERT_EMT(pVM);
2127
2128 pgmLock(pVM);
2129
2130 /*
2131 * Unfix any fixed mappings and disable CR3 monitoring.
2132 */
2133 pVM->pgm.s.fMappingsFixed = false;
2134 pVM->pgm.s.GCPtrMappingFixed = 0;
2135 pVM->pgm.s.cbMappingFixed = 0;
2136
2137 /* Exit the guest paging mode before the pgm pool gets reset.
2138 * Important to clean up the amd64 case.
2139 */
2140 int rc = PGM_GST_PFN(Exit, pVM)(pVM);
2141 AssertRC(rc);
2142#ifdef DEBUG
2143 DBGFR3InfoLog(pVM, "mappings", NULL);
2144 DBGFR3InfoLog(pVM, "handlers", "all nostat");
2145#endif
2146
2147 /*
2148 * Reset the shadow page pool.
2149 */
2150 pgmR3PoolReset(pVM);
2151
2152 /*
2153 * Re-init other members.
2154 */
2155 pVM->pgm.s.fA20Enabled = true;
2156
2157 /*
2158 * Clear the FFs PGM owns.
2159 */
2160 VM_FF_CLEAR(pVM, VM_FF_PGM_SYNC_CR3);
2161 VM_FF_CLEAR(pVM, VM_FF_PGM_SYNC_CR3_NON_GLOBAL);
2162
2163 /*
2164 * Reset (zero) RAM pages.
2165 */
2166 rc = pgmR3PhysRamReset(pVM);
2167 if (RT_SUCCESS(rc))
2168 {
2169#ifdef VBOX_WITH_NEW_PHYS_CODE
2170 /*
2171 * Reset (zero) shadow ROM pages.
2172 */
2173 rc = pgmR3PhysRomReset(pVM);
2174#endif
2175 if (RT_SUCCESS(rc))
2176 {
2177 /*
2178 * Switch mode back to real mode.
2179 */
2180 rc = PGMR3ChangeMode(pVM, PGMMODE_REAL);
2181 STAM_REL_COUNTER_RESET(&pVM->pgm.s.cGuestModeChanges);
2182 }
2183 }
2184
2185 pgmUnlock(pVM);
2186 //return rc;
2187 AssertReleaseRC(rc);
2188}
2189
2190
2191#ifdef VBOX_STRICT
2192/**
2193 * VM state change callback for clearing fNoMorePhysWrites after
2194 * a snapshot has been created.
2195 */
2196static DECLCALLBACK(void) pgmR3ResetNoMorePhysWritesFlag(PVM pVM, VMSTATE enmState, VMSTATE enmOldState, void *pvUser)
2197{
2198 if (enmState == VMSTATE_RUNNING)
2199 pVM->pgm.s.fNoMorePhysWrites = false;
2200}
2201#endif
2202
2203
2204/**
2205 * Terminates the PGM.
2206 *
2207 * @returns VBox status code.
2208 * @param pVM Pointer to VM structure.
2209 */
2210VMMR3DECL(int) PGMR3Term(PVM pVM)
2211{
2212 return PDMR3CritSectDelete(&pVM->pgm.s.CritSect);
2213}
2214
2215
2216/**
2217 * Terminates the per-VCPU PGM.
2218 *
2219 * Termination means cleaning up and freeing all resources,
2220 * the VM it self is at this point powered off or suspended.
2221 *
2222 * @returns VBox status code.
2223 * @param pVM The VM to operate on.
2224 */
2225VMMR3DECL(int) PGMR3TermCPU(PVM pVM)
2226{
2227 return 0;
2228}
2229
2230
2231/**
2232 * Execute state save operation.
2233 *
2234 * @returns VBox status code.
2235 * @param pVM VM Handle.
2236 * @param pSSM SSM operation handle.
2237 */
2238static DECLCALLBACK(int) pgmR3Save(PVM pVM, PSSMHANDLE pSSM)
2239{
2240 PPGM pPGM = &pVM->pgm.s;
2241
2242 /* No more writes to physical memory after this point! */
2243 pVM->pgm.s.fNoMorePhysWrites = true;
2244
2245 /*
2246 * Save basic data (required / unaffected by relocation).
2247 */
2248#if 1
2249 SSMR3PutBool(pSSM, pPGM->fMappingsFixed);
2250#else
2251 SSMR3PutUInt(pSSM, pPGM->fMappingsFixed);
2252#endif
2253 SSMR3PutGCPtr(pSSM, pPGM->GCPtrMappingFixed);
2254 SSMR3PutU32(pSSM, pPGM->cbMappingFixed);
2255 SSMR3PutUInt(pSSM, pPGM->cbRamSize);
2256 SSMR3PutGCPhys(pSSM, pPGM->GCPhysA20Mask);
2257 SSMR3PutUInt(pSSM, pPGM->fA20Enabled);
2258 SSMR3PutUInt(pSSM, pPGM->fSyncFlags);
2259 SSMR3PutUInt(pSSM, pPGM->enmGuestMode);
2260 SSMR3PutU32(pSSM, ~0); /* Separator. */
2261
2262 /*
2263 * The guest mappings.
2264 */
2265 uint32_t i = 0;
2266 for (PPGMMAPPING pMapping = pPGM->pMappingsR3; pMapping; pMapping = pMapping->pNextR3, i++)
2267 {
2268 SSMR3PutU32(pSSM, i);
2269 SSMR3PutStrZ(pSSM, pMapping->pszDesc); /* This is the best unique id we have... */
2270 SSMR3PutGCPtr(pSSM, pMapping->GCPtr);
2271 SSMR3PutGCUIntPtr(pSSM, pMapping->cPTs);
2272 /* flags are done by the mapping owners! */
2273 }
2274 SSMR3PutU32(pSSM, ~0); /* terminator. */
2275
2276 /*
2277 * Ram range flags and bits.
2278 */
2279 i = 0;
2280 for (PPGMRAMRANGE pRam = pPGM->pRamRangesR3; pRam; pRam = pRam->pNextR3, i++)
2281 {
2282 /** @todo MMIO ranges may move (PCI reconfig), we currently assume they don't. */
2283
2284 SSMR3PutU32(pSSM, i);
2285 SSMR3PutGCPhys(pSSM, pRam->GCPhys);
2286 SSMR3PutGCPhys(pSSM, pRam->GCPhysLast);
2287 SSMR3PutGCPhys(pSSM, pRam->cb);
2288 SSMR3PutU8(pSSM, !!pRam->pvR3); /* boolean indicating memory or not. */
2289
2290 /* Flags. */
2291 const unsigned cPages = pRam->cb >> PAGE_SHIFT;
2292 for (unsigned iPage = 0; iPage < cPages; iPage++)
2293 SSMR3PutU16(pSSM, (uint16_t)(pRam->aPages[iPage].HCPhys & ~X86_PTE_PAE_PG_MASK)); /** @todo PAGE FLAGS */
2294
2295 /* any memory associated with the range. */
2296 if (pRam->fFlags & MM_RAM_FLAGS_DYNAMIC_ALLOC)
2297 {
2298 for (unsigned iChunk = 0; iChunk < (pRam->cb >> PGM_DYNAMIC_CHUNK_SHIFT); iChunk++)
2299 {
2300 if (pRam->paChunkR3Ptrs[iChunk])
2301 {
2302 SSMR3PutU8(pSSM, 1); /* chunk present */
2303 SSMR3PutMem(pSSM, (void *)pRam->paChunkR3Ptrs[iChunk], PGM_DYNAMIC_CHUNK_SIZE);
2304 }
2305 else
2306 SSMR3PutU8(pSSM, 0); /* no chunk present */
2307 }
2308 }
2309 else if (pRam->pvR3)
2310 {
2311 int rc = SSMR3PutMem(pSSM, pRam->pvR3, pRam->cb);
2312 if (RT_FAILURE(rc))
2313 {
2314 Log(("pgmR3Save: SSMR3PutMem(, %p, %#x) -> %Rrc\n", pRam->pvR3, pRam->cb, rc));
2315 return rc;
2316 }
2317 }
2318 }
2319 return SSMR3PutU32(pSSM, ~0); /* terminator. */
2320}
2321
2322
2323/**
2324 * Execute state load operation.
2325 *
2326 * @returns VBox status code.
2327 * @param pVM VM Handle.
2328 * @param pSSM SSM operation handle.
2329 * @param u32Version Data layout version.
2330 */
2331static DECLCALLBACK(int) pgmR3Load(PVM pVM, PSSMHANDLE pSSM, uint32_t u32Version)
2332{
2333 /*
2334 * Validate version.
2335 */
2336 if (u32Version != PGM_SAVED_STATE_VERSION)
2337 {
2338 AssertMsgFailed(("pgmR3Load: Invalid version u32Version=%d (current %d)!\n", u32Version, PGM_SAVED_STATE_VERSION));
2339 return VERR_SSM_UNSUPPORTED_DATA_UNIT_VERSION;
2340 }
2341
2342 /*
2343 * Call the reset function to make sure all the memory is cleared.
2344 */
2345 PGMR3Reset(pVM);
2346
2347 /*
2348 * Load basic data (required / unaffected by relocation).
2349 */
2350 PPGM pPGM = &pVM->pgm.s;
2351#if 1
2352 SSMR3GetBool(pSSM, &pPGM->fMappingsFixed);
2353#else
2354 uint32_t u;
2355 SSMR3GetU32(pSSM, &u);
2356 pPGM->fMappingsFixed = u;
2357#endif
2358 SSMR3GetGCPtr(pSSM, &pPGM->GCPtrMappingFixed);
2359 SSMR3GetU32(pSSM, &pPGM->cbMappingFixed);
2360
2361 RTUINT cbRamSize;
2362 int rc = SSMR3GetU32(pSSM, &cbRamSize);
2363 if (RT_FAILURE(rc))
2364 return rc;
2365 if (cbRamSize != pPGM->cbRamSize)
2366 return VERR_SSM_LOAD_MEMORY_SIZE_MISMATCH;
2367 SSMR3GetGCPhys(pSSM, &pPGM->GCPhysA20Mask);
2368 SSMR3GetUInt(pSSM, &pPGM->fA20Enabled);
2369 SSMR3GetUInt(pSSM, &pPGM->fSyncFlags);
2370 RTUINT uGuestMode;
2371 SSMR3GetUInt(pSSM, &uGuestMode);
2372 pPGM->enmGuestMode = (PGMMODE)uGuestMode;
2373
2374 /* check separator. */
2375 uint32_t u32Sep;
2376 SSMR3GetU32(pSSM, &u32Sep);
2377 if (RT_FAILURE(rc))
2378 return rc;
2379 if (u32Sep != (uint32_t)~0)
2380 {
2381 AssertMsgFailed(("u32Sep=%#x (first)\n", u32Sep));
2382 return VERR_SSM_DATA_UNIT_FORMAT_CHANGED;
2383 }
2384
2385 /*
2386 * The guest mappings.
2387 */
2388 uint32_t i = 0;
2389 for (;; i++)
2390 {
2391 /* Check the seqence number / separator. */
2392 rc = SSMR3GetU32(pSSM, &u32Sep);
2393 if (RT_FAILURE(rc))
2394 return rc;
2395 if (u32Sep == ~0U)
2396 break;
2397 if (u32Sep != i)
2398 {
2399 AssertMsgFailed(("u32Sep=%#x (last)\n", u32Sep));
2400 return VERR_SSM_DATA_UNIT_FORMAT_CHANGED;
2401 }
2402
2403 /* get the mapping details. */
2404 char szDesc[256];
2405 szDesc[0] = '\0';
2406 rc = SSMR3GetStrZ(pSSM, szDesc, sizeof(szDesc));
2407 if (RT_FAILURE(rc))
2408 return rc;
2409 RTGCPTR GCPtr;
2410 SSMR3GetGCPtr(pSSM, &GCPtr);
2411 RTGCPTR cPTs;
2412 rc = SSMR3GetGCUIntPtr(pSSM, &cPTs);
2413 if (RT_FAILURE(rc))
2414 return rc;
2415
2416 /* find matching range. */
2417 PPGMMAPPING pMapping;
2418 for (pMapping = pPGM->pMappingsR3; pMapping; pMapping = pMapping->pNextR3)
2419 if ( pMapping->cPTs == cPTs
2420 && !strcmp(pMapping->pszDesc, szDesc))
2421 break;
2422 if (!pMapping)
2423 {
2424 LogRel(("Couldn't find mapping: cPTs=%#x szDesc=%s (GCPtr=%RGv)\n",
2425 cPTs, szDesc, GCPtr));
2426 AssertFailed();
2427 return VERR_SSM_LOAD_CONFIG_MISMATCH;
2428 }
2429
2430 /* relocate it. */
2431 if (pMapping->GCPtr != GCPtr)
2432 {
2433 AssertMsg((GCPtr >> X86_PD_SHIFT << X86_PD_SHIFT) == GCPtr, ("GCPtr=%RGv\n", GCPtr));
2434 pgmR3MapRelocate(pVM, pMapping, pMapping->GCPtr, GCPtr);
2435 }
2436 else
2437 Log(("pgmR3Load: '%s' needed no relocation (%RGv)\n", szDesc, GCPtr));
2438 }
2439
2440 /*
2441 * Ram range flags and bits.
2442 */
2443 i = 0;
2444 for (PPGMRAMRANGE pRam = pPGM->pRamRangesR3; pRam; pRam = pRam->pNextR3, i++)
2445 {
2446 /** @todo MMIO ranges may move (PCI reconfig), we currently assume they don't. */
2447 /* Check the seqence number / separator. */
2448 rc = SSMR3GetU32(pSSM, &u32Sep);
2449 if (RT_FAILURE(rc))
2450 return rc;
2451 if (u32Sep == ~0U)
2452 break;
2453 if (u32Sep != i)
2454 {
2455 AssertMsgFailed(("u32Sep=%#x (last)\n", u32Sep));
2456 return VERR_SSM_DATA_UNIT_FORMAT_CHANGED;
2457 }
2458
2459 /* Get the range details. */
2460 RTGCPHYS GCPhys;
2461 SSMR3GetGCPhys(pSSM, &GCPhys);
2462 RTGCPHYS GCPhysLast;
2463 SSMR3GetGCPhys(pSSM, &GCPhysLast);
2464 RTGCPHYS cb;
2465 SSMR3GetGCPhys(pSSM, &cb);
2466 uint8_t fHaveBits;
2467 rc = SSMR3GetU8(pSSM, &fHaveBits);
2468 if (RT_FAILURE(rc))
2469 return rc;
2470 if (fHaveBits & ~1)
2471 {
2472 AssertMsgFailed(("u32Sep=%#x (last)\n", u32Sep));
2473 return VERR_SSM_DATA_UNIT_FORMAT_CHANGED;
2474 }
2475
2476 /* Match it up with the current range. */
2477 if ( GCPhys != pRam->GCPhys
2478 || GCPhysLast != pRam->GCPhysLast
2479 || cb != pRam->cb
2480 || fHaveBits != !!pRam->pvR3)
2481 {
2482 LogRel(("Ram range: %RGp-%RGp %RGp bytes %s\n"
2483 "State : %RGp-%RGp %RGp bytes %s\n",
2484 pRam->GCPhys, pRam->GCPhysLast, pRam->cb, pRam->pvR3 ? "bits" : "nobits",
2485 GCPhys, GCPhysLast, cb, fHaveBits ? "bits" : "nobits"));
2486 /*
2487 * If we're loading a state for debugging purpose, don't make a fuss if
2488 * the MMIO[2] and ROM stuff isn't 100% right, just skip the mismatches.
2489 */
2490 if ( SSMR3HandleGetAfter(pSSM) != SSMAFTER_DEBUG_IT
2491 || GCPhys < 8 * _1M)
2492 AssertFailedReturn(VERR_SSM_LOAD_CONFIG_MISMATCH);
2493
2494 RTGCPHYS cPages = ((GCPhysLast - GCPhys) + 1) >> PAGE_SHIFT;
2495 while (cPages-- > 0)
2496 {
2497 uint16_t u16Ignore;
2498 SSMR3GetU16(pSSM, &u16Ignore);
2499 }
2500 continue;
2501 }
2502
2503 /* Flags. */
2504 const unsigned cPages = pRam->cb >> PAGE_SHIFT;
2505 for (unsigned iPage = 0; iPage < cPages; iPage++)
2506 {
2507 uint16_t u16 = 0;
2508 SSMR3GetU16(pSSM, &u16);
2509 u16 &= PAGE_OFFSET_MASK & ~( RT_BIT(4) | RT_BIT(5) | RT_BIT(6)
2510 | RT_BIT(7) | RT_BIT(8) | RT_BIT(9) | RT_BIT(10) );
2511 // &= MM_RAM_FLAGS_DYNAMIC_ALLOC | MM_RAM_FLAGS_RESERVED | MM_RAM_FLAGS_ROM | MM_RAM_FLAGS_MMIO | MM_RAM_FLAGS_MMIO2
2512 pRam->aPages[iPage].HCPhys = PGM_PAGE_GET_HCPHYS(&pRam->aPages[iPage]) | (RTHCPHYS)u16; /** @todo PAGE FLAGS */
2513 }
2514
2515 /* any memory associated with the range. */
2516 if (pRam->fFlags & MM_RAM_FLAGS_DYNAMIC_ALLOC)
2517 {
2518 for (unsigned iChunk = 0; iChunk < (pRam->cb >> PGM_DYNAMIC_CHUNK_SHIFT); iChunk++)
2519 {
2520 uint8_t fValidChunk;
2521
2522 rc = SSMR3GetU8(pSSM, &fValidChunk);
2523 if (RT_FAILURE(rc))
2524 return rc;
2525 if (fValidChunk > 1)
2526 return VERR_SSM_DATA_UNIT_FORMAT_CHANGED;
2527
2528 if (fValidChunk)
2529 {
2530 if (!pRam->paChunkR3Ptrs[iChunk])
2531 {
2532 rc = pgmr3PhysGrowRange(pVM, pRam->GCPhys + iChunk * PGM_DYNAMIC_CHUNK_SIZE);
2533 if (RT_FAILURE(rc))
2534 return rc;
2535 }
2536 Assert(pRam->paChunkR3Ptrs[iChunk]);
2537
2538 SSMR3GetMem(pSSM, (void *)pRam->paChunkR3Ptrs[iChunk], PGM_DYNAMIC_CHUNK_SIZE);
2539 }
2540 /* else nothing to do */
2541 }
2542 }
2543 else if (pRam->pvR3)
2544 {
2545 int rc = SSMR3GetMem(pSSM, pRam->pvR3, pRam->cb);
2546 if (RT_FAILURE(rc))
2547 {
2548 Log(("pgmR3Save: SSMR3GetMem(, %p, %#x) -> %Rrc\n", pRam->pvR3, pRam->cb, rc));
2549 return rc;
2550 }
2551 }
2552 }
2553
2554 /*
2555 * We require a full resync now.
2556 */
2557 VM_FF_SET(pVM, VM_FF_PGM_SYNC_CR3_NON_GLOBAL);
2558 VM_FF_SET(pVM, VM_FF_PGM_SYNC_CR3);
2559 pPGM->fSyncFlags |= PGM_SYNC_UPDATE_PAGE_BIT_VIRTUAL;
2560 pPGM->fPhysCacheFlushPending = true;
2561 pgmR3HandlerPhysicalUpdateAll(pVM);
2562
2563 /*
2564 * Change the paging mode.
2565 */
2566 rc = PGMR3ChangeMode(pVM, pPGM->enmGuestMode);
2567
2568 /* Restore pVM->pgm.s.GCPhysCR3. */
2569 Assert(pVM->pgm.s.GCPhysCR3 == NIL_RTGCPHYS);
2570 RTGCPHYS GCPhysCR3 = CPUMGetGuestCR3(pVM);
2571 if ( pVM->pgm.s.enmGuestMode == PGMMODE_PAE
2572 || pVM->pgm.s.enmGuestMode == PGMMODE_PAE_NX
2573 || pVM->pgm.s.enmGuestMode == PGMMODE_AMD64
2574 || pVM->pgm.s.enmGuestMode == PGMMODE_AMD64_NX)
2575 GCPhysCR3 = (GCPhysCR3 & X86_CR3_PAE_PAGE_MASK);
2576 else
2577 GCPhysCR3 = (GCPhysCR3 & X86_CR3_PAGE_MASK);
2578 pVM->pgm.s.GCPhysCR3 = GCPhysCR3;
2579
2580 return rc;
2581}
2582
2583
2584/**
2585 * Show paging mode.
2586 *
2587 * @param pVM VM Handle.
2588 * @param pHlp The info helpers.
2589 * @param pszArgs "all" (default), "guest", "shadow" or "host".
2590 */
2591static DECLCALLBACK(void) pgmR3InfoMode(PVM pVM, PCDBGFINFOHLP pHlp, const char *pszArgs)
2592{
2593 /* digest argument. */
2594 bool fGuest, fShadow, fHost;
2595 if (pszArgs)
2596 pszArgs = RTStrStripL(pszArgs);
2597 if (!pszArgs || !*pszArgs || strstr(pszArgs, "all"))
2598 fShadow = fHost = fGuest = true;
2599 else
2600 {
2601 fShadow = fHost = fGuest = false;
2602 if (strstr(pszArgs, "guest"))
2603 fGuest = true;
2604 if (strstr(pszArgs, "shadow"))
2605 fShadow = true;
2606 if (strstr(pszArgs, "host"))
2607 fHost = true;
2608 }
2609
2610 /* print info. */
2611 if (fGuest)
2612 pHlp->pfnPrintf(pHlp, "Guest paging mode: %s, changed %RU64 times, A20 %s\n",
2613 PGMGetModeName(pVM->pgm.s.enmGuestMode), pVM->pgm.s.cGuestModeChanges.c,
2614 pVM->pgm.s.fA20Enabled ? "enabled" : "disabled");
2615 if (fShadow)
2616 pHlp->pfnPrintf(pHlp, "Shadow paging mode: %s\n", PGMGetModeName(pVM->pgm.s.enmShadowMode));
2617 if (fHost)
2618 {
2619 const char *psz;
2620 switch (pVM->pgm.s.enmHostMode)
2621 {
2622 case SUPPAGINGMODE_INVALID: psz = "invalid"; break;
2623 case SUPPAGINGMODE_32_BIT: psz = "32-bit"; break;
2624 case SUPPAGINGMODE_32_BIT_GLOBAL: psz = "32-bit+G"; break;
2625 case SUPPAGINGMODE_PAE: psz = "PAE"; break;
2626 case SUPPAGINGMODE_PAE_GLOBAL: psz = "PAE+G"; break;
2627 case SUPPAGINGMODE_PAE_NX: psz = "PAE+NX"; break;
2628 case SUPPAGINGMODE_PAE_GLOBAL_NX: psz = "PAE+G+NX"; break;
2629 case SUPPAGINGMODE_AMD64: psz = "AMD64"; break;
2630 case SUPPAGINGMODE_AMD64_GLOBAL: psz = "AMD64+G"; break;
2631 case SUPPAGINGMODE_AMD64_NX: psz = "AMD64+NX"; break;
2632 case SUPPAGINGMODE_AMD64_GLOBAL_NX: psz = "AMD64+G+NX"; break;
2633 default: psz = "unknown"; break;
2634 }
2635 pHlp->pfnPrintf(pHlp, "Host paging mode: %s\n", psz);
2636 }
2637}
2638
2639
2640/**
2641 * Dump registered MMIO ranges to the log.
2642 *
2643 * @param pVM VM Handle.
2644 * @param pHlp The info helpers.
2645 * @param pszArgs Arguments, ignored.
2646 */
2647static DECLCALLBACK(void) pgmR3PhysInfo(PVM pVM, PCDBGFINFOHLP pHlp, const char *pszArgs)
2648{
2649 NOREF(pszArgs);
2650 pHlp->pfnPrintf(pHlp,
2651 "RAM ranges (pVM=%p)\n"
2652 "%.*s %.*s\n",
2653 pVM,
2654 sizeof(RTGCPHYS) * 4 + 1, "GC Phys Range ",
2655 sizeof(RTHCPTR) * 2, "pvHC ");
2656
2657 for (PPGMRAMRANGE pCur = pVM->pgm.s.pRamRangesR3; pCur; pCur = pCur->pNextR3)
2658 pHlp->pfnPrintf(pHlp,
2659 "%RGp-%RGp %RHv %s\n",
2660 pCur->GCPhys,
2661 pCur->GCPhysLast,
2662 pCur->pvR3,
2663 pCur->pszDesc);
2664}
2665
2666/**
2667 * Dump the page directory to the log.
2668 *
2669 * @param pVM VM Handle.
2670 * @param pHlp The info helpers.
2671 * @param pszArgs Arguments, ignored.
2672 */
2673static DECLCALLBACK(void) pgmR3InfoCr3(PVM pVM, PCDBGFINFOHLP pHlp, const char *pszArgs)
2674{
2675/** @todo fix this! Convert the PGMR3DumpHierarchyHC functions to do guest stuff. */
2676 /* Big pages supported? */
2677 const bool fPSE = !!(CPUMGetGuestCR4(pVM) & X86_CR4_PSE);
2678
2679 /* Global pages supported? */
2680 const bool fPGE = !!(CPUMGetGuestCR4(pVM) & X86_CR4_PGE);
2681
2682 NOREF(pszArgs);
2683
2684 /*
2685 * Get page directory addresses.
2686 */
2687 PX86PD pPDSrc = pVM->pgm.s.pGst32BitPdR3;
2688 Assert(pPDSrc);
2689 Assert(PGMPhysGCPhys2R3PtrAssert(pVM, (RTGCPHYS)(CPUMGetGuestCR3(pVM) & X86_CR3_PAGE_MASK), sizeof(*pPDSrc)) == pPDSrc);
2690
2691 /*
2692 * Iterate the page directory.
2693 */
2694 for (unsigned iPD = 0; iPD < RT_ELEMENTS(pPDSrc->a); iPD++)
2695 {
2696 X86PDE PdeSrc = pPDSrc->a[iPD];
2697 if (PdeSrc.n.u1Present)
2698 {
2699 if (PdeSrc.b.u1Size && fPSE)
2700 pHlp->pfnPrintf(pHlp,
2701 "%04X - %RGp P=%d U=%d RW=%d G=%d - BIG\n",
2702 iPD,
2703 pgmGstGet4MBPhysPage(&pVM->pgm.s, PdeSrc),
2704 PdeSrc.b.u1Present, PdeSrc.b.u1User, PdeSrc.b.u1Write, PdeSrc.b.u1Global && fPGE);
2705 else
2706 pHlp->pfnPrintf(pHlp,
2707 "%04X - %RGp P=%d U=%d RW=%d [G=%d]\n",
2708 iPD,
2709 (RTGCPHYS)(PdeSrc.u & X86_PDE_PG_MASK),
2710 PdeSrc.n.u1Present, PdeSrc.n.u1User, PdeSrc.n.u1Write, PdeSrc.b.u1Global && fPGE);
2711 }
2712 }
2713}
2714
2715
2716/**
2717 * Serivce a VMMCALLHOST_PGM_LOCK call.
2718 *
2719 * @returns VBox status code.
2720 * @param pVM The VM handle.
2721 */
2722VMMR3DECL(int) PGMR3LockCall(PVM pVM)
2723{
2724 int rc = PDMR3CritSectEnterEx(&pVM->pgm.s.CritSect, true /* fHostCall */);
2725 AssertRC(rc);
2726 return rc;
2727}
2728
2729
2730/**
2731 * Converts a PGMMODE value to a PGM_TYPE_* \#define.
2732 *
2733 * @returns PGM_TYPE_*.
2734 * @param pgmMode The mode value to convert.
2735 */
2736DECLINLINE(unsigned) pgmModeToType(PGMMODE pgmMode)
2737{
2738 switch (pgmMode)
2739 {
2740 case PGMMODE_REAL: return PGM_TYPE_REAL;
2741 case PGMMODE_PROTECTED: return PGM_TYPE_PROT;
2742 case PGMMODE_32_BIT: return PGM_TYPE_32BIT;
2743 case PGMMODE_PAE:
2744 case PGMMODE_PAE_NX: return PGM_TYPE_PAE;
2745 case PGMMODE_AMD64:
2746 case PGMMODE_AMD64_NX: return PGM_TYPE_AMD64;
2747 case PGMMODE_NESTED: return PGM_TYPE_NESTED;
2748 case PGMMODE_EPT: return PGM_TYPE_EPT;
2749 default:
2750 AssertFatalMsgFailed(("pgmMode=%d\n", pgmMode));
2751 }
2752}
2753
2754
2755/**
2756 * Gets the index into the paging mode data array of a SHW+GST mode.
2757 *
2758 * @returns PGM::paPagingData index.
2759 * @param uShwType The shadow paging mode type.
2760 * @param uGstType The guest paging mode type.
2761 */
2762DECLINLINE(unsigned) pgmModeDataIndex(unsigned uShwType, unsigned uGstType)
2763{
2764 Assert(uShwType >= PGM_TYPE_32BIT && uShwType <= PGM_TYPE_MAX);
2765 Assert(uGstType >= PGM_TYPE_REAL && uGstType <= PGM_TYPE_AMD64);
2766 return (uShwType - PGM_TYPE_32BIT) * (PGM_TYPE_AMD64 - PGM_TYPE_REAL + 1)
2767 + (uGstType - PGM_TYPE_REAL);
2768}
2769
2770
2771/**
2772 * Gets the index into the paging mode data array of a SHW+GST mode.
2773 *
2774 * @returns PGM::paPagingData index.
2775 * @param enmShw The shadow paging mode.
2776 * @param enmGst The guest paging mode.
2777 */
2778DECLINLINE(unsigned) pgmModeDataIndexByMode(PGMMODE enmShw, PGMMODE enmGst)
2779{
2780 Assert(enmShw >= PGMMODE_32_BIT && enmShw <= PGMMODE_MAX);
2781 Assert(enmGst > PGMMODE_INVALID && enmGst < PGMMODE_MAX);
2782 return pgmModeDataIndex(pgmModeToType(enmShw), pgmModeToType(enmGst));
2783}
2784
2785
2786/**
2787 * Calculates the max data index.
2788 * @returns The number of entries in the paging data array.
2789 */
2790DECLINLINE(unsigned) pgmModeDataMaxIndex(void)
2791{
2792 return pgmModeDataIndex(PGM_TYPE_MAX, PGM_TYPE_AMD64) + 1;
2793}
2794
2795
2796/**
2797 * Initializes the paging mode data kept in PGM::paModeData.
2798 *
2799 * @param pVM The VM handle.
2800 * @param fResolveGCAndR0 Indicate whether or not GC and Ring-0 symbols can be resolved now.
2801 * This is used early in the init process to avoid trouble with PDM
2802 * not being initialized yet.
2803 */
2804static int pgmR3ModeDataInit(PVM pVM, bool fResolveGCAndR0)
2805{
2806 PPGMMODEDATA pModeData;
2807 int rc;
2808
2809 /*
2810 * Allocate the array on the first call.
2811 */
2812 if (!pVM->pgm.s.paModeData)
2813 {
2814 pVM->pgm.s.paModeData = (PPGMMODEDATA)MMR3HeapAllocZ(pVM, MM_TAG_PGM, sizeof(PGMMODEDATA) * pgmModeDataMaxIndex());
2815 AssertReturn(pVM->pgm.s.paModeData, VERR_NO_MEMORY);
2816 }
2817
2818 /*
2819 * Initialize the array entries.
2820 */
2821 pModeData = &pVM->pgm.s.paModeData[pgmModeDataIndex(PGM_TYPE_32BIT, PGM_TYPE_REAL)];
2822 pModeData->uShwType = PGM_TYPE_32BIT;
2823 pModeData->uGstType = PGM_TYPE_REAL;
2824 rc = PGM_SHW_NAME_32BIT(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2825 rc = PGM_GST_NAME_REAL(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2826 rc = PGM_BTH_NAME_32BIT_REAL(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2827
2828 pModeData = &pVM->pgm.s.paModeData[pgmModeDataIndex(PGM_TYPE_32BIT, PGMMODE_PROTECTED)];
2829 pModeData->uShwType = PGM_TYPE_32BIT;
2830 pModeData->uGstType = PGM_TYPE_PROT;
2831 rc = PGM_SHW_NAME_32BIT(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2832 rc = PGM_GST_NAME_PROT(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2833 rc = PGM_BTH_NAME_32BIT_PROT(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2834
2835 pModeData = &pVM->pgm.s.paModeData[pgmModeDataIndex(PGM_TYPE_32BIT, PGM_TYPE_32BIT)];
2836 pModeData->uShwType = PGM_TYPE_32BIT;
2837 pModeData->uGstType = PGM_TYPE_32BIT;
2838 rc = PGM_SHW_NAME_32BIT(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2839 rc = PGM_GST_NAME_32BIT(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2840 rc = PGM_BTH_NAME_32BIT_32BIT(InitData)(pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2841
2842 pModeData = &pVM->pgm.s.paModeData[pgmModeDataIndex(PGM_TYPE_PAE, PGM_TYPE_REAL)];
2843 pModeData->uShwType = PGM_TYPE_PAE;
2844 pModeData->uGstType = PGM_TYPE_REAL;
2845 rc = PGM_SHW_NAME_PAE(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2846 rc = PGM_GST_NAME_REAL(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2847 rc = PGM_BTH_NAME_PAE_REAL(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2848
2849 pModeData = &pVM->pgm.s.paModeData[pgmModeDataIndex(PGM_TYPE_PAE, PGM_TYPE_PROT)];
2850 pModeData->uShwType = PGM_TYPE_PAE;
2851 pModeData->uGstType = PGM_TYPE_PROT;
2852 rc = PGM_SHW_NAME_PAE(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2853 rc = PGM_GST_NAME_PROT(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2854 rc = PGM_BTH_NAME_PAE_PROT(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2855
2856 pModeData = &pVM->pgm.s.paModeData[pgmModeDataIndex(PGM_TYPE_PAE, PGM_TYPE_32BIT)];
2857 pModeData->uShwType = PGM_TYPE_PAE;
2858 pModeData->uGstType = PGM_TYPE_32BIT;
2859 rc = PGM_SHW_NAME_PAE(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2860 rc = PGM_GST_NAME_32BIT(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2861 rc = PGM_BTH_NAME_PAE_32BIT(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2862
2863 pModeData = &pVM->pgm.s.paModeData[pgmModeDataIndex(PGM_TYPE_PAE, PGM_TYPE_PAE)];
2864 pModeData->uShwType = PGM_TYPE_PAE;
2865 pModeData->uGstType = PGM_TYPE_PAE;
2866 rc = PGM_SHW_NAME_PAE(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2867 rc = PGM_GST_NAME_PAE(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2868 rc = PGM_BTH_NAME_PAE_PAE(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2869
2870#ifdef VBOX_WITH_64_BITS_GUESTS
2871 pModeData = &pVM->pgm.s.paModeData[pgmModeDataIndex(PGM_TYPE_AMD64, PGM_TYPE_AMD64)];
2872 pModeData->uShwType = PGM_TYPE_AMD64;
2873 pModeData->uGstType = PGM_TYPE_AMD64;
2874 rc = PGM_SHW_NAME_AMD64(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2875 rc = PGM_GST_NAME_AMD64(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2876 rc = PGM_BTH_NAME_AMD64_AMD64(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2877#endif
2878
2879 /* The nested paging mode. */
2880 pModeData = &pVM->pgm.s.paModeData[pgmModeDataIndex(PGM_TYPE_NESTED, PGM_TYPE_REAL)];
2881 pModeData->uShwType = PGM_TYPE_NESTED;
2882 pModeData->uGstType = PGM_TYPE_REAL;
2883 rc = PGM_GST_NAME_REAL(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2884 rc = PGM_BTH_NAME_NESTED_REAL(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2885
2886 pModeData = &pVM->pgm.s.paModeData[pgmModeDataIndex(PGM_TYPE_NESTED, PGMMODE_PROTECTED)];
2887 pModeData->uShwType = PGM_TYPE_NESTED;
2888 pModeData->uGstType = PGM_TYPE_PROT;
2889 rc = PGM_GST_NAME_PROT(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2890 rc = PGM_BTH_NAME_NESTED_PROT(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2891
2892 pModeData = &pVM->pgm.s.paModeData[pgmModeDataIndex(PGM_TYPE_NESTED, PGM_TYPE_32BIT)];
2893 pModeData->uShwType = PGM_TYPE_NESTED;
2894 pModeData->uGstType = PGM_TYPE_32BIT;
2895 rc = PGM_GST_NAME_32BIT(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2896 rc = PGM_BTH_NAME_NESTED_32BIT(InitData)(pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2897
2898 pModeData = &pVM->pgm.s.paModeData[pgmModeDataIndex(PGM_TYPE_NESTED, PGM_TYPE_PAE)];
2899 pModeData->uShwType = PGM_TYPE_NESTED;
2900 pModeData->uGstType = PGM_TYPE_PAE;
2901 rc = PGM_GST_NAME_PAE(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2902 rc = PGM_BTH_NAME_NESTED_PAE(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2903
2904#ifdef VBOX_WITH_64_BITS_GUESTS
2905 pModeData = &pVM->pgm.s.paModeData[pgmModeDataIndex(PGM_TYPE_NESTED, PGM_TYPE_AMD64)];
2906 pModeData->uShwType = PGM_TYPE_NESTED;
2907 pModeData->uGstType = PGM_TYPE_AMD64;
2908 rc = PGM_GST_NAME_AMD64(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2909 rc = PGM_BTH_NAME_NESTED_AMD64(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2910#endif
2911
2912 /* The shadow part of the nested callback mode depends on the host paging mode (AMD-V only). */
2913 switch (pVM->pgm.s.enmHostMode)
2914 {
2915#if HC_ARCH_BITS == 32
2916 case SUPPAGINGMODE_32_BIT:
2917 case SUPPAGINGMODE_32_BIT_GLOBAL:
2918 for (unsigned i = PGM_TYPE_REAL; i <= PGM_TYPE_PAE; i++)
2919 {
2920 pModeData = &pVM->pgm.s.paModeData[pgmModeDataIndex(PGM_TYPE_NESTED, i)];
2921 rc = PGM_SHW_NAME_32BIT(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2922 }
2923# ifdef VBOX_WITH_64_BITS_GUESTS
2924 pModeData = &pVM->pgm.s.paModeData[pgmModeDataIndex(PGM_TYPE_NESTED, PGM_TYPE_AMD64)];
2925 rc = PGM_SHW_NAME_AMD64(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2926# endif
2927 break;
2928
2929 case SUPPAGINGMODE_PAE:
2930 case SUPPAGINGMODE_PAE_NX:
2931 case SUPPAGINGMODE_PAE_GLOBAL:
2932 case SUPPAGINGMODE_PAE_GLOBAL_NX:
2933 for (unsigned i = PGM_TYPE_REAL; i <= PGM_TYPE_PAE; i++)
2934 {
2935 pModeData = &pVM->pgm.s.paModeData[pgmModeDataIndex(PGM_TYPE_NESTED, i)];
2936 rc = PGM_SHW_NAME_PAE(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2937 }
2938# ifdef VBOX_WITH_64_BITS_GUESTS
2939 pModeData = &pVM->pgm.s.paModeData[pgmModeDataIndex(PGM_TYPE_NESTED, PGM_TYPE_AMD64)];
2940 rc = PGM_SHW_NAME_AMD64(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2941# endif
2942 break;
2943#endif /* HC_ARCH_BITS == 32 */
2944
2945#if HC_ARCH_BITS == 64 || defined(RT_OS_DARWIN)
2946 case SUPPAGINGMODE_AMD64:
2947 case SUPPAGINGMODE_AMD64_GLOBAL:
2948 case SUPPAGINGMODE_AMD64_NX:
2949 case SUPPAGINGMODE_AMD64_GLOBAL_NX:
2950# ifdef VBOX_WITH_64_BITS_GUESTS
2951 for (unsigned i = PGM_TYPE_REAL; i <= PGM_TYPE_AMD64; i++)
2952# else
2953 for (unsigned i = PGM_TYPE_REAL; i <= PGM_TYPE_PAE; i++)
2954# endif
2955 {
2956 pModeData = &pVM->pgm.s.paModeData[pgmModeDataIndex(PGM_TYPE_NESTED, i)];
2957 rc = PGM_SHW_NAME_AMD64(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2958 }
2959 break;
2960#endif /* HC_ARCH_BITS == 64 || RT_OS_DARWIN */
2961
2962 default:
2963 AssertFailed();
2964 break;
2965 }
2966
2967 /* Extended paging (EPT) / Intel VT-x */
2968 pModeData = &pVM->pgm.s.paModeData[pgmModeDataIndex(PGM_TYPE_EPT, PGM_TYPE_REAL)];
2969 pModeData->uShwType = PGM_TYPE_EPT;
2970 pModeData->uGstType = PGM_TYPE_REAL;
2971 rc = PGM_SHW_NAME_EPT(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2972 rc = PGM_GST_NAME_REAL(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2973 rc = PGM_BTH_NAME_EPT_REAL(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2974
2975 pModeData = &pVM->pgm.s.paModeData[pgmModeDataIndex(PGM_TYPE_EPT, PGM_TYPE_PROT)];
2976 pModeData->uShwType = PGM_TYPE_EPT;
2977 pModeData->uGstType = PGM_TYPE_PROT;
2978 rc = PGM_SHW_NAME_EPT(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2979 rc = PGM_GST_NAME_PROT(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2980 rc = PGM_BTH_NAME_EPT_PROT(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2981
2982 pModeData = &pVM->pgm.s.paModeData[pgmModeDataIndex(PGM_TYPE_EPT, PGM_TYPE_32BIT)];
2983 pModeData->uShwType = PGM_TYPE_EPT;
2984 pModeData->uGstType = PGM_TYPE_32BIT;
2985 rc = PGM_SHW_NAME_EPT(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2986 rc = PGM_GST_NAME_32BIT(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2987 rc = PGM_BTH_NAME_EPT_32BIT(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2988
2989 pModeData = &pVM->pgm.s.paModeData[pgmModeDataIndex(PGM_TYPE_EPT, PGM_TYPE_PAE)];
2990 pModeData->uShwType = PGM_TYPE_EPT;
2991 pModeData->uGstType = PGM_TYPE_PAE;
2992 rc = PGM_SHW_NAME_EPT(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2993 rc = PGM_GST_NAME_PAE(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2994 rc = PGM_BTH_NAME_EPT_PAE(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
2995
2996#ifdef VBOX_WITH_64_BITS_GUESTS
2997 pModeData = &pVM->pgm.s.paModeData[pgmModeDataIndex(PGM_TYPE_EPT, PGM_TYPE_AMD64)];
2998 pModeData->uShwType = PGM_TYPE_EPT;
2999 pModeData->uGstType = PGM_TYPE_AMD64;
3000 rc = PGM_SHW_NAME_EPT(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
3001 rc = PGM_GST_NAME_AMD64(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
3002 rc = PGM_BTH_NAME_EPT_AMD64(InitData)( pVM, pModeData, fResolveGCAndR0); AssertRCReturn(rc, rc);
3003#endif
3004 return VINF_SUCCESS;
3005}
3006
3007
3008/**
3009 * Switch to different (or relocated in the relocate case) mode data.
3010 *
3011 * @param pVM The VM handle.
3012 * @param enmShw The the shadow paging mode.
3013 * @param enmGst The the guest paging mode.
3014 */
3015static void pgmR3ModeDataSwitch(PVM pVM, PGMMODE enmShw, PGMMODE enmGst)
3016{
3017 PPGMMODEDATA pModeData = &pVM->pgm.s.paModeData[pgmModeDataIndexByMode(enmShw, enmGst)];
3018
3019 Assert(pModeData->uGstType == pgmModeToType(enmGst));
3020 Assert(pModeData->uShwType == pgmModeToType(enmShw));
3021
3022 /* shadow */
3023 pVM->pgm.s.pfnR3ShwRelocate = pModeData->pfnR3ShwRelocate;
3024 pVM->pgm.s.pfnR3ShwExit = pModeData->pfnR3ShwExit;
3025 pVM->pgm.s.pfnR3ShwGetPage = pModeData->pfnR3ShwGetPage;
3026 Assert(pVM->pgm.s.pfnR3ShwGetPage);
3027 pVM->pgm.s.pfnR3ShwModifyPage = pModeData->pfnR3ShwModifyPage;
3028
3029 pVM->pgm.s.pfnRCShwGetPage = pModeData->pfnRCShwGetPage;
3030 pVM->pgm.s.pfnRCShwModifyPage = pModeData->pfnRCShwModifyPage;
3031
3032 pVM->pgm.s.pfnR0ShwGetPage = pModeData->pfnR0ShwGetPage;
3033 pVM->pgm.s.pfnR0ShwModifyPage = pModeData->pfnR0ShwModifyPage;
3034
3035
3036 /* guest */
3037 pVM->pgm.s.pfnR3GstRelocate = pModeData->pfnR3GstRelocate;
3038 pVM->pgm.s.pfnR3GstExit = pModeData->pfnR3GstExit;
3039 pVM->pgm.s.pfnR3GstGetPage = pModeData->pfnR3GstGetPage;
3040 Assert(pVM->pgm.s.pfnR3GstGetPage);
3041 pVM->pgm.s.pfnR3GstModifyPage = pModeData->pfnR3GstModifyPage;
3042 pVM->pgm.s.pfnR3GstGetPDE = pModeData->pfnR3GstGetPDE;
3043#ifndef VBOX_WITH_PGMPOOL_PAGING_ONLY
3044 pVM->pgm.s.pfnR3GstMonitorCR3 = pModeData->pfnR3GstMonitorCR3;
3045 pVM->pgm.s.pfnR3GstUnmonitorCR3 = pModeData->pfnR3GstUnmonitorCR3;
3046#endif
3047 pVM->pgm.s.pfnR3GstMapCR3 = pModeData->pfnR3GstMapCR3;
3048 pVM->pgm.s.pfnR3GstUnmapCR3 = pModeData->pfnR3GstUnmapCR3;
3049#ifndef VBOX_WITH_PGMPOOL_PAGING_ONLY
3050 pVM->pgm.s.pfnR3GstWriteHandlerCR3 = pModeData->pfnR3GstWriteHandlerCR3;
3051 pVM->pgm.s.pszR3GstWriteHandlerCR3 = pModeData->pszR3GstWriteHandlerCR3;
3052 pVM->pgm.s.pfnR3GstPAEWriteHandlerCR3 = pModeData->pfnR3GstPAEWriteHandlerCR3;
3053 pVM->pgm.s.pszR3GstPAEWriteHandlerCR3 = pModeData->pszR3GstPAEWriteHandlerCR3;
3054#endif
3055 pVM->pgm.s.pfnRCGstGetPage = pModeData->pfnRCGstGetPage;
3056 pVM->pgm.s.pfnRCGstModifyPage = pModeData->pfnRCGstModifyPage;
3057 pVM->pgm.s.pfnRCGstGetPDE = pModeData->pfnRCGstGetPDE;
3058#ifndef VBOX_WITH_PGMPOOL_PAGING_ONLY
3059 pVM->pgm.s.pfnRCGstMonitorCR3 = pModeData->pfnRCGstMonitorCR3;
3060 pVM->pgm.s.pfnRCGstUnmonitorCR3 = pModeData->pfnRCGstUnmonitorCR3;
3061#endif
3062 pVM->pgm.s.pfnRCGstMapCR3 = pModeData->pfnRCGstMapCR3;
3063 pVM->pgm.s.pfnRCGstUnmapCR3 = pModeData->pfnRCGstUnmapCR3;
3064#ifndef VBOX_WITH_PGMPOOL_PAGING_ONLY
3065 pVM->pgm.s.pfnRCGstWriteHandlerCR3 = pModeData->pfnRCGstWriteHandlerCR3;
3066 pVM->pgm.s.pfnRCGstPAEWriteHandlerCR3 = pModeData->pfnRCGstPAEWriteHandlerCR3;
3067#endif
3068 pVM->pgm.s.pfnR0GstGetPage = pModeData->pfnR0GstGetPage;
3069 pVM->pgm.s.pfnR0GstModifyPage = pModeData->pfnR0GstModifyPage;
3070 pVM->pgm.s.pfnR0GstGetPDE = pModeData->pfnR0GstGetPDE;
3071#ifndef VBOX_WITH_PGMPOOL_PAGING_ONLY
3072 pVM->pgm.s.pfnR0GstMonitorCR3 = pModeData->pfnR0GstMonitorCR3;
3073 pVM->pgm.s.pfnR0GstUnmonitorCR3 = pModeData->pfnR0GstUnmonitorCR3;
3074#endif
3075 pVM->pgm.s.pfnR0GstMapCR3 = pModeData->pfnR0GstMapCR3;
3076 pVM->pgm.s.pfnR0GstUnmapCR3 = pModeData->pfnR0GstUnmapCR3;
3077#ifndef VBOX_WITH_PGMPOOL_PAGING_ONLY
3078 pVM->pgm.s.pfnR0GstWriteHandlerCR3 = pModeData->pfnR0GstWriteHandlerCR3;
3079 pVM->pgm.s.pfnR0GstPAEWriteHandlerCR3 = pModeData->pfnR0GstPAEWriteHandlerCR3;
3080#endif
3081
3082 /* both */
3083 pVM->pgm.s.pfnR3BthRelocate = pModeData->pfnR3BthRelocate;
3084 pVM->pgm.s.pfnR3BthInvalidatePage = pModeData->pfnR3BthInvalidatePage;
3085 pVM->pgm.s.pfnR3BthSyncCR3 = pModeData->pfnR3BthSyncCR3;
3086 Assert(pVM->pgm.s.pfnR3BthSyncCR3);
3087 pVM->pgm.s.pfnR3BthSyncPage = pModeData->pfnR3BthSyncPage;
3088 pVM->pgm.s.pfnR3BthPrefetchPage = pModeData->pfnR3BthPrefetchPage;
3089 pVM->pgm.s.pfnR3BthVerifyAccessSyncPage = pModeData->pfnR3BthVerifyAccessSyncPage;
3090#ifdef VBOX_STRICT
3091 pVM->pgm.s.pfnR3BthAssertCR3 = pModeData->pfnR3BthAssertCR3;
3092#endif
3093
3094 pVM->pgm.s.pfnRCBthTrap0eHandler = pModeData->pfnRCBthTrap0eHandler;
3095 pVM->pgm.s.pfnRCBthInvalidatePage = pModeData->pfnRCBthInvalidatePage;
3096 pVM->pgm.s.pfnRCBthSyncCR3 = pModeData->pfnRCBthSyncCR3;
3097 pVM->pgm.s.pfnRCBthSyncPage = pModeData->pfnRCBthSyncPage;
3098 pVM->pgm.s.pfnRCBthPrefetchPage = pModeData->pfnRCBthPrefetchPage;
3099 pVM->pgm.s.pfnRCBthVerifyAccessSyncPage = pModeData->pfnRCBthVerifyAccessSyncPage;
3100#ifdef VBOX_STRICT
3101 pVM->pgm.s.pfnRCBthAssertCR3 = pModeData->pfnRCBthAssertCR3;
3102#endif
3103
3104 pVM->pgm.s.pfnR0BthTrap0eHandler = pModeData->pfnR0BthTrap0eHandler;
3105 pVM->pgm.s.pfnR0BthInvalidatePage = pModeData->pfnR0BthInvalidatePage;
3106 pVM->pgm.s.pfnR0BthSyncCR3 = pModeData->pfnR0BthSyncCR3;
3107 pVM->pgm.s.pfnR0BthSyncPage = pModeData->pfnR0BthSyncPage;
3108 pVM->pgm.s.pfnR0BthPrefetchPage = pModeData->pfnR0BthPrefetchPage;
3109 pVM->pgm.s.pfnR0BthVerifyAccessSyncPage = pModeData->pfnR0BthVerifyAccessSyncPage;
3110#ifdef VBOX_STRICT
3111 pVM->pgm.s.pfnR0BthAssertCR3 = pModeData->pfnR0BthAssertCR3;
3112#endif
3113}
3114
3115
3116/**
3117 * Calculates the shadow paging mode.
3118 *
3119 * @returns The shadow paging mode.
3120 * @param pVM VM handle.
3121 * @param enmGuestMode The guest mode.
3122 * @param enmHostMode The host mode.
3123 * @param enmShadowMode The current shadow mode.
3124 * @param penmSwitcher Where to store the switcher to use.
3125 * VMMSWITCHER_INVALID means no change.
3126 */
3127static PGMMODE pgmR3CalcShadowMode(PVM pVM, PGMMODE enmGuestMode, SUPPAGINGMODE enmHostMode, PGMMODE enmShadowMode, VMMSWITCHER *penmSwitcher)
3128{
3129 VMMSWITCHER enmSwitcher = VMMSWITCHER_INVALID;
3130 switch (enmGuestMode)
3131 {
3132 /*
3133 * When switching to real or protected mode we don't change
3134 * anything since it's likely that we'll switch back pretty soon.
3135 *
3136 * During pgmR3InitPaging we'll end up here with PGMMODE_INVALID
3137 * and is supposed to determine which shadow paging and switcher to
3138 * use during init.
3139 */
3140 case PGMMODE_REAL:
3141 case PGMMODE_PROTECTED:
3142 if ( enmShadowMode != PGMMODE_INVALID
3143 && !HWACCMIsEnabled(pVM) /* always switch in hwaccm mode! */)
3144 break; /* (no change) */
3145
3146 switch (enmHostMode)
3147 {
3148 case SUPPAGINGMODE_32_BIT:
3149 case SUPPAGINGMODE_32_BIT_GLOBAL:
3150 enmShadowMode = PGMMODE_32_BIT;
3151 enmSwitcher = VMMSWITCHER_32_TO_32;
3152 break;
3153
3154 case SUPPAGINGMODE_PAE:
3155 case SUPPAGINGMODE_PAE_NX:
3156 case SUPPAGINGMODE_PAE_GLOBAL:
3157 case SUPPAGINGMODE_PAE_GLOBAL_NX:
3158 enmShadowMode = PGMMODE_PAE;
3159 enmSwitcher = VMMSWITCHER_PAE_TO_PAE;
3160#ifdef DEBUG_bird
3161 if (RTEnvExist("VBOX_32BIT"))
3162 {
3163 enmShadowMode = PGMMODE_32_BIT;
3164 enmSwitcher = VMMSWITCHER_PAE_TO_32;
3165 }
3166#endif
3167 break;
3168
3169 case SUPPAGINGMODE_AMD64:
3170 case SUPPAGINGMODE_AMD64_GLOBAL:
3171 case SUPPAGINGMODE_AMD64_NX:
3172 case SUPPAGINGMODE_AMD64_GLOBAL_NX:
3173 enmShadowMode = PGMMODE_PAE;
3174 enmSwitcher = VMMSWITCHER_AMD64_TO_PAE;
3175#ifdef DEBUG_bird
3176 if (RTEnvExist("VBOX_32BIT"))
3177 {
3178 enmShadowMode = PGMMODE_32_BIT;
3179 enmSwitcher = VMMSWITCHER_AMD64_TO_32;
3180 }
3181#endif
3182 break;
3183
3184 default: AssertMsgFailed(("enmHostMode=%d\n", enmHostMode)); break;
3185 }
3186 break;
3187
3188 case PGMMODE_32_BIT:
3189 switch (enmHostMode)
3190 {
3191 case SUPPAGINGMODE_32_BIT:
3192 case SUPPAGINGMODE_32_BIT_GLOBAL:
3193 enmShadowMode = PGMMODE_32_BIT;
3194 enmSwitcher = VMMSWITCHER_32_TO_32;
3195 break;
3196
3197 case SUPPAGINGMODE_PAE:
3198 case SUPPAGINGMODE_PAE_NX:
3199 case SUPPAGINGMODE_PAE_GLOBAL:
3200 case SUPPAGINGMODE_PAE_GLOBAL_NX:
3201 enmShadowMode = PGMMODE_PAE;
3202 enmSwitcher = VMMSWITCHER_PAE_TO_PAE;
3203#ifdef DEBUG_bird
3204 if (RTEnvExist("VBOX_32BIT"))
3205 {
3206 enmShadowMode = PGMMODE_32_BIT;
3207 enmSwitcher = VMMSWITCHER_PAE_TO_32;
3208 }
3209#endif
3210 break;
3211
3212 case SUPPAGINGMODE_AMD64:
3213 case SUPPAGINGMODE_AMD64_GLOBAL:
3214 case SUPPAGINGMODE_AMD64_NX:
3215 case SUPPAGINGMODE_AMD64_GLOBAL_NX:
3216 enmShadowMode = PGMMODE_PAE;
3217 enmSwitcher = VMMSWITCHER_AMD64_TO_PAE;
3218#ifdef DEBUG_bird
3219 if (RTEnvExist("VBOX_32BIT"))
3220 {
3221 enmShadowMode = PGMMODE_32_BIT;
3222 enmSwitcher = VMMSWITCHER_AMD64_TO_32;
3223 }
3224#endif
3225 break;
3226
3227 default: AssertMsgFailed(("enmHostMode=%d\n", enmHostMode)); break;
3228 }
3229 break;
3230
3231 case PGMMODE_PAE:
3232 case PGMMODE_PAE_NX: /** @todo This might require more switchers and guest+both modes. */
3233 switch (enmHostMode)
3234 {
3235 case SUPPAGINGMODE_32_BIT:
3236 case SUPPAGINGMODE_32_BIT_GLOBAL:
3237 enmShadowMode = PGMMODE_PAE;
3238 enmSwitcher = VMMSWITCHER_32_TO_PAE;
3239 break;
3240
3241 case SUPPAGINGMODE_PAE:
3242 case SUPPAGINGMODE_PAE_NX:
3243 case SUPPAGINGMODE_PAE_GLOBAL:
3244 case SUPPAGINGMODE_PAE_GLOBAL_NX:
3245 enmShadowMode = PGMMODE_PAE;
3246 enmSwitcher = VMMSWITCHER_PAE_TO_PAE;
3247 break;
3248
3249 case SUPPAGINGMODE_AMD64:
3250 case SUPPAGINGMODE_AMD64_GLOBAL:
3251 case SUPPAGINGMODE_AMD64_NX:
3252 case SUPPAGINGMODE_AMD64_GLOBAL_NX:
3253 enmShadowMode = PGMMODE_PAE;
3254 enmSwitcher = VMMSWITCHER_AMD64_TO_PAE;
3255 break;
3256
3257 default: AssertMsgFailed(("enmHostMode=%d\n", enmHostMode)); break;
3258 }
3259 break;
3260
3261 case PGMMODE_AMD64:
3262 case PGMMODE_AMD64_NX:
3263 switch (enmHostMode)
3264 {
3265 case SUPPAGINGMODE_32_BIT:
3266 case SUPPAGINGMODE_32_BIT_GLOBAL:
3267 enmShadowMode = PGMMODE_AMD64;
3268 enmSwitcher = VMMSWITCHER_32_TO_AMD64;
3269 break;
3270
3271 case SUPPAGINGMODE_PAE:
3272 case SUPPAGINGMODE_PAE_NX:
3273 case SUPPAGINGMODE_PAE_GLOBAL:
3274 case SUPPAGINGMODE_PAE_GLOBAL_NX:
3275 enmShadowMode = PGMMODE_AMD64;
3276 enmSwitcher = VMMSWITCHER_PAE_TO_AMD64;
3277 break;
3278
3279 case SUPPAGINGMODE_AMD64:
3280 case SUPPAGINGMODE_AMD64_GLOBAL:
3281 case SUPPAGINGMODE_AMD64_NX:
3282 case SUPPAGINGMODE_AMD64_GLOBAL_NX:
3283 enmShadowMode = PGMMODE_AMD64;
3284 enmSwitcher = VMMSWITCHER_AMD64_TO_AMD64;
3285 break;
3286
3287 default: AssertMsgFailed(("enmHostMode=%d\n", enmHostMode)); break;
3288 }
3289 break;
3290
3291
3292 default:
3293 AssertReleaseMsgFailed(("enmGuestMode=%d\n", enmGuestMode));
3294 return PGMMODE_INVALID;
3295 }
3296 /* Override the shadow mode is nested paging is active. */
3297 if (HWACCMIsNestedPagingActive(pVM))
3298 enmShadowMode = HWACCMGetShwPagingMode(pVM);
3299
3300 *penmSwitcher = enmSwitcher;
3301 return enmShadowMode;
3302}
3303
3304
3305/**
3306 * Performs the actual mode change.
3307 * This is called by PGMChangeMode and pgmR3InitPaging().
3308 *
3309 * @returns VBox status code.
3310 * @param pVM VM handle.
3311 * @param enmGuestMode The new guest mode. This is assumed to be different from
3312 * the current mode.
3313 */
3314VMMR3DECL(int) PGMR3ChangeMode(PVM pVM, PGMMODE enmGuestMode)
3315{
3316 Log(("PGMR3ChangeMode: Guest mode: %s -> %s\n", PGMGetModeName(pVM->pgm.s.enmGuestMode), PGMGetModeName(enmGuestMode)));
3317 STAM_REL_COUNTER_INC(&pVM->pgm.s.cGuestModeChanges);
3318
3319 /*
3320 * Calc the shadow mode and switcher.
3321 */
3322 VMMSWITCHER enmSwitcher;
3323 PGMMODE enmShadowMode = pgmR3CalcShadowMode(pVM, enmGuestMode, pVM->pgm.s.enmHostMode, pVM->pgm.s.enmShadowMode, &enmSwitcher);
3324 if (enmSwitcher != VMMSWITCHER_INVALID)
3325 {
3326 /*
3327 * Select new switcher.
3328 */
3329 int rc = VMMR3SelectSwitcher(pVM, enmSwitcher);
3330 if (RT_FAILURE(rc))
3331 {
3332 AssertReleaseMsgFailed(("VMMR3SelectSwitcher(%d) -> %Rrc\n", enmSwitcher, rc));
3333 return rc;
3334 }
3335 }
3336
3337 /*
3338 * Exit old mode(s).
3339 */
3340 /* shadow */
3341 if (enmShadowMode != pVM->pgm.s.enmShadowMode)
3342 {
3343 LogFlow(("PGMR3ChangeMode: Shadow mode: %s -> %s\n", PGMGetModeName(pVM->pgm.s.enmShadowMode), PGMGetModeName(enmShadowMode)));
3344 if (PGM_SHW_PFN(Exit, pVM))
3345 {
3346 int rc = PGM_SHW_PFN(Exit, pVM)(pVM);
3347 if (RT_FAILURE(rc))
3348 {
3349 AssertMsgFailed(("Exit failed for shadow mode %d: %Rrc\n", pVM->pgm.s.enmShadowMode, rc));
3350 return rc;
3351 }
3352 }
3353
3354 }
3355 else
3356 LogFlow(("PGMR3ChangeMode: Shadow mode remains: %s\n", PGMGetModeName(pVM->pgm.s.enmShadowMode)));
3357
3358 /* guest */
3359 if (PGM_GST_PFN(Exit, pVM))
3360 {
3361 int rc = PGM_GST_PFN(Exit, pVM)(pVM);
3362 if (RT_FAILURE(rc))
3363 {
3364 AssertMsgFailed(("Exit failed for guest mode %d: %Rrc\n", pVM->pgm.s.enmGuestMode, rc));
3365 return rc;
3366 }
3367 }
3368
3369 /*
3370 * Load new paging mode data.
3371 */
3372 pgmR3ModeDataSwitch(pVM, enmShadowMode, enmGuestMode);
3373
3374 /*
3375 * Enter new shadow mode (if changed).
3376 */
3377 if (enmShadowMode != pVM->pgm.s.enmShadowMode)
3378 {
3379 int rc;
3380 pVM->pgm.s.enmShadowMode = enmShadowMode;
3381 switch (enmShadowMode)
3382 {
3383 case PGMMODE_32_BIT:
3384 rc = PGM_SHW_NAME_32BIT(Enter)(pVM);
3385 break;
3386 case PGMMODE_PAE:
3387 case PGMMODE_PAE_NX:
3388 rc = PGM_SHW_NAME_PAE(Enter)(pVM);
3389 break;
3390 case PGMMODE_AMD64:
3391 case PGMMODE_AMD64_NX:
3392 rc = PGM_SHW_NAME_AMD64(Enter)(pVM);
3393 break;
3394 case PGMMODE_NESTED:
3395 rc = PGM_SHW_NAME_NESTED(Enter)(pVM);
3396 break;
3397 case PGMMODE_EPT:
3398 rc = PGM_SHW_NAME_EPT(Enter)(pVM);
3399 break;
3400 case PGMMODE_REAL:
3401 case PGMMODE_PROTECTED:
3402 default:
3403 AssertReleaseMsgFailed(("enmShadowMode=%d\n", enmShadowMode));
3404 return VERR_INTERNAL_ERROR;
3405 }
3406 if (RT_FAILURE(rc))
3407 {
3408 AssertReleaseMsgFailed(("Entering enmShadowMode=%d failed: %Rrc\n", enmShadowMode, rc));
3409 pVM->pgm.s.enmShadowMode = PGMMODE_INVALID;
3410 return rc;
3411 }
3412 }
3413
3414 /** @todo This is a bug!
3415 *
3416 * We must flush the PGM pool cache if the guest mode changes; we don't always
3417 * switch shadow paging mode (e.g. protected->32-bit) and shouldn't reuse
3418 * the shadow page tables.
3419 *
3420 * That only applies when switching between paging and non-paging modes.
3421 */
3422 /** @todo A20 setting */
3423 if ( pVM->pgm.s.CTX_SUFF(pPool)
3424 && !HWACCMIsNestedPagingActive(pVM)
3425 && PGMMODE_WITH_PAGING(pVM->pgm.s.enmGuestMode) != PGMMODE_WITH_PAGING(enmGuestMode))
3426 {
3427 Log(("PGMR3ChangeMode: changing guest paging mode -> flush pgm pool cache!\n"));
3428 pgmPoolFlushAll(pVM);
3429 }
3430
3431 /*
3432 * Enter the new guest and shadow+guest modes.
3433 */
3434 int rc = -1;
3435 int rc2 = -1;
3436 RTGCPHYS GCPhysCR3 = NIL_RTGCPHYS;
3437 pVM->pgm.s.enmGuestMode = enmGuestMode;
3438 switch (enmGuestMode)
3439 {
3440 case PGMMODE_REAL:
3441 rc = PGM_GST_NAME_REAL(Enter)(pVM, NIL_RTGCPHYS);
3442 switch (pVM->pgm.s.enmShadowMode)
3443 {
3444 case PGMMODE_32_BIT:
3445 rc2 = PGM_BTH_NAME_32BIT_REAL(Enter)(pVM, NIL_RTGCPHYS);
3446 break;
3447 case PGMMODE_PAE:
3448 case PGMMODE_PAE_NX:
3449 rc2 = PGM_BTH_NAME_PAE_REAL(Enter)(pVM, NIL_RTGCPHYS);
3450 break;
3451 case PGMMODE_NESTED:
3452 rc2 = PGM_BTH_NAME_NESTED_REAL(Enter)(pVM, NIL_RTGCPHYS);
3453 break;
3454 case PGMMODE_EPT:
3455 rc2 = PGM_BTH_NAME_EPT_REAL(Enter)(pVM, NIL_RTGCPHYS);
3456 break;
3457 case PGMMODE_AMD64:
3458 case PGMMODE_AMD64_NX:
3459 AssertMsgFailed(("Should use PAE shadow mode!\n"));
3460 default: AssertFailed(); break;
3461 }
3462 break;
3463
3464 case PGMMODE_PROTECTED:
3465 rc = PGM_GST_NAME_PROT(Enter)(pVM, NIL_RTGCPHYS);
3466 switch (pVM->pgm.s.enmShadowMode)
3467 {
3468 case PGMMODE_32_BIT:
3469 rc2 = PGM_BTH_NAME_32BIT_PROT(Enter)(pVM, NIL_RTGCPHYS);
3470 break;
3471 case PGMMODE_PAE:
3472 case PGMMODE_PAE_NX:
3473 rc2 = PGM_BTH_NAME_PAE_PROT(Enter)(pVM, NIL_RTGCPHYS);
3474 break;
3475 case PGMMODE_NESTED:
3476 rc2 = PGM_BTH_NAME_NESTED_PROT(Enter)(pVM, NIL_RTGCPHYS);
3477 break;
3478 case PGMMODE_EPT:
3479 rc2 = PGM_BTH_NAME_EPT_PROT(Enter)(pVM, NIL_RTGCPHYS);
3480 break;
3481 case PGMMODE_AMD64:
3482 case PGMMODE_AMD64_NX:
3483 AssertMsgFailed(("Should use PAE shadow mode!\n"));
3484 default: AssertFailed(); break;
3485 }
3486 break;
3487
3488 case PGMMODE_32_BIT:
3489 GCPhysCR3 = CPUMGetGuestCR3(pVM) & X86_CR3_PAGE_MASK;
3490 rc = PGM_GST_NAME_32BIT(Enter)(pVM, GCPhysCR3);
3491 switch (pVM->pgm.s.enmShadowMode)
3492 {
3493 case PGMMODE_32_BIT:
3494 rc2 = PGM_BTH_NAME_32BIT_32BIT(Enter)(pVM, GCPhysCR3);
3495 break;
3496 case PGMMODE_PAE:
3497 case PGMMODE_PAE_NX:
3498 rc2 = PGM_BTH_NAME_PAE_32BIT(Enter)(pVM, GCPhysCR3);
3499 break;
3500 case PGMMODE_NESTED:
3501 rc2 = PGM_BTH_NAME_NESTED_32BIT(Enter)(pVM, GCPhysCR3);
3502 break;
3503 case PGMMODE_EPT:
3504 rc2 = PGM_BTH_NAME_EPT_32BIT(Enter)(pVM, GCPhysCR3);
3505 break;
3506 case PGMMODE_AMD64:
3507 case PGMMODE_AMD64_NX:
3508 AssertMsgFailed(("Should use PAE shadow mode!\n"));
3509 default: AssertFailed(); break;
3510 }
3511 break;
3512
3513 case PGMMODE_PAE_NX:
3514 case PGMMODE_PAE:
3515 {
3516 uint32_t u32Dummy, u32Features;
3517
3518 CPUMGetGuestCpuId(pVM, 1, &u32Dummy, &u32Dummy, &u32Dummy, &u32Features);
3519 if (!(u32Features & X86_CPUID_FEATURE_EDX_PAE))
3520 {
3521 /* Pause first, then inform Main. */
3522 rc = VMR3SuspendNoSave(pVM);
3523 AssertRC(rc);
3524
3525 VMSetRuntimeError(pVM, true, "PAEmode",
3526 N_("The guest is trying to switch to the PAE mode which is currently disabled by default in VirtualBox. PAE support can be enabled using the VM settings (General/Advanced)"));
3527 /* we must return VINF_SUCCESS here otherwise the recompiler will assert */
3528 return VINF_SUCCESS;
3529 }
3530 GCPhysCR3 = CPUMGetGuestCR3(pVM) & X86_CR3_PAE_PAGE_MASK;
3531 rc = PGM_GST_NAME_PAE(Enter)(pVM, GCPhysCR3);
3532 switch (pVM->pgm.s.enmShadowMode)
3533 {
3534 case PGMMODE_PAE:
3535 case PGMMODE_PAE_NX:
3536 rc2 = PGM_BTH_NAME_PAE_PAE(Enter)(pVM, GCPhysCR3);
3537 break;
3538 case PGMMODE_NESTED:
3539 rc2 = PGM_BTH_NAME_NESTED_PAE(Enter)(pVM, GCPhysCR3);
3540 break;
3541 case PGMMODE_EPT:
3542 rc2 = PGM_BTH_NAME_EPT_PAE(Enter)(pVM, GCPhysCR3);
3543 break;
3544 case PGMMODE_32_BIT:
3545 case PGMMODE_AMD64:
3546 case PGMMODE_AMD64_NX:
3547 AssertMsgFailed(("Should use PAE shadow mode!\n"));
3548 default: AssertFailed(); break;
3549 }
3550 break;
3551 }
3552
3553#ifdef VBOX_WITH_64_BITS_GUESTS
3554 case PGMMODE_AMD64_NX:
3555 case PGMMODE_AMD64:
3556 GCPhysCR3 = CPUMGetGuestCR3(pVM) & UINT64_C(0xfffffffffffff000); /** @todo define this mask! */
3557 rc = PGM_GST_NAME_AMD64(Enter)(pVM, GCPhysCR3);
3558 switch (pVM->pgm.s.enmShadowMode)
3559 {
3560 case PGMMODE_AMD64:
3561 case PGMMODE_AMD64_NX:
3562 rc2 = PGM_BTH_NAME_AMD64_AMD64(Enter)(pVM, GCPhysCR3);
3563 break;
3564 case PGMMODE_NESTED:
3565 rc2 = PGM_BTH_NAME_NESTED_AMD64(Enter)(pVM, GCPhysCR3);
3566 break;
3567 case PGMMODE_EPT:
3568 rc2 = PGM_BTH_NAME_EPT_AMD64(Enter)(pVM, GCPhysCR3);
3569 break;
3570 case PGMMODE_32_BIT:
3571 case PGMMODE_PAE:
3572 case PGMMODE_PAE_NX:
3573 AssertMsgFailed(("Should use AMD64 shadow mode!\n"));
3574 default: AssertFailed(); break;
3575 }
3576 break;
3577#endif
3578
3579 default:
3580 AssertReleaseMsgFailed(("enmGuestMode=%d\n", enmGuestMode));
3581 rc = VERR_NOT_IMPLEMENTED;
3582 break;
3583 }
3584
3585 /* status codes. */
3586 AssertRC(rc);
3587 AssertRC(rc2);
3588 if (RT_SUCCESS(rc))
3589 {
3590 rc = rc2;
3591 if (RT_SUCCESS(rc)) /* no informational status codes. */
3592 rc = VINF_SUCCESS;
3593 }
3594
3595 /*
3596 * Notify SELM so it can update the TSSes with correct CR3s.
3597 */
3598 SELMR3PagingModeChanged(pVM);
3599
3600 /* Notify HWACCM as well. */
3601 HWACCMR3PagingModeChanged(pVM, pVM->pgm.s.enmShadowMode, pVM->pgm.s.enmGuestMode);
3602 return rc;
3603}
3604
3605
3606/**
3607 * Dumps a PAE shadow page table.
3608 *
3609 * @returns VBox status code (VINF_SUCCESS).
3610 * @param pVM The VM handle.
3611 * @param pPT Pointer to the page table.
3612 * @param u64Address The virtual address of the page table starts.
3613 * @param fLongMode Set if this a long mode table; clear if it's a legacy mode table.
3614 * @param cMaxDepth The maxium depth.
3615 * @param pHlp Pointer to the output functions.
3616 */
3617static int pgmR3DumpHierarchyHCPaePT(PVM pVM, PX86PTPAE pPT, uint64_t u64Address, bool fLongMode, unsigned cMaxDepth, PCDBGFINFOHLP pHlp)
3618{
3619 for (unsigned i = 0; i < RT_ELEMENTS(pPT->a); i++)
3620 {
3621 X86PTEPAE Pte = pPT->a[i];
3622 if (Pte.n.u1Present)
3623 {
3624 pHlp->pfnPrintf(pHlp,
3625 fLongMode /*P R S A D G WT CD AT NX 4M a p ? */
3626 ? "%016llx 3 | P %c %c %c %c %c %s %s %s %s 4K %c%c%c %016llx\n"
3627 : "%08llx 2 | P %c %c %c %c %c %s %s %s %s 4K %c%c%c %016llx\n",
3628 u64Address + ((uint64_t)i << X86_PT_PAE_SHIFT),
3629 Pte.n.u1Write ? 'W' : 'R',
3630 Pte.n.u1User ? 'U' : 'S',
3631 Pte.n.u1Accessed ? 'A' : '-',
3632 Pte.n.u1Dirty ? 'D' : '-',
3633 Pte.n.u1Global ? 'G' : '-',
3634 Pte.n.u1WriteThru ? "WT" : "--",
3635 Pte.n.u1CacheDisable? "CD" : "--",
3636 Pte.n.u1PAT ? "AT" : "--",
3637 Pte.n.u1NoExecute ? "NX" : "--",
3638 Pte.u & PGM_PTFLAGS_TRACK_DIRTY ? 'd' : '-',
3639 Pte.u & RT_BIT(10) ? '1' : '0',
3640 Pte.u & PGM_PTFLAGS_CSAM_VALIDATED? 'v' : '-',
3641 Pte.u & X86_PTE_PAE_PG_MASK);
3642 }
3643 }
3644 return VINF_SUCCESS;
3645}
3646
3647
3648/**
3649 * Dumps a PAE shadow page directory table.
3650 *
3651 * @returns VBox status code (VINF_SUCCESS).
3652 * @param pVM The VM handle.
3653 * @param HCPhys The physical address of the page directory table.
3654 * @param u64Address The virtual address of the page table starts.
3655 * @param cr4 The CR4, PSE is currently used.
3656 * @param fLongMode Set if this a long mode table; clear if it's a legacy mode table.
3657 * @param cMaxDepth The maxium depth.
3658 * @param pHlp Pointer to the output functions.
3659 */
3660static int pgmR3DumpHierarchyHCPaePD(PVM pVM, RTHCPHYS HCPhys, uint64_t u64Address, uint32_t cr4, bool fLongMode, unsigned cMaxDepth, PCDBGFINFOHLP pHlp)
3661{
3662 PX86PDPAE pPD = (PX86PDPAE)MMPagePhys2Page(pVM, HCPhys);
3663 if (!pPD)
3664 {
3665 pHlp->pfnPrintf(pHlp, "%0*llx error! Page directory at HCPhys=%RHp was not found in the page pool!\n",
3666 fLongMode ? 16 : 8, u64Address, HCPhys);
3667 return VERR_INVALID_PARAMETER;
3668 }
3669 const bool fBigPagesSupported = fLongMode || !!(cr4 & X86_CR4_PSE);
3670
3671 int rc = VINF_SUCCESS;
3672 for (unsigned i = 0; i < RT_ELEMENTS(pPD->a); i++)
3673 {
3674 X86PDEPAE Pde = pPD->a[i];
3675 if (Pde.n.u1Present)
3676 {
3677 if (fBigPagesSupported && Pde.b.u1Size)
3678 pHlp->pfnPrintf(pHlp,
3679 fLongMode /*P R S A D G WT CD AT NX 4M a p ? */
3680 ? "%016llx 2 | P %c %c %c %c %c %s %s %s %s 4M %c%c%c %016llx\n"
3681 : "%08llx 1 | P %c %c %c %c %c %s %s %s %s 4M %c%c%c %016llx\n",
3682 u64Address + ((uint64_t)i << X86_PD_PAE_SHIFT),
3683 Pde.b.u1Write ? 'W' : 'R',
3684 Pde.b.u1User ? 'U' : 'S',
3685 Pde.b.u1Accessed ? 'A' : '-',
3686 Pde.b.u1Dirty ? 'D' : '-',
3687 Pde.b.u1Global ? 'G' : '-',
3688 Pde.b.u1WriteThru ? "WT" : "--",
3689 Pde.b.u1CacheDisable? "CD" : "--",
3690 Pde.b.u1PAT ? "AT" : "--",
3691 Pde.b.u1NoExecute ? "NX" : "--",
3692 Pde.u & RT_BIT_64(9) ? '1' : '0',
3693 Pde.u & PGM_PDFLAGS_MAPPING ? 'm' : '-',
3694 Pde.u & PGM_PDFLAGS_TRACK_DIRTY ? 'd' : '-',
3695 Pde.u & X86_PDE_PAE_PG_MASK);
3696 else
3697 {
3698 pHlp->pfnPrintf(pHlp,
3699 fLongMode /*P R S A D G WT CD AT NX 4M a p ? */
3700 ? "%016llx 2 | P %c %c %c %c %c %s %s .. %s 4K %c%c%c %016llx\n"
3701 : "%08llx 1 | P %c %c %c %c %c %s %s .. %s 4K %c%c%c %016llx\n",
3702 u64Address + ((uint64_t)i << X86_PD_PAE_SHIFT),
3703 Pde.n.u1Write ? 'W' : 'R',
3704 Pde.n.u1User ? 'U' : 'S',
3705 Pde.n.u1Accessed ? 'A' : '-',
3706 Pde.n.u1Reserved0 ? '?' : '.', /* ignored */
3707 Pde.n.u1Reserved1 ? '?' : '.', /* ignored */
3708 Pde.n.u1WriteThru ? "WT" : "--",
3709 Pde.n.u1CacheDisable? "CD" : "--",
3710 Pde.n.u1NoExecute ? "NX" : "--",
3711 Pde.u & RT_BIT_64(9) ? '1' : '0',
3712 Pde.u & PGM_PDFLAGS_MAPPING ? 'm' : '-',
3713 Pde.u & PGM_PDFLAGS_TRACK_DIRTY ? 'd' : '-',
3714 Pde.u & X86_PDE_PAE_PG_MASK);
3715 if (cMaxDepth >= 1)
3716 {
3717 /** @todo what about using the page pool for mapping PTs? */
3718 uint64_t u64AddressPT = u64Address + ((uint64_t)i << X86_PD_PAE_SHIFT);
3719 RTHCPHYS HCPhysPT = Pde.u & X86_PDE_PAE_PG_MASK;
3720 PX86PTPAE pPT = NULL;
3721 if (!(Pde.u & PGM_PDFLAGS_MAPPING))
3722 pPT = (PX86PTPAE)MMPagePhys2Page(pVM, HCPhysPT);
3723 else
3724 {
3725 for (PPGMMAPPING pMap = pVM->pgm.s.pMappingsR3; pMap; pMap = pMap->pNextR3)
3726 {
3727 uint64_t off = u64AddressPT - pMap->GCPtr;
3728 if (off < pMap->cb)
3729 {
3730 const int iPDE = (uint32_t)(off >> X86_PD_SHIFT);
3731 const int iSub = (int)((off >> X86_PD_PAE_SHIFT) & 1); /* MSC is a pain sometimes */
3732 if ((iSub ? pMap->aPTs[iPDE].HCPhysPaePT1 : pMap->aPTs[iPDE].HCPhysPaePT0) != HCPhysPT)
3733 pHlp->pfnPrintf(pHlp, "%0*llx error! Mapping error! PT %d has HCPhysPT=%RHp not %RHp is in the PD.\n",
3734 fLongMode ? 16 : 8, u64AddressPT, iPDE,
3735 iSub ? pMap->aPTs[iPDE].HCPhysPaePT1 : pMap->aPTs[iPDE].HCPhysPaePT0, HCPhysPT);
3736 pPT = &pMap->aPTs[iPDE].paPaePTsR3[iSub];
3737 }
3738 }
3739 }
3740 int rc2 = VERR_INVALID_PARAMETER;
3741 if (pPT)
3742 rc2 = pgmR3DumpHierarchyHCPaePT(pVM, pPT, u64AddressPT, fLongMode, cMaxDepth - 1, pHlp);
3743 else
3744 pHlp->pfnPrintf(pHlp, "%0*llx error! Page table at HCPhys=%RHp was not found in the page pool!\n",
3745 fLongMode ? 16 : 8, u64AddressPT, HCPhysPT);
3746 if (rc2 < rc && RT_SUCCESS(rc))
3747 rc = rc2;
3748 }
3749 }
3750 }
3751 }
3752 return rc;
3753}
3754
3755
3756/**
3757 * Dumps a PAE shadow page directory pointer table.
3758 *
3759 * @returns VBox status code (VINF_SUCCESS).
3760 * @param pVM The VM handle.
3761 * @param HCPhys The physical address of the page directory pointer table.
3762 * @param u64Address The virtual address of the page table starts.
3763 * @param cr4 The CR4, PSE is currently used.
3764 * @param fLongMode Set if this a long mode table; clear if it's a legacy mode table.
3765 * @param cMaxDepth The maxium depth.
3766 * @param pHlp Pointer to the output functions.
3767 */
3768static int pgmR3DumpHierarchyHCPaePDPT(PVM pVM, RTHCPHYS HCPhys, uint64_t u64Address, uint32_t cr4, bool fLongMode, unsigned cMaxDepth, PCDBGFINFOHLP pHlp)
3769{
3770 PX86PDPT pPDPT = (PX86PDPT)MMPagePhys2Page(pVM, HCPhys);
3771 if (!pPDPT)
3772 {
3773 pHlp->pfnPrintf(pHlp, "%0*llx error! Page directory pointer table at HCPhys=%RHp was not found in the page pool!\n",
3774 fLongMode ? 16 : 8, u64Address, HCPhys);
3775 return VERR_INVALID_PARAMETER;
3776 }
3777
3778 int rc = VINF_SUCCESS;
3779 const unsigned c = fLongMode ? RT_ELEMENTS(pPDPT->a) : X86_PG_PAE_PDPE_ENTRIES;
3780 for (unsigned i = 0; i < c; i++)
3781 {
3782 X86PDPE Pdpe = pPDPT->a[i];
3783 if (Pdpe.n.u1Present)
3784 {
3785 if (fLongMode)
3786 pHlp->pfnPrintf(pHlp, /*P R S A D G WT CD AT NX 4M a p ? */
3787 "%016llx 1 | P %c %c %c %c %c %s %s %s %s .. %c%c%c %016llx\n",
3788 u64Address + ((uint64_t)i << X86_PDPT_SHIFT),
3789 Pdpe.lm.u1Write ? 'W' : 'R',
3790 Pdpe.lm.u1User ? 'U' : 'S',
3791 Pdpe.lm.u1Accessed ? 'A' : '-',
3792 Pdpe.lm.u3Reserved & 1? '?' : '.', /* ignored */
3793 Pdpe.lm.u3Reserved & 4? '!' : '.', /* mbz */
3794 Pdpe.lm.u1WriteThru ? "WT" : "--",
3795 Pdpe.lm.u1CacheDisable? "CD" : "--",
3796 Pdpe.lm.u3Reserved & 2? "!" : "..",/* mbz */
3797 Pdpe.lm.u1NoExecute ? "NX" : "--",
3798 Pdpe.u & RT_BIT(9) ? '1' : '0',
3799 Pdpe.u & PGM_PLXFLAGS_PERMANENT ? 'p' : '-',
3800 Pdpe.u & RT_BIT(11) ? '1' : '0',
3801 Pdpe.u & X86_PDPE_PG_MASK);
3802 else
3803 pHlp->pfnPrintf(pHlp, /*P G WT CD AT NX 4M a p ? */
3804 "%08x 0 | P %c %s %s %s %s .. %c%c%c %016llx\n",
3805 i << X86_PDPT_SHIFT,
3806 Pdpe.n.u4Reserved & 1? '!' : '.', /* mbz */
3807 Pdpe.n.u4Reserved & 4? '!' : '.', /* mbz */
3808 Pdpe.n.u1WriteThru ? "WT" : "--",
3809 Pdpe.n.u1CacheDisable? "CD" : "--",
3810 Pdpe.n.u4Reserved & 2? "!" : "..",/* mbz */
3811 Pdpe.u & RT_BIT(9) ? '1' : '0',
3812 Pdpe.u & PGM_PLXFLAGS_PERMANENT ? 'p' : '-',
3813 Pdpe.u & RT_BIT(11) ? '1' : '0',
3814 Pdpe.u & X86_PDPE_PG_MASK);
3815 if (cMaxDepth >= 1)
3816 {
3817 int rc2 = pgmR3DumpHierarchyHCPaePD(pVM, Pdpe.u & X86_PDPE_PG_MASK, u64Address + ((uint64_t)i << X86_PDPT_SHIFT),
3818 cr4, fLongMode, cMaxDepth - 1, pHlp);
3819 if (rc2 < rc && RT_SUCCESS(rc))
3820 rc = rc2;
3821 }
3822 }
3823 }
3824 return rc;
3825}
3826
3827
3828/**
3829 * Dumps a 32-bit shadow page table.
3830 *
3831 * @returns VBox status code (VINF_SUCCESS).
3832 * @param pVM The VM handle.
3833 * @param HCPhys The physical address of the table.
3834 * @param cr4 The CR4, PSE is currently used.
3835 * @param cMaxDepth The maxium depth.
3836 * @param pHlp Pointer to the output functions.
3837 */
3838static int pgmR3DumpHierarchyHcPaePML4(PVM pVM, RTHCPHYS HCPhys, uint32_t cr4, unsigned cMaxDepth, PCDBGFINFOHLP pHlp)
3839{
3840 PX86PML4 pPML4 = (PX86PML4)MMPagePhys2Page(pVM, HCPhys);
3841 if (!pPML4)
3842 {
3843 pHlp->pfnPrintf(pHlp, "Page map level 4 at HCPhys=%RHp was not found in the page pool!\n", HCPhys);
3844 return VERR_INVALID_PARAMETER;
3845 }
3846
3847 int rc = VINF_SUCCESS;
3848 for (unsigned i = 0; i < RT_ELEMENTS(pPML4->a); i++)
3849 {
3850 X86PML4E Pml4e = pPML4->a[i];
3851 if (Pml4e.n.u1Present)
3852 {
3853 uint64_t u64Address = ((uint64_t)i << X86_PML4_SHIFT) | (((uint64_t)i >> (X86_PML4_SHIFT - X86_PDPT_SHIFT - 1)) * 0xffff000000000000ULL);
3854 pHlp->pfnPrintf(pHlp, /*P R S A D G WT CD AT NX 4M a p ? */
3855 "%016llx 0 | P %c %c %c %c %c %s %s %s %s .. %c%c%c %016llx\n",
3856 u64Address,
3857 Pml4e.n.u1Write ? 'W' : 'R',
3858 Pml4e.n.u1User ? 'U' : 'S',
3859 Pml4e.n.u1Accessed ? 'A' : '-',
3860 Pml4e.n.u3Reserved & 1? '?' : '.', /* ignored */
3861 Pml4e.n.u3Reserved & 4? '!' : '.', /* mbz */
3862 Pml4e.n.u1WriteThru ? "WT" : "--",
3863 Pml4e.n.u1CacheDisable? "CD" : "--",
3864 Pml4e.n.u3Reserved & 2? "!" : "..",/* mbz */
3865 Pml4e.n.u1NoExecute ? "NX" : "--",
3866 Pml4e.u & RT_BIT(9) ? '1' : '0',
3867 Pml4e.u & PGM_PLXFLAGS_PERMANENT ? 'p' : '-',
3868 Pml4e.u & RT_BIT(11) ? '1' : '0',
3869 Pml4e.u & X86_PML4E_PG_MASK);
3870
3871 if (cMaxDepth >= 1)
3872 {
3873 int rc2 = pgmR3DumpHierarchyHCPaePDPT(pVM, Pml4e.u & X86_PML4E_PG_MASK, u64Address, cr4, true, cMaxDepth - 1, pHlp);
3874 if (rc2 < rc && RT_SUCCESS(rc))
3875 rc = rc2;
3876 }
3877 }
3878 }
3879 return rc;
3880}
3881
3882
3883/**
3884 * Dumps a 32-bit shadow page table.
3885 *
3886 * @returns VBox status code (VINF_SUCCESS).
3887 * @param pVM The VM handle.
3888 * @param pPT Pointer to the page table.
3889 * @param u32Address The virtual address this table starts at.
3890 * @param pHlp Pointer to the output functions.
3891 */
3892int pgmR3DumpHierarchyHC32BitPT(PVM pVM, PX86PT pPT, uint32_t u32Address, PCDBGFINFOHLP pHlp)
3893{
3894 for (unsigned i = 0; i < RT_ELEMENTS(pPT->a); i++)
3895 {
3896 X86PTE Pte = pPT->a[i];
3897 if (Pte.n.u1Present)
3898 {
3899 pHlp->pfnPrintf(pHlp, /*P R S A D G WT CD AT NX 4M a m d */
3900 "%08x 1 | P %c %c %c %c %c %s %s %s .. 4K %c%c%c %08x\n",
3901 u32Address + (i << X86_PT_SHIFT),
3902 Pte.n.u1Write ? 'W' : 'R',
3903 Pte.n.u1User ? 'U' : 'S',
3904 Pte.n.u1Accessed ? 'A' : '-',
3905 Pte.n.u1Dirty ? 'D' : '-',
3906 Pte.n.u1Global ? 'G' : '-',
3907 Pte.n.u1WriteThru ? "WT" : "--",
3908 Pte.n.u1CacheDisable? "CD" : "--",
3909 Pte.n.u1PAT ? "AT" : "--",
3910 Pte.u & PGM_PTFLAGS_TRACK_DIRTY ? 'd' : '-',
3911 Pte.u & RT_BIT(10) ? '1' : '0',
3912 Pte.u & PGM_PTFLAGS_CSAM_VALIDATED ? 'v' : '-',
3913 Pte.u & X86_PDE_PG_MASK);
3914 }
3915 }
3916 return VINF_SUCCESS;
3917}
3918
3919
3920/**
3921 * Dumps a 32-bit shadow page directory and page tables.
3922 *
3923 * @returns VBox status code (VINF_SUCCESS).
3924 * @param pVM The VM handle.
3925 * @param cr3 The root of the hierarchy.
3926 * @param cr4 The CR4, PSE is currently used.
3927 * @param cMaxDepth How deep into the hierarchy the dumper should go.
3928 * @param pHlp Pointer to the output functions.
3929 */
3930int pgmR3DumpHierarchyHC32BitPD(PVM pVM, uint32_t cr3, uint32_t cr4, unsigned cMaxDepth, PCDBGFINFOHLP pHlp)
3931{
3932 PX86PD pPD = (PX86PD)MMPagePhys2Page(pVM, cr3 & X86_CR3_PAGE_MASK);
3933 if (!pPD)
3934 {
3935 pHlp->pfnPrintf(pHlp, "Page directory at %#x was not found in the page pool!\n", cr3 & X86_CR3_PAGE_MASK);
3936 return VERR_INVALID_PARAMETER;
3937 }
3938
3939 int rc = VINF_SUCCESS;
3940 for (unsigned i = 0; i < RT_ELEMENTS(pPD->a); i++)
3941 {
3942 X86PDE Pde = pPD->a[i];
3943 if (Pde.n.u1Present)
3944 {
3945 const uint32_t u32Address = i << X86_PD_SHIFT;
3946 if ((cr4 & X86_CR4_PSE) && Pde.b.u1Size)
3947 pHlp->pfnPrintf(pHlp, /*P R S A D G WT CD AT NX 4M a m d */
3948 "%08x 0 | P %c %c %c %c %c %s %s %s .. 4M %c%c%c %08x\n",
3949 u32Address,
3950 Pde.b.u1Write ? 'W' : 'R',
3951 Pde.b.u1User ? 'U' : 'S',
3952 Pde.b.u1Accessed ? 'A' : '-',
3953 Pde.b.u1Dirty ? 'D' : '-',
3954 Pde.b.u1Global ? 'G' : '-',
3955 Pde.b.u1WriteThru ? "WT" : "--",
3956 Pde.b.u1CacheDisable? "CD" : "--",
3957 Pde.b.u1PAT ? "AT" : "--",
3958 Pde.u & RT_BIT_64(9) ? '1' : '0',
3959 Pde.u & PGM_PDFLAGS_MAPPING ? 'm' : '-',
3960 Pde.u & PGM_PDFLAGS_TRACK_DIRTY ? 'd' : '-',
3961 Pde.u & X86_PDE4M_PG_MASK);
3962 else
3963 {
3964 pHlp->pfnPrintf(pHlp, /*P R S A D G WT CD AT NX 4M a m d */
3965 "%08x 0 | P %c %c %c %c %c %s %s .. .. 4K %c%c%c %08x\n",
3966 u32Address,
3967 Pde.n.u1Write ? 'W' : 'R',
3968 Pde.n.u1User ? 'U' : 'S',
3969 Pde.n.u1Accessed ? 'A' : '-',
3970 Pde.n.u1Reserved0 ? '?' : '.', /* ignored */
3971 Pde.n.u1Reserved1 ? '?' : '.', /* ignored */
3972 Pde.n.u1WriteThru ? "WT" : "--",
3973 Pde.n.u1CacheDisable? "CD" : "--",
3974 Pde.u & RT_BIT_64(9) ? '1' : '0',
3975 Pde.u & PGM_PDFLAGS_MAPPING ? 'm' : '-',
3976 Pde.u & PGM_PDFLAGS_TRACK_DIRTY ? 'd' : '-',
3977 Pde.u & X86_PDE_PG_MASK);
3978 if (cMaxDepth >= 1)
3979 {
3980 /** @todo what about using the page pool for mapping PTs? */
3981 RTHCPHYS HCPhys = Pde.u & X86_PDE_PG_MASK;
3982 PX86PT pPT = NULL;
3983 if (!(Pde.u & PGM_PDFLAGS_MAPPING))
3984 pPT = (PX86PT)MMPagePhys2Page(pVM, HCPhys);
3985 else
3986 {
3987 for (PPGMMAPPING pMap = pVM->pgm.s.pMappingsR3; pMap; pMap = pMap->pNextR3)
3988 if (u32Address - pMap->GCPtr < pMap->cb)
3989 {
3990 int iPDE = (u32Address - pMap->GCPtr) >> X86_PD_SHIFT;
3991 if (pMap->aPTs[iPDE].HCPhysPT != HCPhys)
3992 pHlp->pfnPrintf(pHlp, "%08x error! Mapping error! PT %d has HCPhysPT=%RHp not %RHp is in the PD.\n",
3993 u32Address, iPDE, pMap->aPTs[iPDE].HCPhysPT, HCPhys);
3994 pPT = pMap->aPTs[iPDE].pPTR3;
3995 }
3996 }
3997 int rc2 = VERR_INVALID_PARAMETER;
3998 if (pPT)
3999 rc2 = pgmR3DumpHierarchyHC32BitPT(pVM, pPT, u32Address, pHlp);
4000 else
4001 pHlp->pfnPrintf(pHlp, "%08x error! Page table at %#x was not found in the page pool!\n", u32Address, HCPhys);
4002 if (rc2 < rc && RT_SUCCESS(rc))
4003 rc = rc2;
4004 }
4005 }
4006 }
4007 }
4008
4009 return rc;
4010}
4011
4012
4013/**
4014 * Dumps a 32-bit shadow page table.
4015 *
4016 * @returns VBox status code (VINF_SUCCESS).
4017 * @param pVM The VM handle.
4018 * @param pPT Pointer to the page table.
4019 * @param u32Address The virtual address this table starts at.
4020 * @param PhysSearch Address to search for.
4021 */
4022int pgmR3DumpHierarchyGC32BitPT(PVM pVM, PX86PT pPT, uint32_t u32Address, RTGCPHYS PhysSearch)
4023{
4024 for (unsigned i = 0; i < RT_ELEMENTS(pPT->a); i++)
4025 {
4026 X86PTE Pte = pPT->a[i];
4027 if (Pte.n.u1Present)
4028 {
4029 Log(( /*P R S A D G WT CD AT NX 4M a m d */
4030 "%08x 1 | P %c %c %c %c %c %s %s %s .. 4K %c%c%c %08x\n",
4031 u32Address + (i << X86_PT_SHIFT),
4032 Pte.n.u1Write ? 'W' : 'R',
4033 Pte.n.u1User ? 'U' : 'S',
4034 Pte.n.u1Accessed ? 'A' : '-',
4035 Pte.n.u1Dirty ? 'D' : '-',
4036 Pte.n.u1Global ? 'G' : '-',
4037 Pte.n.u1WriteThru ? "WT" : "--",
4038 Pte.n.u1CacheDisable? "CD" : "--",
4039 Pte.n.u1PAT ? "AT" : "--",
4040 Pte.u & PGM_PTFLAGS_TRACK_DIRTY ? 'd' : '-',
4041 Pte.u & RT_BIT(10) ? '1' : '0',
4042 Pte.u & PGM_PTFLAGS_CSAM_VALIDATED ? 'v' : '-',
4043 Pte.u & X86_PDE_PG_MASK));
4044
4045 if ((Pte.u & X86_PDE_PG_MASK) == PhysSearch)
4046 {
4047 uint64_t fPageShw = 0;
4048 RTHCPHYS pPhysHC = 0;
4049
4050 PGMShwGetPage(pVM, (RTGCPTR)(u32Address + (i << X86_PT_SHIFT)), &fPageShw, &pPhysHC);
4051 Log(("Found %RGp at %RGv -> flags=%llx\n", PhysSearch, (RTGCPTR)(u32Address + (i << X86_PT_SHIFT)), fPageShw));
4052 }
4053 }
4054 }
4055 return VINF_SUCCESS;
4056}
4057
4058
4059/**
4060 * Dumps a 32-bit guest page directory and page tables.
4061 *
4062 * @returns VBox status code (VINF_SUCCESS).
4063 * @param pVM The VM handle.
4064 * @param cr3 The root of the hierarchy.
4065 * @param cr4 The CR4, PSE is currently used.
4066 * @param PhysSearch Address to search for.
4067 */
4068VMMR3DECL(int) PGMR3DumpHierarchyGC(PVM pVM, uint64_t cr3, uint64_t cr4, RTGCPHYS PhysSearch)
4069{
4070 bool fLongMode = false;
4071 const unsigned cch = fLongMode ? 16 : 8; NOREF(cch);
4072 PX86PD pPD = 0;
4073
4074 int rc = PGM_GCPHYS_2_PTR(pVM, cr3 & X86_CR3_PAGE_MASK, &pPD);
4075 if (RT_FAILURE(rc) || !pPD)
4076 {
4077 Log(("Page directory at %#x was not found in the page pool!\n", cr3 & X86_CR3_PAGE_MASK));
4078 return VERR_INVALID_PARAMETER;
4079 }
4080
4081 Log(("cr3=%08x cr4=%08x%s\n"
4082 "%-*s P - Present\n"
4083 "%-*s | R/W - Read (0) / Write (1)\n"
4084 "%-*s | | U/S - User (1) / Supervisor (0)\n"
4085 "%-*s | | | A - Accessed\n"
4086 "%-*s | | | | D - Dirty\n"
4087 "%-*s | | | | | G - Global\n"
4088 "%-*s | | | | | | WT - Write thru\n"
4089 "%-*s | | | | | | | CD - Cache disable\n"
4090 "%-*s | | | | | | | | AT - Attribute table (PAT)\n"
4091 "%-*s | | | | | | | | | NX - No execute (K8)\n"
4092 "%-*s | | | | | | | | | | 4K/4M/2M - Page size.\n"
4093 "%-*s | | | | | | | | | | | AVL - a=allocated; m=mapping; d=track dirty;\n"
4094 "%-*s | | | | | | | | | | | | p=permanent; v=validated;\n"
4095 "%-*s Level | | | | | | | | | | | | Page\n"
4096 /* xxxx n **** P R S A D G WT CD AT NX 4M AVL xxxxxxxxxxxxx
4097 - W U - - - -- -- -- -- -- 010 */
4098 , cr3, cr4, fLongMode ? " Long Mode" : "",
4099 cch, "", cch, "", cch, "", cch, "", cch, "", cch, "", cch, "",
4100 cch, "", cch, "", cch, "", cch, "", cch, "", cch, "", cch, "Address"));
4101
4102 for (unsigned i = 0; i < RT_ELEMENTS(pPD->a); i++)
4103 {
4104 X86PDE Pde = pPD->a[i];
4105 if (Pde.n.u1Present)
4106 {
4107 const uint32_t u32Address = i << X86_PD_SHIFT;
4108
4109 if ((cr4 & X86_CR4_PSE) && Pde.b.u1Size)
4110 Log(( /*P R S A D G WT CD AT NX 4M a m d */
4111 "%08x 0 | P %c %c %c %c %c %s %s %s .. 4M %c%c%c %08x\n",
4112 u32Address,
4113 Pde.b.u1Write ? 'W' : 'R',
4114 Pde.b.u1User ? 'U' : 'S',
4115 Pde.b.u1Accessed ? 'A' : '-',
4116 Pde.b.u1Dirty ? 'D' : '-',
4117 Pde.b.u1Global ? 'G' : '-',
4118 Pde.b.u1WriteThru ? "WT" : "--",
4119 Pde.b.u1CacheDisable? "CD" : "--",
4120 Pde.b.u1PAT ? "AT" : "--",
4121 Pde.u & RT_BIT(9) ? '1' : '0',
4122 Pde.u & RT_BIT(10) ? '1' : '0',
4123 Pde.u & RT_BIT(11) ? '1' : '0',
4124 pgmGstGet4MBPhysPage(&pVM->pgm.s, Pde)));
4125 /** @todo PhysSearch */
4126 else
4127 {
4128 Log(( /*P R S A D G WT CD AT NX 4M a m d */
4129 "%08x 0 | P %c %c %c %c %c %s %s .. .. 4K %c%c%c %08x\n",
4130 u32Address,
4131 Pde.n.u1Write ? 'W' : 'R',
4132 Pde.n.u1User ? 'U' : 'S',
4133 Pde.n.u1Accessed ? 'A' : '-',
4134 Pde.n.u1Reserved0 ? '?' : '.', /* ignored */
4135 Pde.n.u1Reserved1 ? '?' : '.', /* ignored */
4136 Pde.n.u1WriteThru ? "WT" : "--",
4137 Pde.n.u1CacheDisable? "CD" : "--",
4138 Pde.u & RT_BIT(9) ? '1' : '0',
4139 Pde.u & RT_BIT(10) ? '1' : '0',
4140 Pde.u & RT_BIT(11) ? '1' : '0',
4141 Pde.u & X86_PDE_PG_MASK));
4142 ////if (cMaxDepth >= 1)
4143 {
4144 /** @todo what about using the page pool for mapping PTs? */
4145 RTGCPHYS GCPhys = Pde.u & X86_PDE_PG_MASK;
4146 PX86PT pPT = NULL;
4147
4148 rc = PGM_GCPHYS_2_PTR(pVM, GCPhys, &pPT);
4149
4150 int rc2 = VERR_INVALID_PARAMETER;
4151 if (pPT)
4152 rc2 = pgmR3DumpHierarchyGC32BitPT(pVM, pPT, u32Address, PhysSearch);
4153 else
4154 Log(("%08x error! Page table at %#x was not found in the page pool!\n", u32Address, GCPhys));
4155 if (rc2 < rc && RT_SUCCESS(rc))
4156 rc = rc2;
4157 }
4158 }
4159 }
4160 }
4161
4162 return rc;
4163}
4164
4165
4166/**
4167 * Dumps a page table hierarchy use only physical addresses and cr4/lm flags.
4168 *
4169 * @returns VBox status code (VINF_SUCCESS).
4170 * @param pVM The VM handle.
4171 * @param cr3 The root of the hierarchy.
4172 * @param cr4 The cr4, only PAE and PSE is currently used.
4173 * @param fLongMode Set if long mode, false if not long mode.
4174 * @param cMaxDepth Number of levels to dump.
4175 * @param pHlp Pointer to the output functions.
4176 */
4177VMMR3DECL(int) PGMR3DumpHierarchyHC(PVM pVM, uint64_t cr3, uint64_t cr4, bool fLongMode, unsigned cMaxDepth, PCDBGFINFOHLP pHlp)
4178{
4179 if (!pHlp)
4180 pHlp = DBGFR3InfoLogHlp();
4181 if (!cMaxDepth)
4182 return VINF_SUCCESS;
4183 const unsigned cch = fLongMode ? 16 : 8;
4184 pHlp->pfnPrintf(pHlp,
4185 "cr3=%08x cr4=%08x%s\n"
4186 "%-*s P - Present\n"
4187 "%-*s | R/W - Read (0) / Write (1)\n"
4188 "%-*s | | U/S - User (1) / Supervisor (0)\n"
4189 "%-*s | | | A - Accessed\n"
4190 "%-*s | | | | D - Dirty\n"
4191 "%-*s | | | | | G - Global\n"
4192 "%-*s | | | | | | WT - Write thru\n"
4193 "%-*s | | | | | | | CD - Cache disable\n"
4194 "%-*s | | | | | | | | AT - Attribute table (PAT)\n"
4195 "%-*s | | | | | | | | | NX - No execute (K8)\n"
4196 "%-*s | | | | | | | | | | 4K/4M/2M - Page size.\n"
4197 "%-*s | | | | | | | | | | | AVL - a=allocated; m=mapping; d=track dirty;\n"
4198 "%-*s | | | | | | | | | | | | p=permanent; v=validated;\n"
4199 "%-*s Level | | | | | | | | | | | | Page\n"
4200 /* xxxx n **** P R S A D G WT CD AT NX 4M AVL xxxxxxxxxxxxx
4201 - W U - - - -- -- -- -- -- 010 */
4202 , cr3, cr4, fLongMode ? " Long Mode" : "",
4203 cch, "", cch, "", cch, "", cch, "", cch, "", cch, "", cch, "",
4204 cch, "", cch, "", cch, "", cch, "", cch, "", cch, "", cch, "Address");
4205 if (cr4 & X86_CR4_PAE)
4206 {
4207 if (fLongMode)
4208 return pgmR3DumpHierarchyHcPaePML4(pVM, cr3 & X86_CR3_PAGE_MASK, cr4, cMaxDepth, pHlp);
4209 return pgmR3DumpHierarchyHCPaePDPT(pVM, cr3 & X86_CR3_PAE_PAGE_MASK, 0, cr4, false, cMaxDepth, pHlp);
4210 }
4211 return pgmR3DumpHierarchyHC32BitPD(pVM, cr3 & X86_CR3_PAGE_MASK, cr4, cMaxDepth, pHlp);
4212}
4213
4214#ifdef VBOX_WITH_DEBUGGER
4215
4216/**
4217 * The '.pgmram' command.
4218 *
4219 * @returns VBox status.
4220 * @param pCmd Pointer to the command descriptor (as registered).
4221 * @param pCmdHlp Pointer to command helper functions.
4222 * @param pVM Pointer to the current VM (if any).
4223 * @param paArgs Pointer to (readonly) array of arguments.
4224 * @param cArgs Number of arguments in the array.
4225 */
4226static DECLCALLBACK(int) pgmR3CmdRam(PCDBGCCMD pCmd, PDBGCCMDHLP pCmdHlp, PVM pVM, PCDBGCVAR paArgs, unsigned cArgs, PDBGCVAR pResult)
4227{
4228 /*
4229 * Validate input.
4230 */
4231 if (!pVM)
4232 return pCmdHlp->pfnPrintf(pCmdHlp, NULL, "error: The command requires a VM to be selected.\n");
4233 if (!pVM->pgm.s.pRamRangesRC)
4234 return pCmdHlp->pfnPrintf(pCmdHlp, NULL, "Sorry, no Ram is registered.\n");
4235
4236 /*
4237 * Dump the ranges.
4238 */
4239 int rc = pCmdHlp->pfnPrintf(pCmdHlp, NULL, "From - To (incl) pvHC\n");
4240 PPGMRAMRANGE pRam;
4241 for (pRam = pVM->pgm.s.pRamRangesR3; pRam; pRam = pRam->pNextR3)
4242 {
4243 rc = pCmdHlp->pfnPrintf(pCmdHlp, NULL,
4244 "%RGp - %RGp %p\n",
4245 pRam->GCPhys, pRam->GCPhysLast, pRam->pvR3);
4246 if (RT_FAILURE(rc))
4247 return rc;
4248 }
4249
4250 return VINF_SUCCESS;
4251}
4252
4253
4254/**
4255 * The '.pgmmap' command.
4256 *
4257 * @returns VBox status.
4258 * @param pCmd Pointer to the command descriptor (as registered).
4259 * @param pCmdHlp Pointer to command helper functions.
4260 * @param pVM Pointer to the current VM (if any).
4261 * @param paArgs Pointer to (readonly) array of arguments.
4262 * @param cArgs Number of arguments in the array.
4263 */
4264static DECLCALLBACK(int) pgmR3CmdMap(PCDBGCCMD pCmd, PDBGCCMDHLP pCmdHlp, PVM pVM, PCDBGCVAR paArgs, unsigned cArgs, PDBGCVAR pResult)
4265{
4266 /*
4267 * Validate input.
4268 */
4269 if (!pVM)
4270 return pCmdHlp->pfnPrintf(pCmdHlp, NULL, "error: The command requires a VM to be selected.\n");
4271 if (!pVM->pgm.s.pMappingsR3)
4272 return pCmdHlp->pfnPrintf(pCmdHlp, NULL, "Sorry, no mappings are registered.\n");
4273
4274 /*
4275 * Print message about the fixedness of the mappings.
4276 */
4277 int rc = pCmdHlp->pfnPrintf(pCmdHlp, NULL, pVM->pgm.s.fMappingsFixed ? "The mappings are FIXED.\n" : "The mappings are FLOATING.\n");
4278 if (RT_FAILURE(rc))
4279 return rc;
4280
4281 /*
4282 * Dump the ranges.
4283 */
4284 PPGMMAPPING pCur;
4285 for (pCur = pVM->pgm.s.pMappingsR3; pCur; pCur = pCur->pNextR3)
4286 {
4287 rc = pCmdHlp->pfnPrintf(pCmdHlp, NULL,
4288 "%08x - %08x %s\n",
4289 pCur->GCPtr, pCur->GCPtrLast, pCur->pszDesc);
4290 if (RT_FAILURE(rc))
4291 return rc;
4292 }
4293
4294 return VINF_SUCCESS;
4295}
4296
4297
4298/**
4299 * The '.pgmsync' command.
4300 *
4301 * @returns VBox status.
4302 * @param pCmd Pointer to the command descriptor (as registered).
4303 * @param pCmdHlp Pointer to command helper functions.
4304 * @param pVM Pointer to the current VM (if any).
4305 * @param paArgs Pointer to (readonly) array of arguments.
4306 * @param cArgs Number of arguments in the array.
4307 */
4308static DECLCALLBACK(int) pgmR3CmdSync(PCDBGCCMD pCmd, PDBGCCMDHLP pCmdHlp, PVM pVM, PCDBGCVAR paArgs, unsigned cArgs, PDBGCVAR pResult)
4309{
4310 /*
4311 * Validate input.
4312 */
4313 if (!pVM)
4314 return pCmdHlp->pfnPrintf(pCmdHlp, NULL, "error: The command requires a VM to be selected.\n");
4315
4316 /*
4317 * Force page directory sync.
4318 */
4319 VM_FF_SET(pVM, VM_FF_PGM_SYNC_CR3);
4320
4321 int rc = pCmdHlp->pfnPrintf(pCmdHlp, NULL, "Forcing page directory sync.\n");
4322 if (RT_FAILURE(rc))
4323 return rc;
4324
4325 return VINF_SUCCESS;
4326}
4327
4328
4329#ifdef VBOX_STRICT
4330/**
4331 * The '.pgmassertcr3' command.
4332 *
4333 * @returns VBox status.
4334 * @param pCmd Pointer to the command descriptor (as registered).
4335 * @param pCmdHlp Pointer to command helper functions.
4336 * @param pVM Pointer to the current VM (if any).
4337 * @param paArgs Pointer to (readonly) array of arguments.
4338 * @param cArgs Number of arguments in the array.
4339 */
4340static DECLCALLBACK(int) pgmR3CmdAssertCR3(PCDBGCCMD pCmd, PDBGCCMDHLP pCmdHlp, PVM pVM, PCDBGCVAR paArgs, unsigned cArgs, PDBGCVAR pResult)
4341{
4342 /*
4343 * Validate input.
4344 */
4345 if (!pVM)
4346 return pCmdHlp->pfnPrintf(pCmdHlp, NULL, "error: The command requires a VM to be selected.\n");
4347
4348 int rc = pCmdHlp->pfnPrintf(pCmdHlp, NULL, "Checking shadow CR3 page tables for consistency.\n");
4349 if (RT_FAILURE(rc))
4350 return rc;
4351
4352 PGMAssertCR3(pVM, CPUMGetGuestCR3(pVM), CPUMGetGuestCR4(pVM));
4353
4354 return VINF_SUCCESS;
4355}
4356#endif /* VBOX_STRICT */
4357
4358
4359/**
4360 * The '.pgmsyncalways' command.
4361 *
4362 * @returns VBox status.
4363 * @param pCmd Pointer to the command descriptor (as registered).
4364 * @param pCmdHlp Pointer to command helper functions.
4365 * @param pVM Pointer to the current VM (if any).
4366 * @param paArgs Pointer to (readonly) array of arguments.
4367 * @param cArgs Number of arguments in the array.
4368 */
4369static DECLCALLBACK(int) pgmR3CmdSyncAlways(PCDBGCCMD pCmd, PDBGCCMDHLP pCmdHlp, PVM pVM, PCDBGCVAR paArgs, unsigned cArgs, PDBGCVAR pResult)
4370{
4371 /*
4372 * Validate input.
4373 */
4374 if (!pVM)
4375 return pCmdHlp->pfnPrintf(pCmdHlp, NULL, "error: The command requires a VM to be selected.\n");
4376
4377 /*
4378 * Force page directory sync.
4379 */
4380 if (pVM->pgm.s.fSyncFlags & PGM_SYNC_ALWAYS)
4381 {
4382 ASMAtomicAndU32(&pVM->pgm.s.fSyncFlags, ~PGM_SYNC_ALWAYS);
4383 return pCmdHlp->pfnPrintf(pCmdHlp, NULL, "Disabled permanent forced page directory syncing.\n");
4384 }
4385 else
4386 {
4387 ASMAtomicOrU32(&pVM->pgm.s.fSyncFlags, PGM_SYNC_ALWAYS);
4388 VM_FF_SET(pVM, VM_FF_PGM_SYNC_CR3);
4389 return pCmdHlp->pfnPrintf(pCmdHlp, NULL, "Enabled permanent forced page directory syncing.\n");
4390 }
4391}
4392
4393#endif /* VBOX_WITH_DEBUGGER */
4394
4395/**
4396 * pvUser argument of the pgmR3CheckIntegrity*Node callbacks.
4397 */
4398typedef struct PGMCHECKINTARGS
4399{
4400 bool fLeftToRight; /**< true: left-to-right; false: right-to-left. */
4401 PPGMPHYSHANDLER pPrevPhys;
4402 PPGMVIRTHANDLER pPrevVirt;
4403 PPGMPHYS2VIRTHANDLER pPrevPhys2Virt;
4404 PVM pVM;
4405} PGMCHECKINTARGS, *PPGMCHECKINTARGS;
4406
4407/**
4408 * Validate a node in the physical handler tree.
4409 *
4410 * @returns 0 on if ok, other wise 1.
4411 * @param pNode The handler node.
4412 * @param pvUser pVM.
4413 */
4414static DECLCALLBACK(int) pgmR3CheckIntegrityPhysHandlerNode(PAVLROGCPHYSNODECORE pNode, void *pvUser)
4415{
4416 PPGMCHECKINTARGS pArgs = (PPGMCHECKINTARGS)pvUser;
4417 PPGMPHYSHANDLER pCur = (PPGMPHYSHANDLER)pNode;
4418 AssertReleaseReturn(!((uintptr_t)pCur & 7), 1);
4419 AssertReleaseMsg(pCur->Core.Key <= pCur->Core.KeyLast,("pCur=%p %RGp-%RGp %s\n", pCur, pCur->Core.Key, pCur->Core.KeyLast, pCur->pszDesc));
4420 AssertReleaseMsg( !pArgs->pPrevPhys
4421 || (pArgs->fLeftToRight ? pArgs->pPrevPhys->Core.KeyLast < pCur->Core.Key : pArgs->pPrevPhys->Core.KeyLast > pCur->Core.Key),
4422 ("pPrevPhys=%p %RGp-%RGp %s\n"
4423 " pCur=%p %RGp-%RGp %s\n",
4424 pArgs->pPrevPhys, pArgs->pPrevPhys->Core.Key, pArgs->pPrevPhys->Core.KeyLast, pArgs->pPrevPhys->pszDesc,
4425 pCur, pCur->Core.Key, pCur->Core.KeyLast, pCur->pszDesc));
4426 pArgs->pPrevPhys = pCur;
4427 return 0;
4428}
4429
4430
4431/**
4432 * Validate a node in the virtual handler tree.
4433 *
4434 * @returns 0 on if ok, other wise 1.
4435 * @param pNode The handler node.
4436 * @param pvUser pVM.
4437 */
4438static DECLCALLBACK(int) pgmR3CheckIntegrityVirtHandlerNode(PAVLROGCPTRNODECORE pNode, void *pvUser)
4439{
4440 PPGMCHECKINTARGS pArgs = (PPGMCHECKINTARGS)pvUser;
4441 PPGMVIRTHANDLER pCur = (PPGMVIRTHANDLER)pNode;
4442 AssertReleaseReturn(!((uintptr_t)pCur & 7), 1);
4443 AssertReleaseMsg(pCur->Core.Key <= pCur->Core.KeyLast,("pCur=%p %RGv-%RGv %s\n", pCur, pCur->Core.Key, pCur->Core.KeyLast, pCur->pszDesc));
4444 AssertReleaseMsg( !pArgs->pPrevVirt
4445 || (pArgs->fLeftToRight ? pArgs->pPrevVirt->Core.KeyLast < pCur->Core.Key : pArgs->pPrevVirt->Core.KeyLast > pCur->Core.Key),
4446 ("pPrevVirt=%p %RGv-%RGv %s\n"
4447 " pCur=%p %RGv-%RGv %s\n",
4448 pArgs->pPrevVirt, pArgs->pPrevVirt->Core.Key, pArgs->pPrevVirt->Core.KeyLast, pArgs->pPrevVirt->pszDesc,
4449 pCur, pCur->Core.Key, pCur->Core.KeyLast, pCur->pszDesc));
4450 for (unsigned iPage = 0; iPage < pCur->cPages; iPage++)
4451 {
4452 AssertReleaseMsg(pCur->aPhysToVirt[iPage].offVirtHandler == -RT_OFFSETOF(PGMVIRTHANDLER, aPhysToVirt[iPage]),
4453 ("pCur=%p %RGv-%RGv %s\n"
4454 "iPage=%d offVirtHandle=%#x expected %#x\n",
4455 pCur, pCur->Core.Key, pCur->Core.KeyLast, pCur->pszDesc,
4456 iPage, pCur->aPhysToVirt[iPage].offVirtHandler, -RT_OFFSETOF(PGMVIRTHANDLER, aPhysToVirt[iPage])));
4457 }
4458 pArgs->pPrevVirt = pCur;
4459 return 0;
4460}
4461
4462
4463/**
4464 * Validate a node in the virtual handler tree.
4465 *
4466 * @returns 0 on if ok, other wise 1.
4467 * @param pNode The handler node.
4468 * @param pvUser pVM.
4469 */
4470static DECLCALLBACK(int) pgmR3CheckIntegrityPhysToVirtHandlerNode(PAVLROGCPHYSNODECORE pNode, void *pvUser)
4471{
4472 PPGMCHECKINTARGS pArgs = (PPGMCHECKINTARGS)pvUser;
4473 PPGMPHYS2VIRTHANDLER pCur = (PPGMPHYS2VIRTHANDLER)pNode;
4474 AssertReleaseMsgReturn(!((uintptr_t)pCur & 3), ("\n"), 1);
4475 AssertReleaseMsgReturn(!(pCur->offVirtHandler & 3), ("\n"), 1);
4476 AssertReleaseMsg(pCur->Core.Key <= pCur->Core.KeyLast,("pCur=%p %RGp-%RGp\n", pCur, pCur->Core.Key, pCur->Core.KeyLast));
4477 AssertReleaseMsg( !pArgs->pPrevPhys2Virt
4478 || (pArgs->fLeftToRight ? pArgs->pPrevPhys2Virt->Core.KeyLast < pCur->Core.Key : pArgs->pPrevPhys2Virt->Core.KeyLast > pCur->Core.Key),
4479 ("pPrevPhys2Virt=%p %RGp-%RGp\n"
4480 " pCur=%p %RGp-%RGp\n",
4481 pArgs->pPrevPhys2Virt, pArgs->pPrevPhys2Virt->Core.Key, pArgs->pPrevPhys2Virt->Core.KeyLast,
4482 pCur, pCur->Core.Key, pCur->Core.KeyLast));
4483 AssertReleaseMsg( !pArgs->pPrevPhys2Virt
4484 || (pArgs->fLeftToRight ? pArgs->pPrevPhys2Virt->Core.KeyLast < pCur->Core.Key : pArgs->pPrevPhys2Virt->Core.KeyLast > pCur->Core.Key),
4485 ("pPrevPhys2Virt=%p %RGp-%RGp\n"
4486 " pCur=%p %RGp-%RGp\n",
4487 pArgs->pPrevPhys2Virt, pArgs->pPrevPhys2Virt->Core.Key, pArgs->pPrevPhys2Virt->Core.KeyLast,
4488 pCur, pCur->Core.Key, pCur->Core.KeyLast));
4489 AssertReleaseMsg((pCur->offNextAlias & (PGMPHYS2VIRTHANDLER_IN_TREE | PGMPHYS2VIRTHANDLER_IS_HEAD)) == (PGMPHYS2VIRTHANDLER_IN_TREE | PGMPHYS2VIRTHANDLER_IS_HEAD),
4490 ("pCur=%p:{.Core.Key=%RGp, .Core.KeyLast=%RGp, .offVirtHandler=%#RX32, .offNextAlias=%#RX32}\n",
4491 pCur, pCur->Core.Key, pCur->Core.KeyLast, pCur->offVirtHandler, pCur->offNextAlias));
4492 if (pCur->offNextAlias & PGMPHYS2VIRTHANDLER_OFF_MASK)
4493 {
4494 PPGMPHYS2VIRTHANDLER pCur2 = pCur;
4495 for (;;)
4496 {
4497 pCur2 = (PPGMPHYS2VIRTHANDLER)((intptr_t)pCur + (pCur->offNextAlias & PGMPHYS2VIRTHANDLER_OFF_MASK));
4498 AssertReleaseMsg(pCur2 != pCur,
4499 (" pCur=%p:{.Core.Key=%RGp, .Core.KeyLast=%RGp, .offVirtHandler=%#RX32, .offNextAlias=%#RX32}\n",
4500 pCur, pCur->Core.Key, pCur->Core.KeyLast, pCur->offVirtHandler, pCur->offNextAlias));
4501 AssertReleaseMsg((pCur2->offNextAlias & (PGMPHYS2VIRTHANDLER_IN_TREE | PGMPHYS2VIRTHANDLER_IS_HEAD)) == PGMPHYS2VIRTHANDLER_IN_TREE,
4502 (" pCur=%p:{.Core.Key=%RGp, .Core.KeyLast=%RGp, .offVirtHandler=%#RX32, .offNextAlias=%#RX32}\n"
4503 "pCur2=%p:{.Core.Key=%RGp, .Core.KeyLast=%RGp, .offVirtHandler=%#RX32, .offNextAlias=%#RX32}\n",
4504 pCur, pCur->Core.Key, pCur->Core.KeyLast, pCur->offVirtHandler, pCur->offNextAlias,
4505 pCur2, pCur2->Core.Key, pCur2->Core.KeyLast, pCur2->offVirtHandler, pCur2->offNextAlias));
4506 AssertReleaseMsg((pCur2->Core.Key ^ pCur->Core.Key) < PAGE_SIZE,
4507 (" pCur=%p:{.Core.Key=%RGp, .Core.KeyLast=%RGp, .offVirtHandler=%#RX32, .offNextAlias=%#RX32}\n"
4508 "pCur2=%p:{.Core.Key=%RGp, .Core.KeyLast=%RGp, .offVirtHandler=%#RX32, .offNextAlias=%#RX32}\n",
4509 pCur, pCur->Core.Key, pCur->Core.KeyLast, pCur->offVirtHandler, pCur->offNextAlias,
4510 pCur2, pCur2->Core.Key, pCur2->Core.KeyLast, pCur2->offVirtHandler, pCur2->offNextAlias));
4511 AssertReleaseMsg((pCur2->Core.KeyLast ^ pCur->Core.KeyLast) < PAGE_SIZE,
4512 (" pCur=%p:{.Core.Key=%RGp, .Core.KeyLast=%RGp, .offVirtHandler=%#RX32, .offNextAlias=%#RX32}\n"
4513 "pCur2=%p:{.Core.Key=%RGp, .Core.KeyLast=%RGp, .offVirtHandler=%#RX32, .offNextAlias=%#RX32}\n",
4514 pCur, pCur->Core.Key, pCur->Core.KeyLast, pCur->offVirtHandler, pCur->offNextAlias,
4515 pCur2, pCur2->Core.Key, pCur2->Core.KeyLast, pCur2->offVirtHandler, pCur2->offNextAlias));
4516 if (!(pCur2->offNextAlias & PGMPHYS2VIRTHANDLER_OFF_MASK))
4517 break;
4518 }
4519 }
4520
4521 pArgs->pPrevPhys2Virt = pCur;
4522 return 0;
4523}
4524
4525
4526/**
4527 * Perform an integrity check on the PGM component.
4528 *
4529 * @returns VINF_SUCCESS if everything is fine.
4530 * @returns VBox error status after asserting on integrity breach.
4531 * @param pVM The VM handle.
4532 */
4533VMMR3DECL(int) PGMR3CheckIntegrity(PVM pVM)
4534{
4535 AssertReleaseReturn(pVM->pgm.s.offVM, VERR_INTERNAL_ERROR);
4536
4537 /*
4538 * Check the trees.
4539 */
4540 int cErrors = 0;
4541 const static PGMCHECKINTARGS s_LeftToRight = { true, NULL, NULL, NULL, pVM };
4542 const static PGMCHECKINTARGS s_RightToLeft = { false, NULL, NULL, NULL, pVM };
4543 PGMCHECKINTARGS Args = s_LeftToRight;
4544 cErrors += RTAvlroGCPhysDoWithAll(&pVM->pgm.s.pTreesR3->PhysHandlers, true, pgmR3CheckIntegrityPhysHandlerNode, &Args);
4545 Args = s_RightToLeft;
4546 cErrors += RTAvlroGCPhysDoWithAll(&pVM->pgm.s.pTreesR3->PhysHandlers, false, pgmR3CheckIntegrityPhysHandlerNode, &Args);
4547 Args = s_LeftToRight;
4548 cErrors += RTAvlroGCPtrDoWithAll( &pVM->pgm.s.pTreesR3->VirtHandlers, true, pgmR3CheckIntegrityVirtHandlerNode, &Args);
4549 Args = s_RightToLeft;
4550 cErrors += RTAvlroGCPtrDoWithAll( &pVM->pgm.s.pTreesR3->VirtHandlers, false, pgmR3CheckIntegrityVirtHandlerNode, &Args);
4551 Args = s_LeftToRight;
4552 cErrors += RTAvlroGCPtrDoWithAll( &pVM->pgm.s.pTreesR3->HyperVirtHandlers, true, pgmR3CheckIntegrityVirtHandlerNode, &Args);
4553 Args = s_RightToLeft;
4554 cErrors += RTAvlroGCPtrDoWithAll( &pVM->pgm.s.pTreesR3->HyperVirtHandlers, false, pgmR3CheckIntegrityVirtHandlerNode, &Args);
4555 Args = s_LeftToRight;
4556 cErrors += RTAvlroGCPhysDoWithAll(&pVM->pgm.s.pTreesR3->PhysToVirtHandlers, true, pgmR3CheckIntegrityPhysToVirtHandlerNode, &Args);
4557 Args = s_RightToLeft;
4558 cErrors += RTAvlroGCPhysDoWithAll(&pVM->pgm.s.pTreesR3->PhysToVirtHandlers, false, pgmR3CheckIntegrityPhysToVirtHandlerNode, &Args);
4559
4560 return !cErrors ? VINF_SUCCESS : VERR_INTERNAL_ERROR;
4561}
4562
4563
4564/**
4565 * Inform PGM if we want all mappings to be put into the shadow page table. (necessary for e.g. VMX)
4566 *
4567 * @returns VBox status code.
4568 * @param pVM VM handle.
4569 * @param fEnable Enable or disable shadow mappings
4570 */
4571VMMR3DECL(int) PGMR3ChangeShwPDMappings(PVM pVM, bool fEnable)
4572{
4573 pVM->pgm.s.fDisableMappings = !fEnable;
4574
4575 uint32_t cb;
4576 int rc = PGMR3MappingsSize(pVM, &cb);
4577 AssertRCReturn(rc, rc);
4578
4579 /* Pretend the mappings are now fixed; to force a refresh of the reserved PDEs. */
4580 rc = PGMR3MappingsFix(pVM, MM_HYPER_AREA_ADDRESS, cb);
4581 AssertRCReturn(rc, rc);
4582
4583 return VINF_SUCCESS;
4584}
4585
Note: See TracBrowser for help on using the repository browser.

© 2024 Oracle Support Privacy / Do Not Sell My Info Terms of Use Trademark Policy Automated Access Etiquette