Changeset 74588 in vbox

Timestamp:

Oct 2, 2018 11:44:30 PM (6 years ago)

Author:

vboxsync

Message:

NEM/win: Doc updating for build 17757, 17763. bugref:9044

Location:

trunk/src/VBox/VMM

Files:

• VMMR3/NEMR3Native-win.cpp (modified) (7 diffs)
• testcase/NemRawBench-1.cpp (modified) (2 diffs)

trunk/src/VBox/VMM/VMMR3/NEMR3Native-win.cpp

@@ -2070 +2070 @@
  *   packed.
  *
+ *   Update: Somewhere along the security fixes or/and microcode updates this
+ *   summer (2018), performance dropped even more.
+ *
+ *   Update [build 17757]: Some performance improvements here, but they don't
+ *   yet make up for what was lost this summer.
+ *
  *
  * - We need a way to directly modify the TSC offset (or bias if you like).
@@ -2100 +2106 @@
  *        there is no way to support X2APIC.
  *
+ *
+ * - Not sure if this is a thing, but WHvCancelVirtualProcessor seems to cause
+ *   cause a lot more spurious WHvRunVirtualProcessor returns that what we get
+ *   with the replacement code.  By spurious returns we mean that the
+ *   subsequent call to WHvRunVirtualProcessor would return immediately.
+ *
+ *
+ * - There is no API for modifying protection of a page within a GPA range.
+ *
+ *   From what we can tell, the only way to modify the protection (like readonly
+ *   -> writable, or vice versa) is to first unmap the range and then remap it
+ *   with the new protection.
+ *
+ *   We are for instance doing this quite a bit in order to track dirty VRAM
+ *   pages.  VRAM pages starts out as readonly, when the guest writes to a page
+ *   we take an exit, notes down which page it is, makes it writable and restart
+ *   the instruction.  After refreshing the display, we reset all the writable
+ *   pages to readonly again, bulk fashion.
+ *
+ *   Now to work around this issue, we do page sized GPA ranges.  In addition to
+ *   add a lot of tracking overhead to WinHvPlatform and VID.SYS, this also
+ *   causes us to exceed our quota before we've even mapped a default sized
+ *   (128MB) VRAM page-by-page.  So, to work around this quota issue we have to
+ *   lazily map pages and actively restrict the number of mappings.
+ *
+ *   Our best workaround thus far is bypassing WinHvPlatform and VID entirely
+ *   when in comes to guest memory management and instead use the underlying
+ *   hypercalls (HvCallMapGpaPages, HvCallUnmapGpaPages) to do it ourselves.
+ *   (This also maps a whole lot better into our own guest page management
+ *   infrastructure.)
+ *
+ *   Update [build 17757]: Introduces a KVM like dirty logging API which could
+ *   help tracking dirty VGA pages, while being useless for shadow ROM and
+ *   devices trying catch the guest updating descriptors and such.
+ *
+ *
+ * - Observed problems doing WHvUnmapGpaRange immediately followed by
+ *   WHvMapGpaRange.
+ *
+ *   As mentioned above, we've been forced to use this sequence when modifying
+ *   page protection.   However, when transitioning from readonly to writable,
+ *   we've ended up looping forever with the same write to readonly memory
+ *   VMEXIT.  We're wondering if this issue might be related to the lazy mapping
+ *   logic in WinHvPlatform.
+ *
+ *   Workaround: Insert a WHvRunVirtualProcessor call and make sure to get a GPA
+ *   unmapped exit between the two calls.  Not entirely great performance wise
+ *   (or the santity of our code).
+ *
+ *
+ * - Implementing A20 gate behavior is tedious, where as correctly emulating the
+ *   A20M# pin (present on 486 and later) is near impossible for SMP setups
+ *   (e.g. possiblity of two CPUs with different A20 status).
+ *
+ *   Workaround: Only do A20 on CPU 0, restricting the emulation to HMA.  We
+ *   unmap all pages related to HMA (0x100000..0x10ffff) when the A20 state
+ *   changes, lazily syncing the right pages back when accessed.
+ *
+ *
+ * - WHVRunVirtualProcessor wastes time converting VID/Hyper-V messages to its
+ *   own format (WHV_RUN_VP_EXIT_CONTEXT).
+ *
+ *   We understand this might be because Microsoft wishes to remain free to
+ *   modify the VID/Hyper-V messages, but it's still rather silly and does slow
+ *   things down a little.  We'd much rather just process the messages directly.
+ *
+ *
+ * - WHVRunVirtualProcessor would've benefited from using a callback interface:
+ *
+ *      - The potential size changes of the exit context structure wouldn't be
+ *        an issue, since the function could manage that itself.
+ *
+ *      - State handling could probably be simplified (like cancelation).
+ *
+ *
+ * - WHvGetVirtualProcessorRegisters and WHvSetVirtualProcessorRegisters
+ *   internally converts register names, probably using temporary heap buffers.
+ *
+ *   From the looks of things, they are converting from WHV_REGISTER_NAME to
+ *   HV_REGISTER_NAME from in the "Virtual Processor Register Names" section in
+ *   the "Hypervisor Top-Level Functional Specification" document.  This feels
+ *   like an awful waste of time.
+ *
+ *   We simply cannot understand why HV_REGISTER_NAME isn't used directly here,
+ *   or at least the same values, making any conversion reduntant.  Restricting
+ *   access to certain registers could easily be implement by scanning the
+ *   inputs.
+ *
+ *   To avoid the heap + conversion overhead, we're currently using the
+ *   HvCallGetVpRegisters and HvCallSetVpRegisters calls directly, at least for
+ *   the ring-0 code.
+ *
+ *   Update [build 17757]: Register translation has been very cleverly
+ *   optimized and made table driven (2 top level tables, 4 + 1 leaf tables).
+ *   Register information consists of the 32-bit HV register name, register page
+ *   offset, and flags (giving valid offset, size and more).  Register
+ *   getting/settings seems to be done by hoping that the register page provides
+ *   it all, and falling back on the VidSetVirtualProcessorState if one or more
+ *   registers are not available there.
+ *
+ *   Note! We have currently not updated our ring-0 code to take the register
+ *   page into account, so it's suffering a little compared to the ring-3 code
+ *   that now uses the offical APIs for registers.
+ *
+ *
+ * - The YMM and XCR0 registers are not yet named (17083).  This probably
+ *   wouldn't be a problem if HV_REGISTER_NAME was used, see previous point.
+ *
+ *   Update [build 17757]: XCR0 is added. YMM register values seems to be put
+ *   into a yet undocumented XsaveState interface.  Approach is a little bulky,
+ *   but saves number of enums and dispenses with register transation.  Also,
+ *   the underlying Vid setter API duplicates the input buffer on the heap,
+ *   adding a 16 byte header.
+ *
+ *
+ * - Why does VID.SYS only query/set 32 registers at the time thru the
+ *   HvCallGetVpRegisters and HvCallSetVpRegisters hypercalls?
+ *
+ *   We've not trouble getting/setting all the registers defined by
+ *   WHV_REGISTER_NAME in one hypercall (around 80).  Some kind of stack
+ *   buffering or similar?
+ *
+ *
+ * - To handle the VMMCALL / VMCALL instructions, it seems we need to intercept
+ *   \#UD exceptions and inspect the opcodes.  A dedicated exit for hypercalls
+ *   would be more efficient, esp. for guests using \#UD for other purposes..
+ *
+ *
+ * - Wrong instruction length in the VpContext with unmapped GPA memory exit
+ *   contexts on 17115/AMD.
+ *
+ *   One byte "PUSH CS" was reported as 2 bytes, while a two byte
+ *   "MOV [EBX],EAX" was reported with a 1 byte instruction length.  Problem
+ *   naturally present in untranslated hyper-v messages.
+ *
+ *
+ * - The I/O port exit context information seems to be missing the address size
+ *   information needed for correct string I/O emulation.
+ *
+ *   VT-x provides this information in bits 7:9 in the instruction information
+ *   field on newer CPUs.  AMD-V in bits 7:9 in the EXITINFO1 field in the VMCB.
+ *
+ *   We can probably work around this by scanning the instruction bytes for
+ *   address size prefixes.  Haven't investigated it any further yet.
+ *
+ *
+ * - Querying WHvCapabilityCodeExceptionExitBitmap returns zero even when
+ *   intercepts demonstrably works (17134).
+ *
+ *
+ * - Querying HvPartitionPropertyDebugChannelId via HvCallGetPartitionProperty
+ *   (hypercall) hangs the host (17134).
+ *
+ *
+ *
+ * Old concerns that have been addressed:
  *
  * - The WHvCancelVirtualProcessor API schedules a dummy usermode APC callback
@@ -2116 +2278 @@
  *   modifications and the extra kernel call.
  *
- *
- * - Not sure if this is a thing, but WHvCancelVirtualProcessor seems to cause
- *   cause a lot more spurious WHvRunVirtualProcessor returns that what we get
- *   with the replacement code.  By spurious returns we mean that the
- *   subsequent call to WHvRunVirtualProcessor would return immediately.
+ *   Update: All concerns have addressed in or about build 17757.
+ *
+ *   The WHvCancelVirtualProcessor API is now implemented using a new
+ *   VidMessageSlotHandleAndGetNext() flag (4).  Codepath is slightly longer
+ *   than NtAlertThread, but has the added benefit that spurious wakeups can be
+ *   more easily reduced.
  *
  *
@@ -2131 +2294 @@
  *   what is missing from his point of view in a single kernel call.
  *
+ *   Update: All concerns have been addressed in or about build 17757.  Selected
+ *   registers are now available via shared memory and thus HLT should (not
+ *   verified) no longer require a system call to compose the exit context data.
+ *
  *
  * - The WHvRunVirtualProcessor implementation does lazy GPA range mappings when
@@ -2143 +2310 @@
  *   point).
  *
- *
- * - There is no API for modifying protection of a page within a GPA range.
- *
- *   From what we can tell, the only way to modify the protection (like readonly
- *   -> writable, or vice versa) is to first unmap the range and then remap it
- *   with the new protection.
- *
- *   We are for instance doing this quite a bit in order to track dirty VRAM
- *   pages.  VRAM pages starts out as readonly, when the guest writes to a page
- *   we take an exit, notes down which page it is, makes it writable and restart
- *   the instruction.  After refreshing the display, we reset all the writable
- *   pages to readonly again, bulk fashion.
- *
- *   Now to work around this issue, we do page sized GPA ranges.  In addition to
- *   add a lot of tracking overhead to WinHvPlatform and VID.SYS, this also
- *   causes us to exceed our quota before we've even mapped a default sized
- *   (128MB) VRAM page-by-page.  So, to work around this quota issue we have to
- *   lazily map pages and actively restrict the number of mappings.
- *
- *   Our best workaround thus far is bypassing WinHvPlatform and VID entirely
- *   when in comes to guest memory management and instead use the underlying
- *   hypercalls (HvCallMapGpaPages, HvCallUnmapGpaPages) to do it ourselves.
- *   (This also maps a whole lot better into our own guest page management
- *   infrastructure.)
- *
- *
- * - Observed problems doing WHvUnmapGpaRange immediately followed by
- *   WHvMapGpaRange.
- *
- *   As mentioned above, we've been forced to use this sequence when modifying
- *   page protection.   However, when transitioning from readonly to writable,
- *   we've ended up looping forever with the same write to readonly memory
- *   VMEXIT.  We're wondering if this issue might be related to the lazy mapping
- *   logic in WinHvPlatform.
- *
- *   Workaround: Insert a WHvRunVirtualProcessor call and make sure to get a GPA
- *   unmapped exit between the two calls.  Not entirely great performance wise
- *   (or the santity of our code).
- *
- *
- * - Implementing A20 gate behavior is tedious, where as correctly emulating the
- *   A20M# pin (present on 486 and later) is near impossible for SMP setups
- *   (e.g. possiblity of two CPUs with different A20 status).
- *
- *   Workaround: Only do A20 on CPU 0, restricting the emulation to HMA.  We
- *   unmap all pages related to HMA (0x100000..0x10ffff) when the A20 state
- *   changes, lazily syncing the right pages back when accessed.
- *
- *
- * - WHVRunVirtualProcessor wastes time converting VID/Hyper-V messages to its
- *   own format (WHV_RUN_VP_EXIT_CONTEXT).
- *
- *   We understand this might be because Microsoft wishes to remain free to
- *   modify the VID/Hyper-V messages, but it's still rather silly and does slow
- *   things down a little.  We'd much rather just process the messages directly.
- *
- *
- * - WHVRunVirtualProcessor would've benefited from using a callback interface:
- *
- *      - The potential size changes of the exit context structure wouldn't be
- *        an issue, since the function could manage that itself.
- *
- *      - State handling could probably be simplified (like cancelation).
- *
- *
- * - WHvGetVirtualProcessorRegisters and WHvSetVirtualProcessorRegisters
- *   internally converts register names, probably using temporary heap buffers.
- *
- *   From the looks of things, they are converting from WHV_REGISTER_NAME to
- *   HV_REGISTER_NAME from in the "Virtual Processor Register Names" section in
- *   the "Hypervisor Top-Level Functional Specification" document.  This feels
- *   like an awful waste of time.
- *
- *   We simply cannot understand why HV_REGISTER_NAME isn't used directly here,
- *   or at least the same values, making any conversion reduntant.  Restricting
- *   access to certain registers could easily be implement by scanning the
- *   inputs.
- *
- *   To avoid the heap + conversion overhead, we're currently using the
- *   HvCallGetVpRegisters and HvCallSetVpRegisters calls directly.
- *
- *
- * - The YMM and XCR0 registers are not yet named (17083).  This probably
- *   wouldn't be a problem if HV_REGISTER_NAME was used, see previous point.
- *
- *
- * - Why does VID.SYS only query/set 32 registers at the time thru the
- *   HvCallGetVpRegisters and HvCallSetVpRegisters hypercalls?
- *
- *   We've not trouble getting/setting all the registers defined by
- *   WHV_REGISTER_NAME in one hypercall (around 80).  Some kind of stack
- *   buffering or similar?
- *
- *
- * - To handle the VMMCALL / VMCALL instructions, it seems we need to intercept
- *   \#UD exceptions and inspect the opcodes.  A dedicated exit for hypercalls
- *   would be more efficient, esp. for guests using \#UD for other purposes..
- *
- *
- * - Wrong instruction length in the VpContext with unmapped GPA memory exit
- *   contexts on 17115/AMD.
- *
- *   One byte "PUSH CS" was reported as 2 bytes, while a two byte
- *   "MOV [EBX],EAX" was reported with a 1 byte instruction length.  Problem
- *   naturally present in untranslated hyper-v messages.
- *
- *
- * - The I/O port exit context information seems to be missing the address size
- *   information needed for correct string I/O emulation.
- *
- *   VT-x provides this information in bits 7:9 in the instruction information
- *   field on newer CPUs.  AMD-V in bits 7:9 in the EXITINFO1 field in the VMCB.
- *
- *   We can probably work around this by scanning the instruction bytes for
- *   address size prefixes.  Haven't investigated it any further yet.
- *
- *
- * - Querying WHvCapabilityCodeExceptionExitBitmap returns zero even when
- *   intercepts demonstrably works (17134).
- *
- *
- * - Querying HvPartitionPropertyDebugChannelId via HvCallGetPartitionProperty
- *   (hypercall) hangs the host (17134).
+ *   Update: All concerns have been addressed in or about build 17757.
  *
  *
@@ -2358 +2403 @@
  * @subsection sec_nem_win_benchmarks           Benchmarks.
  *
- * @subsubsection subsect_nem_win_benchmarks_bs2t1 Bootsector2-test1
+ * @subsubsection subsect_nem_win_benchmarks_bs2t1      17134/2018-06-22: Bootsector2-test1
  *
  * This is ValidationKit/bootsectors/bootsector2-test1.asm as of 2018-06-22
@@ -2467 +2512 @@
  *
  *
- * @subsubsection subsect_nem_win_benchmarks_w2k    Windows 2000 Boot & Shutdown
+ * @subsubsection subsect_nem_win_benchmarks_bs2t1u1   17134/2018-10-02: Bootsector2-test1
+ *
+ * Update on 17134.  While expectantly testing a couple of newer builds (17758,
+ * 17763) hoping for some increases in performance, the numbers turn out
+ * to be generally worse than the initial test June test run.  So, I went back
+ * to the 1803 (17134) installation and re-tested, finding that the numbers had
+ * somehow turned worse over the last 3-4 months.
+ *
+ *
+ *
+ * Suspects are security updates and/or microcode updates installed since then,
+ * either hitting thread switching and/or hyper-V badly.  I'm a bit puzzled why
+ * AMD is affected this badly too.
+ *
+ *
+ * @subsubsection subsect_nem_win_benchmarks_bs2t1u2   17763: Bootsector2-test1
+ *
+ * Some preliminary numbers for build 17763 on the 3.4 GHz AMD 1950X, the second
+ * column will improve we get time to have a look the register page.
+ *
+ * There is a  50%  performance loss here compared to the June numbers with
+ * build 17134.  The RDTSC numbers hits that it isn't in the Hyper-V core
+ * (hvax64.exe), but something on the NT side.
+ *
+ * @verbatim
+TESTING...                                                           WinHv API           Hypercalls + VID    VirtualBox AMD-V
+  32-bit paged protected mode, CPUID                        :           54 145 ins/sec        51 436
+  real mode, CPUID                                          :           54 178 ins/sec        51 713
+  [snip]
+  32-bit paged protected mode, RDTSC                        :       98 927 639 ins/sec   100 254 552
+  real mode, RDTSC                                          :       99 601 206 ins/sec   100 886 699
+  [snip]
+  32-bit paged protected mode, 32-bit IN                    :           54 621 ins/sec        51 524
+  32-bit paged protected mode, 32-bit OUT                   :           54 870 ins/sec        51 671
+  32-bit paged protected mode, 32-bit IN-to-ring-3          :           54 624 ins/sec        43 964
+  32-bit paged protected mode, 32-bit OUT-to-ring-3         :           54 803 ins/sec        44 087
+  [snip]
+  32-bit paged protected mode, 32-bit read                  :           28 230 ins/sec        34 042
+  32-bit paged protected mode, 32-bit write                 :           27 962 ins/sec        34 050
+  32-bit paged protected mode, 32-bit read-to-ring-3        :           27 841 ins/sec        28 397
+  32-bit paged protected mode, 32-bit write-to-ring-3       :           27 896 ins/sec        29 455
+ * @endverbatim
+ *
+ *
+ * @subsubsection subsect_nem_win_benchmarks_w2k    17134/2018-06-22: Windows 2000 Boot & Shutdown
  *
  * Timing the startup and automatic shutdown of a Windows 2000 SP4 guest serves

trunk/src/VBox/VMM/testcase/NemRawBench-1.cpp

@@ -1258 +1258 @@
     if (rcExit == 0)
     {
-        printf("tstNemMini-1: Successfully created test VM...\n");
+        printf("tstNemBench-1: Successfully created test VM...\n");
 
         /*
@@ -1267 +1267 @@
         mmioTest(cFactor);
 
-        printf("tstNemMini-1: done\n");
+        printf("tstNemBench-1: done\n");
     }
     return rcExit;