VirtualBox

Ignore:
Timestamp:
Nov 24, 2021 9:29:59 AM (3 years ago)
Author:
vboxsync
svn:sync-xref-src-repo-rev:
148466
Message:

Audio/Validation Kit: Updated docs based on latest feedback. bugref:10008

Location:
trunk/src/VBox/ValidationKit/docs
Files:
2 edited

Legend:

Unmodified
Added
Removed

  • trunk/src/VBox/ValidationKit/docs/VBoxAudioValidationKitReadMe.html

    r92474 r92584  
    55<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
    66<meta name="generator" content="Docutils 0.18: http://docutils.sourceforge.net/" />
    7 <title>Audio Testing via Validation Kit</title>
     7<title>Audio Testing of VirtualBox</title>
    88<style type="text/css">
    99
    … …  
    361361</head>
    362362<body>
    363 <div class="document" id="audio-testing-via-validation-kit">
    364 <h1 class="title">Audio Testing via Validation Kit</h1>
     363<div class="document" id="audio-testing-of-virtualbox">
     364<h1 class="title">Audio Testing of VirtualBox</h1>
    365365
    366366<div class="section" id="overview-goal">
    367367<h1>Overview / Goal</h1>
    368 <p>The goal was to create a testing framework which utilizes the
    369 VirtualBox Validation Kit to test the VirtualBox audio stack.</p>
     368<p>The goal is to create a flexible testing framework to test the
     369VirtualBox audio stack.</p>
     370<p>It should be runnable with an easy-to-use setup so that also regular users
     371can perform tests on request, without having to install or set up additional
     372dependencies.</p>
    370373<p>That framework must be runnable on all host/guest combinations together with all
    371374audio drivers (&quot;backends&quot;) and device emulations being offered. This makes it a
    372375rather big testing matrix which therefore has to be processed in an automated
    373376fashion.</p>
    374 <p>Additionally it should be flexible enough to add more (custom) tests lateron.</p>
    375 </div>
    376 <div class="section" id="current-status-limitations">
    377 <h1>Current status / limitations</h1>
    378 <ul class="simple">
    379 <li><dl class="first docutils">
    380 <dt>The following test types are currently implemented:</dt>
    381 <dd><ul class="first last">
    382 <li>Test tone (sine wave) playback from the guest</li>
    383 <li>Test tone (sine wave) recording by the guest (injected from the host)</li>
    384 </ul>
    385 </dd>
    386 </dl>
    387 </li>
    388 <li>Only the HDA device emulation has been verified so far.</li>
    389 <li>Only the ALSA audio stack on Debian 10 has been verified so far.
    390 Note: This is different from PulseAudio using the ALSA plugin!</li>
    391 </ul>
     377<p>Additionally it should be flexible enough to add more (custom) tests later on.</p>
    392378</div>
    393379<div class="section" id="operation">
    … …  
    396382the existing audio stack code as possible. This allows the following
    397383operation modes:</p>
    398 <ul class="simple">
    399 <li>Standalone: Playing back / recording audio data (test tones / .WAV files) in a
     384<dl class="docutils">
     385<dt>Standalone</dt>
     386<dd>Playing back / recording audio data (test tones / .WAV files) in a
    400387standalone scenario, i.e. no VirtualBox / VMs required). This mode is using
    401 the audio (mixing) stack and available backend drivers without the need of
    402 VirtualBox being installed.</li>
    403 <li>Manual: Performing single / multiple tests manually on a local machine.
    404 Requires a running and set up test VM.</li>
    405 <li>Automated: Performs single / multiple tests via the Validation Kit audio test
    406 driver and can be triggered via the Validation Kit Test Manager. The test
    407 driver can be found at [1].</li>
    408 <li>(Re-)validation of previously ran tests: This takes two test sets and runs
    409 the validation / analysis on them. See VKAT's &quot;verify&quot; sub command for more.</li>
    410 <li>Self testing mode: Performs standalone self tests to verify / debug the
    411 involved components. See VKAT's &quot;selftest&quot; sub command for more.</li>
    412 </ul>
    413 <p>[1] src/VBox/ValidationKit/tests/audio/tdAudioTest.py</p>
     388VirtualBox' audio (mixing) stack and available backend drivers without the
     389need of VirtualBox being installed.</dd>
     390<dt>Manual</dt>
     391<dd>Performing single / multiple tests manually on a local machine.
     392Requires a running and set up test VM.</dd>
     393<dt>Automated</dt>
     394<dd>Performs single / multiple tests via the Validation Kit audio test
     395driver and can be triggered via the Validation Kit Test Manager.</dd>
     396<dt>(Re-)validation of previously ran tests</dt>
     397<dd>This takes two test sets and runs the validation / analysis on them.</dd>
     398<dt>Self testing mode</dt>
     399<dd>Performs standalone self tests to verify / debug the involved components.</dd>
     400</dl>
    414401</div>
    415402<div class="section" id="components-and-terminology">
    … …  
    418405(depends on the operation mode, see above):</p>
    419406<ul>
    420 <li><p class="first">VKAT (&quot;Validation Kit Audio Test&quot;, also known as VBoxAudioTest):
     407<li><p class="first">VBoxAudioTest (also known as VKAT, &quot;Validation Kit Audio Test&quot;):
    421408A binary which can perform the standalone audio tests mentioned above, as well
    422409as acting as the guest and host service(s) when performing manual or automated
    … …  
    440427<p>See the syntax help (&quot;--help&quot;) for more.</p>
    441428</li>
    442 <li><p class="first">ATS (&quot;Audio Testing Service&quot;): Component which is being used by VKAT and the
     429<li><p class="first">ATS (&quot;Audio Testing Service&quot;): Component which is being used by 1 and the
    443430Validation Kit audio driver (backend) to communicate across guest and host
    444431boundaries. Currently using a TCP/IP transport layer. Also works with VMs
    … …  
    447434<li><p class="first">Validation Kit audio test driver (tdAudioTest.py): Used for integrating and
    448435invoking VKAT for manual and automated tests via the Validation Kit framework
    449 (Test Manager). Optional.</p>
     436(Test Manager). Optional. The test driver can be found at <a class="footnote-reference" href="#footnote-1" id="footnote-reference-1">[1]</a>.</p>
    450437</li>
    451438<li><p class="first">Validation Kit audio driver (backend): A dedicated audio backend which
    … …  
    474461<p>The above components are also included in VirtualBox release builds and can be
    475462optionally enabled (disabled by default).</p>
    476 </div>
    477 <div class="section" id="workflow-for-a-single-test">
    478 <h1>Workflow for a single test</h1>
     463<table class="docutils footnote" frame="void" id="footnote-1" rules="none">
     464<colgroup><col class="label" /><col /></colgroup>
     465<tbody valign="top">
     466<tr><td class="label"><a class="fn-backref" href="#footnote-reference-1">[1]</a></td><td>src/VBox/ValidationKit/tests/audio/tdAudioTest.py</td></tr>
     467</tbody>
     468</table>
     469</div>
     470<div class="section" id="setup-instructions">
     471<h1>Setup instructions</h1>
     472<ul>
     473<li><p class="first">VM needs to be configured to have audio emulation and audio testing enabled
     474(via extra-data, set &quot;VBoxInternal2/Audio/Debug/Enabled&quot; to &quot;true&quot;).</p>
     475</li>
     476<li><p class="first">Audio input / output for the VM needs to be enabled (depending on the test).</p>
     477</li>
     478<li><p class="first">Start VBoxAudioTest on the guest, for example:</p>
     479<p>VBoxAudioTest test --mode guest --tcp-connect-address 10.0.2.2</p>
     480<dl class="docutils">
     481<dt>Note: VBoxAudioTest is included with the Guest Additions starting at</dt>
     482<dd><p class="first last">VirtualBox 7.0.</p>
     483</dd>
     484<dt>Note: Depending on the VM's networking configuration there might be further</dt>
     485<dd><p class="first last">steps necessary in order to be able to reach the host from the guest.
     486See the VirtualBox manual for more information.</p>
     487</dd>
     488</dl>
     489</li>
     490</ul>
     491</div>
     492<div class="section" id="performing-a-manual-test">
     493<h1>Performing a manual test</h1>
     494<ul>
     495<li><p class="first">Follow &quot;Setup instructions&quot;.</p>
     496</li>
     497<li><p class="first">Start VBoxAudioTest on the host with selected test(s), for example:</p>
     498<p>VBoxAudioTest test --mode host</p>
     499<blockquote>
     500<dl class="docutils">
     501<dt>Note: VBoxAudioTest is included with the VirtualBox 7.0 host installers and</dt>
     502<dd><p class="first last">will be installed by default.</p>
     503</dd>
     504</dl>
     505</blockquote>
     506</li>
     507<li><p class="first">By default the test verification will be done automatically after running the
     508tests.</p>
     509</li>
     510</ul>
     511</div>
     512<div class="section" id="advanced-performing-manual-verification">
     513<h1>Advanced: Performing manual verification</h1>
     514<p>VBoxAudioTest can manually be used with the &quot;verify&quot; sub command in order to
     515(re-)verify previously generated test sets. It then will return different exit
     516codes based on the verification result.</p>
     517</div>
     518<div class="section" id="advanced-performing-an-automated-test">
     519<h1>Advanced: Performing an automated test</h1>
     520<ul class="simple">
     521<li>TxS (Test E[x]ecution Service) has to be up and running (part of the
     522Validation Kit) on the guest.</li>
     523<li>Invoke the tdAudioTest.py test driver, either manually or fully automated
     524via Test Manager.</li>
     525</ul>
     526</div>
     527<div class="section" id="internals-workflow-for-a-single-test">
     528<h1>Internals: Workflow for a single test</h1>
    479529<p>When a single test is being executed on a running VM, the following (simplified)
    480530workflow applies:</p>
    … …  
    505555</ul>
    506556</div>
    507 <div class="section" id="setup-instructions">
    508 <h1>Setup instructions</h1>
    509 <ul>
    510 <li><p class="first">VM needs to be configured to have audio emulation and audio testing enabled
    511 (via extra-data, set &quot;VBoxInternal2/Audio/Debug/Enabled&quot; to &quot;true&quot;).</p>
    512 </li>
    513 <li><p class="first">Audio input / output for the VM needs to be enabled (depending on the test).</p>
    514 </li>
    515 <li><p class="first">VKAT needs to be running on the guest and be able to connect to the host via
    516 TCP/IP.</p>
    517 <dl class="docutils">
    518 <dt>Note: Depending on the VM's networking configuration there might be further</dt>
    519 <dd><p class="first last">steps necessary in order to be able to reach the host from the guest.
    520 See the VirtualBox manual for more information.</p>
     557<div class="section" id="current-status-limitations">
     558<h1>Current status / limitations</h1>
     559<ul class="simple">
     560<li><dl class="first docutils">
     561<dt>The following test types are currently implemented:</dt>
     562<dd><ul class="first last">
     563<li>Test tone (sine wave) playback from the guest</li>
     564<li>Test tone (sine wave) recording by the guest (injected from the host)</li>
     565</ul>
    521566</dd>
    522567</dl>
    523568</li>
    524 </ul>
    525 </div>
    526 <div class="section" id="performing-a-manual-test">
    527 <h1>Performing a manual test</h1>
    528 <ul class="simple">
    529 <li>Follow &quot;Setup instructions&quot;.</li>
    530 <li>Start VKAT on the guest (in &quot;guest mode&quot;, e.g. &quot;test --mode guest&quot;).</li>
    531 <li>Start VKAT on the host (in &quot;host mode&quot;, e.g. &quot;test --mode host&quot;) with
    532 selected test(s).</li>
    533 <li>By default the test verification will be done automatically after running the
    534 tests.</li>
    535 </ul>
    536 </div>
    537 <div class="section" id="performing-manual-verification">
    538 <h1>Performing manual verification</h1>
    539 <p>VKAT can manually be used with the &quot;verify&quot; sub command in order to (re-)verify
    540 previously generated test sets. It then will return different exit codes based
    541 on the verification result.</p>
    542 </div>
    543 <div class="section" id="performing-an-automated-test">
    544 <h1>Performing an automated test</h1>
    545 <ul class="simple">
    546 <li>TxS (Test E[x]ecution Service) has to be up and running (part of the
    547 Validation Kit) on the guest.</li>
    548 <li>Invoke the tdAudioTest.py test driver, either manually or fully automated
    549 via Test Manager.</li>
     569<li>Only the HDA device emulation has been verified so far.</li>
     570<li>Only the ALSA audio stack on Debian 10 has been verified so far.
     571Note: This is different from PulseAudio using the ALSA plugin!</li>
    550572</ul>
    551573</div>
    … …  
    555577<li>Make sure that audio device emulation is enabled and can be used within the
    556578guest. Also, audio input / output has to be enabled, depending on the tests.</li>
    557 <li>Make sure that the guest's VKAT instance can reach the host via the selected
    558 transport lay (TCP/IP by default).</li>
     579<li>Make sure that the guest's VBoxAudioTest's instance can reach the host via
     580the selected transport layer (TCP/IP by default).</li>
    559581<li>Increase the hosts audio logging level
    560582(via extra-data, set &quot;VBoxInternal2/Audio/Debug/Level&quot; to &quot;5&quot;).</li>
    561 <li>Increase VKAT's verbosity level (add &quot;-v&quot;, can be specified multiple times).</li>
    562 <li>Check if the the VBox release log contains any warnings / errors with the
     583<li>Increase VBoxAudioTest's verbosity level (add &quot;-v&quot;, can be specified
     584multiple times).</li>
     585<li>Check if the VBox release log contains any warnings / errors with the
    563586&quot;ValKit:&quot; prefix.</li>
    564587</ul>

  • trunk/src/VBox/ValidationKit/docs/VBoxAudioValidationKitReadMe.txt

    r92474 r92584  
    1 Audio Testing via Validation Kit
    2 ================================
     1Audio Testing of VirtualBox
     2===========================
    33
    44
    … …  
    66---------------
    77
    8 The goal was to create a testing framework which utilizes the
    9 VirtualBox Validation Kit to test the VirtualBox audio stack.
     8The goal is to create a flexible testing framework to test the
     9VirtualBox audio stack.
     10
     11It should be runnable with an easy-to-use setup so that also regular users
     12can perform tests on request, without having to install or set up additional
     13dependencies.
    1014
    1115That framework must be runnable on all host/guest combinations together with all
    … …  
    1418fashion.
    1519
    16 Additionally it should be flexible enough to add more (custom) tests lateron.
    17 
    18 
    19 Current status / limitations
    20 ----------------------------
    21 
    22 - The following test types are currently implemented:
    23     * Test tone (sine wave) playback from the guest
    24     * Test tone (sine wave) recording by the guest (injected from the host)
    25 - Only the HDA device emulation has been verified so far.
    26 - Only the ALSA audio stack on Debian 10 has been verified so far.
    27   Note: This is different from PulseAudio using the ALSA plugin!
     20Additionally it should be flexible enough to add more (custom) tests later on.
    2821
    2922
    … …  
    3528operation modes:
    3629
    37 - Standalone: Playing back / recording audio data (test tones / .WAV files) in a
     30Standalone
     31  Playing back / recording audio data (test tones / .WAV files) in a
    3832  standalone scenario, i.e. no VirtualBox / VMs required). This mode is using
    39   the audio (mixing) stack and available backend drivers without the need of
    40   VirtualBox being installed.
    41 
    42 - Manual: Performing single / multiple tests manually on a local machine.
     33  VirtualBox' audio (mixing) stack and available backend drivers without the
     34  need of VirtualBox being installed.
     35
     36Manual
     37  Performing single / multiple tests manually on a local machine.
    4338  Requires a running and set up test VM.
    4439
    45 - Automated: Performs single / multiple tests via the Validation Kit audio test
    46   driver and can be triggered via the Validation Kit Test Manager. The test
    47   driver can be found at [1].
    48 
    49 - (Re-)validation of previously ran tests: This takes two test sets and runs
    50   the validation / analysis on them. See VKAT's "verify" sub command for more.
    51 
    52 - Self testing mode: Performs standalone self tests to verify / debug the
    53   involved components. See VKAT's "selftest" sub command for more.
    54 
    55 
    56 [1] src/VBox/ValidationKit/tests/audio/tdAudioTest.py
     40Automated
     41  Performs single / multiple tests via the Validation Kit audio test
     42  driver and can be triggered via the Validation Kit Test Manager.
     43
     44(Re-)validation of previously ran tests
     45  This takes two test sets and runs the validation / analysis on them.
     46
     47Self testing mode
     48  Performs standalone self tests to verify / debug the involved components.
    5749
    5850
    … …  
    6355(depends on the operation mode, see above):
    6456
    65 - VKAT ("Validation Kit Audio Test", also known as VBoxAudioTest):
     57- VBoxAudioTest (also known as VKAT, "Validation Kit Audio Test"):
    6658  A binary which can perform the standalone audio tests mentioned above, as well
    6759  as acting as the guest and host service(s) when performing manual or automated
    … …  
    8274  See the syntax help ("--help") for more.
    8375
    84 - ATS ("Audio Testing Service"): Component which is being used by VKAT and the
     76- ATS ("Audio Testing Service"): Component which is being used by 1 and the
    8577  Validation Kit audio driver (backend) to communicate across guest and host
    8678  boundaries. Currently using a TCP/IP transport layer. Also works with VMs
    … …  
    8981- Validation Kit audio test driver (tdAudioTest.py): Used for integrating and
    9082  invoking VKAT for manual and automated tests via the Validation Kit framework
    91   (Test Manager). Optional.
     83  (Test Manager). Optional. The test driver can be found at [1]_.
    9284
    9385- Validation Kit audio driver (backend): A dedicated audio backend which
    … …  
    115107optionally enabled (disabled by default).
    116108
    117 
    118 Workflow for a single test
    119 --------------------------
     109.. [1] src/VBox/ValidationKit/tests/audio/tdAudioTest.py
     110
     111
     112Setup instructions
     113------------------
     114
     115- VM needs to be configured to have audio emulation and audio testing enabled
     116  (via extra-data, set "VBoxInternal2/Audio/Debug/Enabled" to "true").
     117- Audio input / output for the VM needs to be enabled (depending on the test).
     118- Start VBoxAudioTest on the guest, for example:
     119
     120  VBoxAudioTest test --mode guest --tcp-connect-address 10.0.2.2
     121
     122  Note: VBoxAudioTest is included with the Guest Additions starting at
     123        VirtualBox 7.0.
     124  Note: Depending on the VM's networking configuration there might be further
     125        steps necessary in order to be able to reach the host from the guest.
     126        See the VirtualBox manual for more information.
     127
     128
     129Performing a manual test
     130------------------------
     131
     132- Follow "Setup instructions".
     133- Start VBoxAudioTest on the host with selected test(s), for example:
     134
     135  VBoxAudioTest test --mode host
     136
     137    Note: VBoxAudioTest is included with the VirtualBox 7.0 host installers and
     138          will be installed by default.
     139
     140- By default the test verification will be done automatically after running the
     141  tests.
     142
     143
     144Advanced: Performing manual verification
     145----------------------------------------
     146
     147VBoxAudioTest can manually be used with the "verify" sub command in order to
     148(re-)verify previously generated test sets. It then will return different exit
     149codes based on the verification result.
     150
     151
     152Advanced: Performing an automated test
     153--------------------------------------
     154
     155- TxS (Test E[x]ecution Service) has to be up and running (part of the
     156  Validation Kit) on the guest.
     157- Invoke the tdAudioTest.py test driver, either manually or fully automated
     158  via Test Manager.
     159
     160
     161Internals: Workflow for a single test
     162-------------------------------------
    120163
    121164When a single test is being executed on a running VM, the following (simplified)
    … …  
    135178
    136179
    137 Setup instructions
    138 ------------------
    139 
    140 - VM needs to be configured to have audio emulation and audio testing enabled
    141   (via extra-data, set "VBoxInternal2/Audio/Debug/Enabled" to "true").
    142 - Audio input / output for the VM needs to be enabled (depending on the test).
    143 - VKAT needs to be running on the guest and be able to connect to the host via
    144   TCP/IP.
    145 
    146   Note: Depending on the VM's networking configuration there might be further
    147         steps necessary in order to be able to reach the host from the guest.
    148         See the VirtualBox manual for more information.
    149 
    150 
    151 Performing a manual test
    152 ------------------------
    153 
    154 - Follow "Setup instructions".
    155 - Start VKAT on the guest (in "guest mode", e.g. "test --mode guest").
    156 - Start VKAT on the host (in "host mode", e.g. "test --mode host") with
    157   selected test(s).
    158 - By default the test verification will be done automatically after running the
    159   tests.
    160 
    161 
    162 Performing manual verification
    163 ------------------------------
    164 
    165 VKAT can manually be used with the "verify" sub command in order to (re-)verify
    166 previously generated test sets. It then will return different exit codes based
    167 on the verification result.
    168 
    169 
    170 Performing an automated test
     180Current status / limitations
    171181----------------------------
    172182
    173 - TxS (Test E[x]ecution Service) has to be up and running (part of the
    174   Validation Kit) on the guest.
    175 - Invoke the tdAudioTest.py test driver, either manually or fully automated
    176   via Test Manager.
     183- The following test types are currently implemented:
     184    * Test tone (sine wave) playback from the guest
     185    * Test tone (sine wave) recording by the guest (injected from the host)
     186- Only the HDA device emulation has been verified so far.
     187- Only the ALSA audio stack on Debian 10 has been verified so far.
     188  Note: This is different from PulseAudio using the ALSA plugin!
    177189
    178190
    … …  
    182194- Make sure that audio device emulation is enabled and can be used within the
    183195  guest. Also, audio input / output has to be enabled, depending on the tests.
    184 - Make sure that the guest's VKAT instance can reach the host via the selected
    185   transport lay (TCP/IP by default).
     196- Make sure that the guest's VBoxAudioTest's instance can reach the host via
     197  the selected transport layer (TCP/IP by default).
    186198- Increase the hosts audio logging level
    187199  (via extra-data, set "VBoxInternal2/Audio/Debug/Level" to "5").
    188 - Increase VKAT's verbosity level (add "-v", can be specified multiple times).
    189 - Check if the the VBox release log contains any warnings / errors with the
     200- Increase VBoxAudioTest's verbosity level (add "-v", can be specified
     201  multiple times).
     202- Check if the VBox release log contains any warnings / errors with the
    190203  "ValKit:" prefix.
    191204
Note: See TracChangeset for help on using the changeset viewer.

© 2025 Oracle Support Privacy / Do Not Sell My Info Terms of Use Trademark Policy Automated Access Etiquette