Changeset 92584 in vbox for trunk/src/VBox/ValidationKit/docs/VBoxAudioValidationKitReadMe.html
- Timestamp:
- Nov 24, 2021 9:29:59 AM (3 years ago)
- File:
-
- 1 edited
Legend:
- Unmodified
- Added
- Removed
-
trunk/src/VBox/ValidationKit/docs/VBoxAudioValidationKitReadMe.html
r92474 r92584 5 5 <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> 6 6 <meta name="generator" content="Docutils 0.18: http://docutils.sourceforge.net/" /> 7 <title>Audio Testing via Validation Kit</title>7 <title>Audio Testing of VirtualBox</title> 8 8 <style type="text/css"> 9 9 … … 361 361 </head> 362 362 <body> 363 <div class="document" id="audio-testing- via-validation-kit">364 <h1 class="title">Audio Testing via Validation Kit</h1>363 <div class="document" id="audio-testing-of-virtualbox"> 364 <h1 class="title">Audio Testing of VirtualBox</h1> 365 365 366 366 <div class="section" id="overview-goal"> 367 367 <h1>Overview / Goal</h1> 368 <p>The goal was to create a testing framework which utilizes the 369 VirtualBox Validation Kit to test the VirtualBox audio stack.</p> 368 <p>The goal is to create a flexible testing framework to test the 369 VirtualBox audio stack.</p> 370 <p>It should be runnable with an easy-to-use setup so that also regular users 371 can perform tests on request, without having to install or set up additional 372 dependencies.</p> 370 373 <p>That framework must be runnable on all host/guest combinations together with all 371 374 audio drivers ("backends") and device emulations being offered. This makes it a 372 375 rather big testing matrix which therefore has to be processed in an automated 373 376 fashion.</p> 374 <p>Additionally it should be flexible enough to add more (custom) tests lateron.</p> 375 </div> 376 <div class="section" id="current-status-limitations"> 377 <h1>Current status / limitations</h1> 378 <ul class="simple"> 379 <li><dl class="first docutils"> 380 <dt>The following test types are currently implemented:</dt> 381 <dd><ul class="first last"> 382 <li>Test tone (sine wave) playback from the guest</li> 383 <li>Test tone (sine wave) recording by the guest (injected from the host)</li> 384 </ul> 385 </dd> 386 </dl> 387 </li> 388 <li>Only the HDA device emulation has been verified so far.</li> 389 <li>Only the ALSA audio stack on Debian 10 has been verified so far. 390 Note: This is different from PulseAudio using the ALSA plugin!</li> 391 </ul> 377 <p>Additionally it should be flexible enough to add more (custom) tests later on.</p> 392 378 </div> 393 379 <div class="section" id="operation"> … … 396 382 the existing audio stack code as possible. This allows the following 397 383 operation modes:</p> 398 <ul class="simple"> 399 <li>Standalone: Playing back / recording audio data (test tones / .WAV files) in a 384 <dl class="docutils"> 385 <dt>Standalone</dt> 386 <dd>Playing back / recording audio data (test tones / .WAV files) in a 400 387 standalone scenario, i.e. no VirtualBox / VMs required). This mode is using 401 the audio (mixing) stack and available backend drivers without the need of 402 VirtualBox being installed.</li>403 < li>Manual: Performing single / multiple tests manually on a local machine.404 Requires a running and set up test VM.</li> 405 <li>Automated: Performs single / multiple tests via the Validation Kit audio test 406 driver and can be triggered via the Validation Kit Test Manager. The test 407 driver can be found at [1].</li> 408 <li>(Re-)validation of previously ran tests: This takes two test sets and runs 409 the validation / analysis on them. See VKAT's "verify" sub command for more.</li>410 < li>Self testing mode: Performs standalone self tests to verify / debug the411 involved components. See VKAT's "selftest" sub command for more.</li>412 < /ul>413 < p>[1] src/VBox/ValidationKit/tests/audio/tdAudioTest.py</p>388 VirtualBox' audio (mixing) stack and available backend drivers without the 389 need of VirtualBox being installed.</dd> 390 <dt>Manual</dt> 391 <dd>Performing single / multiple tests manually on a local machine. 392 Requires a running and set up test VM.</dd> 393 <dt>Automated</dt> 394 <dd>Performs single / multiple tests via the Validation Kit audio test 395 driver and can be triggered via the Validation Kit Test Manager.</dd> 396 <dt>(Re-)validation of previously ran tests</dt> 397 <dd>This takes two test sets and runs the validation / analysis on them.</dd> 398 <dt>Self testing mode</dt> 399 <dd>Performs standalone self tests to verify / debug the involved components.</dd> 400 </dl> 414 401 </div> 415 402 <div class="section" id="components-and-terminology"> … … 418 405 (depends on the operation mode, see above):</p> 419 406 <ul> 420 <li><p class="first">V KAT ("Validation Kit Audio Test", also known as VBoxAudioTest):407 <li><p class="first">VBoxAudioTest (also known as VKAT, "Validation Kit Audio Test"): 421 408 A binary which can perform the standalone audio tests mentioned above, as well 422 409 as acting as the guest and host service(s) when performing manual or automated … … 440 427 <p>See the syntax help ("--help") for more.</p> 441 428 </li> 442 <li><p class="first">ATS ("Audio Testing Service"): Component which is being used by VKATand the429 <li><p class="first">ATS ("Audio Testing Service"): Component which is being used by 1 and the 443 430 Validation Kit audio driver (backend) to communicate across guest and host 444 431 boundaries. Currently using a TCP/IP transport layer. Also works with VMs … … 447 434 <li><p class="first">Validation Kit audio test driver (tdAudioTest.py): Used for integrating and 448 435 invoking VKAT for manual and automated tests via the Validation Kit framework 449 (Test Manager). Optional. </p>436 (Test Manager). Optional. The test driver can be found at <a class="footnote-reference" href="#footnote-1" id="footnote-reference-1">[1]</a>.</p> 450 437 </li> 451 438 <li><p class="first">Validation Kit audio driver (backend): A dedicated audio backend which … … 474 461 <p>The above components are also included in VirtualBox release builds and can be 475 462 optionally enabled (disabled by default).</p> 476 </div> 477 <div class="section" id="workflow-for-a-single-test"> 478 <h1>Workflow for a single test</h1> 463 <table class="docutils footnote" frame="void" id="footnote-1" rules="none"> 464 <colgroup><col class="label" /><col /></colgroup> 465 <tbody valign="top"> 466 <tr><td class="label"><a class="fn-backref" href="#footnote-reference-1">[1]</a></td><td>src/VBox/ValidationKit/tests/audio/tdAudioTest.py</td></tr> 467 </tbody> 468 </table> 469 </div> 470 <div class="section" id="setup-instructions"> 471 <h1>Setup instructions</h1> 472 <ul> 473 <li><p class="first">VM needs to be configured to have audio emulation and audio testing enabled 474 (via extra-data, set "VBoxInternal2/Audio/Debug/Enabled" to "true").</p> 475 </li> 476 <li><p class="first">Audio input / output for the VM needs to be enabled (depending on the test).</p> 477 </li> 478 <li><p class="first">Start VBoxAudioTest on the guest, for example:</p> 479 <p>VBoxAudioTest test --mode guest --tcp-connect-address 10.0.2.2</p> 480 <dl class="docutils"> 481 <dt>Note: VBoxAudioTest is included with the Guest Additions starting at</dt> 482 <dd><p class="first last">VirtualBox 7.0.</p> 483 </dd> 484 <dt>Note: Depending on the VM's networking configuration there might be further</dt> 485 <dd><p class="first last">steps necessary in order to be able to reach the host from the guest. 486 See the VirtualBox manual for more information.</p> 487 </dd> 488 </dl> 489 </li> 490 </ul> 491 </div> 492 <div class="section" id="performing-a-manual-test"> 493 <h1>Performing a manual test</h1> 494 <ul> 495 <li><p class="first">Follow "Setup instructions".</p> 496 </li> 497 <li><p class="first">Start VBoxAudioTest on the host with selected test(s), for example:</p> 498 <p>VBoxAudioTest test --mode host</p> 499 <blockquote> 500 <dl class="docutils"> 501 <dt>Note: VBoxAudioTest is included with the VirtualBox 7.0 host installers and</dt> 502 <dd><p class="first last">will be installed by default.</p> 503 </dd> 504 </dl> 505 </blockquote> 506 </li> 507 <li><p class="first">By default the test verification will be done automatically after running the 508 tests.</p> 509 </li> 510 </ul> 511 </div> 512 <div class="section" id="advanced-performing-manual-verification"> 513 <h1>Advanced: Performing manual verification</h1> 514 <p>VBoxAudioTest can manually be used with the "verify" sub command in order to 515 (re-)verify previously generated test sets. It then will return different exit 516 codes based on the verification result.</p> 517 </div> 518 <div class="section" id="advanced-performing-an-automated-test"> 519 <h1>Advanced: Performing an automated test</h1> 520 <ul class="simple"> 521 <li>TxS (Test E[x]ecution Service) has to be up and running (part of the 522 Validation Kit) on the guest.</li> 523 <li>Invoke the tdAudioTest.py test driver, either manually or fully automated 524 via Test Manager.</li> 525 </ul> 526 </div> 527 <div class="section" id="internals-workflow-for-a-single-test"> 528 <h1>Internals: Workflow for a single test</h1> 479 529 <p>When a single test is being executed on a running VM, the following (simplified) 480 530 workflow applies:</p> … … 505 555 </ul> 506 556 </div> 507 <div class="section" id="setup-instructions"> 508 <h1>Setup instructions</h1> 509 <ul> 510 <li><p class="first">VM needs to be configured to have audio emulation and audio testing enabled 511 (via extra-data, set "VBoxInternal2/Audio/Debug/Enabled" to "true").</p> 512 </li> 513 <li><p class="first">Audio input / output for the VM needs to be enabled (depending on the test).</p> 514 </li> 515 <li><p class="first">VKAT needs to be running on the guest and be able to connect to the host via 516 TCP/IP.</p> 517 <dl class="docutils"> 518 <dt>Note: Depending on the VM's networking configuration there might be further</dt> 519 <dd><p class="first last">steps necessary in order to be able to reach the host from the guest. 520 See the VirtualBox manual for more information.</p> 557 <div class="section" id="current-status-limitations"> 558 <h1>Current status / limitations</h1> 559 <ul class="simple"> 560 <li><dl class="first docutils"> 561 <dt>The following test types are currently implemented:</dt> 562 <dd><ul class="first last"> 563 <li>Test tone (sine wave) playback from the guest</li> 564 <li>Test tone (sine wave) recording by the guest (injected from the host)</li> 565 </ul> 521 566 </dd> 522 567 </dl> 523 568 </li> 524 </ul> 525 </div> 526 <div class="section" id="performing-a-manual-test"> 527 <h1>Performing a manual test</h1> 528 <ul class="simple"> 529 <li>Follow "Setup instructions".</li> 530 <li>Start VKAT on the guest (in "guest mode", e.g. "test --mode guest").</li> 531 <li>Start VKAT on the host (in "host mode", e.g. "test --mode host") with 532 selected test(s).</li> 533 <li>By default the test verification will be done automatically after running the 534 tests.</li> 535 </ul> 536 </div> 537 <div class="section" id="performing-manual-verification"> 538 <h1>Performing manual verification</h1> 539 <p>VKAT can manually be used with the "verify" sub command in order to (re-)verify 540 previously generated test sets. It then will return different exit codes based 541 on the verification result.</p> 542 </div> 543 <div class="section" id="performing-an-automated-test"> 544 <h1>Performing an automated test</h1> 545 <ul class="simple"> 546 <li>TxS (Test E[x]ecution Service) has to be up and running (part of the 547 Validation Kit) on the guest.</li> 548 <li>Invoke the tdAudioTest.py test driver, either manually or fully automated 549 via Test Manager.</li> 569 <li>Only the HDA device emulation has been verified so far.</li> 570 <li>Only the ALSA audio stack on Debian 10 has been verified so far. 571 Note: This is different from PulseAudio using the ALSA plugin!</li> 550 572 </ul> 551 573 </div> … … 555 577 <li>Make sure that audio device emulation is enabled and can be used within the 556 578 guest. Also, audio input / output has to be enabled, depending on the tests.</li> 557 <li>Make sure that the guest's V KAT instance can reach the host via the selected558 t ransport lay(TCP/IP by default).</li>579 <li>Make sure that the guest's VBoxAudioTest's instance can reach the host via 580 the selected transport layer (TCP/IP by default).</li> 559 581 <li>Increase the hosts audio logging level 560 582 (via extra-data, set "VBoxInternal2/Audio/Debug/Level" to "5").</li> 561 <li>Increase VKAT's verbosity level (add "-v", can be specified multiple times).</li> 562 <li>Check if the the VBox release log contains any warnings / errors with the 583 <li>Increase VBoxAudioTest's verbosity level (add "-v", can be specified 584 multiple times).</li> 585 <li>Check if the VBox release log contains any warnings / errors with the 563 586 "ValKit:" prefix.</li> 564 587 </ul>
Note:
See TracChangeset
for help on using the changeset viewer.
