🚀 Testing Framework Quick Start¶

Get from zero to running tests in 5 minutes

Prerequisites¶

Before starting, ensure you have:

✅ CMake 3.20+ installed and in PATH
✅ C++ compiler (MSVC, GCC, or Clang)
✅ Catch2 installed via vcpkg
✅ vcpkg configured with AudioLab

Verify:

cmake --version        # Should show 3.20 or higher
vcpkg list catch2      # Should show catch2:x64-windows

Don't have these? See 03_03_dependency_management/INSTALLATION_GUIDE.md

Step 1: Navigate to Testing Framework¶

# Windows
cd "C:\AudioDev\audio-lab\2 - FOUNDATION\03_INFRA\03_04_testing_framework"

# macOS/Linux
cd ~/audio-lab/2\ -\ FOUNDATION/03_INFRA/03_04_testing_framework

Step 2: Configure with CMake¶

# Windows (MSVC)
cmake -S . -B build -DCMAKE_TOOLCHAIN_FILE=C:/vcpkg/scripts/buildsystems/vcpkg.cmake

# macOS/Linux
cmake -S . -B build -DCMAKE_BUILD_TYPE=Debug -DCMAKE_TOOLCHAIN_FILE=/path/to/vcpkg/scripts/buildsystems/vcpkg.cmake

Expected output:

-- The CXX compiler identification is MSVC 19.41.34120.0
-- Detecting CXX compiler ABI info - done
-- Found Catch2: 3.x.x
-- Configuring done
-- Generating done
-- Build files written to: build

Time: ~30 seconds

Step 3: Build Tests¶

# Windows
cmake --build build --config Debug

# macOS/Linux
cmake --build build -j 4

Expected output:

Building CXX object CMakeFiles/audiolab_test_utils.dir/03_04_02_test_utilities/test_signals.cpp.obj
Building CXX object tests/CMakeFiles/test_example_processor.dir/test_example_processor.cpp.obj
...
[100%] Built target test_example_processor

Time: 1-2 minutes (depending on CPU)

Step 4: Run Tests¶

Option A: Use Test Runner Script (Recommended)¶

# Windows
.\scripts\run_tests.ps1

# macOS/Linux
./scripts/run_tests.sh

Option B: Use CTest Directly¶

# From project root
cd build
ctest -C Debug --output-on-failure

# Or with verbose output
ctest -C Debug --verbose

Option C: Run Individual Test Binary¶

# Windows
.\build\tests\Debug\test_example_processor.exe

# macOS/Linux
./build/tests/test_example_processor

Expected output:

Running AudioLab Tests
Build: build | Config: Debug | Filter: *

▶️  Running CTest...

Test project C:/AudioDev/audio-lab/2 - FOUNDATION/03_INFRA/03_04_testing_framework/build
    Start 1: test_infrastructure
1/5 Test #1: test_infrastructure ............   Passed    0.02 sec
    Start 2: test_example_processor
2/5 Test #2: test_example_processor .........   Passed    0.15 sec
    Start 3: test_l4_framework
3/5 Test #3: test_l4_framework ..............   Passed    0.08 sec
    Start 4: test_l5_framework
4/5 Test #4: test_l5_framework ..............   Passed    0.12 sec
    Start 5: test_plugin_interfaces
5/5 Test #5: test_plugin_interfaces .........   Passed    0.06 sec

100% tests passed, 0 tests failed out of 5

✅ All tests passed!

Time: 5-10 seconds

Step 5: Understand Test Output¶

Successful Test¶

test_example_processor.exe
===============================================================================
All tests passed (42 assertions in 8 test cases)

Failed Test¶

test_example_processor.exe
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test_example_processor.exe is a Catch2 v3.x.x host application.
Run with -? for options

-------------------------------------------------------------------------------
SimpleProcessor processes audio correctly
-------------------------------------------------------------------------------
test_example_processor.cpp:45
...............................................................................

test_example_processor.cpp:58: FAILED:
  REQUIRE( output[i] == Approx(expected[i]) )
with expansion:
  0.500123 == Approx( 0.5 )

===============================================================================
test cases: 8 | 7 passed | 1 failed
assertions: 42 | 41 passed | 1 failed

Key information: - File: test_example_processor.cpp - Line: 58 - Failure: 0.500123 != 0.5 (exceeded tolerance)

Step 6: Run Specific Tests¶

By Test Name¶

# Run only tests with "unit" in the name
./build/tests/Debug/test_example_processor.exe "[unit]"

# Run only integration tests
./build/tests/Debug/test_example_processor.exe "[integration]"

# Run only benchmarks
./build/tests/Debug/test_example_processor.exe "[benchmark]"

By CTest Pattern¶

cd build

# Run only tests starting with "test_example"
ctest -R "test_example.*"

# Run all tests except benchmarks
ctest -E "benchmark"

List Available Tests¶

# List all test cases
./build/tests/Debug/test_example_processor.exe --list-tests

# List all tags
./build/tests/Debug/test_example_processor.exe --list-tags

Common Use Cases¶

Quick Smoke Test (< 10 seconds)¶

# Run only unit tests (fast)
cd build
ctest -R "test_infrastructure|test_example_processor" -C Debug

Full Test Suite (1-2 minutes)¶

# Run all tests
.\scripts\run_tests.ps1

Performance Benchmarks (5-10 minutes)¶

# Run only benchmarks
./build/tests/Debug/test_example_processor.exe "[benchmark]" -s

The -s flag shows detailed benchmark results.

Real-Time Safety Check (< 1 minute)¶

# Verify no allocations/locks in audio thread
./build/tests/Debug/test_example_processor.exe "[rt-safety]"

Troubleshooting¶

❌ "Catch2 not found"¶

Problem: CMake can't find Catch2

Solution:

# Install Catch2 via vcpkg
vcpkg install catch2:x64-windows

# Verify installation
vcpkg list catch2

# Reconfigure with vcpkg toolchain
cmake -S . -B build -DCMAKE_TOOLCHAIN_FILE=C:/vcpkg/scripts/buildsystems/vcpkg.cmake

❌ "test_example_processor.exe not found"¶

Problem: Tests weren't built

Solution:

# Rebuild
cmake --build build --config Debug --target test_example_processor

# Verify executable exists
ls build/tests/Debug/  # Windows
ls build/tests/        # macOS/Linux

❌ "All tests failed with 'DLL not found'"¶

Problem: Missing runtime dependencies

Solution (Windows):

# Copy vcpkg DLLs to output directory
cmake --build build --config Debug
cmake --install build --config Debug --prefix build/install

# Or add vcpkg/installed/x64-windows/bin to PATH
$env:PATH += ";C:\vcpkg\installed\x64-windows\bin"

❌ "Tests pass locally but fail in CI"¶

Common causes: 1. Floating-point differences (different CPU architectures) 2. Timing-dependent tests (CI may be slower) 3. Uninitialized variables (different compilers)

Debug:

# Run test repeatedly to find flaky tests
.\03_04_05_troubleshooting\run_test_repeatedly.ps1 -TestName "test_example_processor" -Iterations 50

# Run with sanitizers (Debug build)
cmake -S . -B build -DENABLE_ASAN=ON
cmake --build build
./build/tests/test_example_processor

❌ "Test takes too long"¶

Problem: Performance regression or inefficient test

Solution:

# Profile the test
# Windows: Use VTune or Tracy
# macOS: Use Instruments
# Linux: Use perf

# Example with perf (Linux)
perf record ./build/tests/test_example_processor
perf report

See 03_04_03_performance_testing/PROFILING_SETUP.md

Next Steps¶

✅ You've successfully run tests! Now what?¶

Beginner Path (1-2 hours): 1. TEST_CHEAT_SHEET.md - Learn common commands 2. examples/ - Study example tests 3. 03_04_05_troubleshooting/FAILURE_DIAGNOSIS.md - Debug failed tests

Intermediate Path (4-6 hours): 1. 03_04_00_test_frameworks/TEST_FRAMEWORK_PHILOSOPHY.md - Understand testing philosophy 2. 03_04_02_test_utilities/TEST_UTILITIES_GUIDE.md - Use test helpers 3. Write your first test for AudioLab code

Advanced Path (8-10 hours): 1. 03_04_03_performance_testing/PERFORMANCE_CRITERIA.md - Performance testing 2. 03_04_04_quality_testing/AUDIO_QUALITY_GUIDE.md - Golden file testing 3. 03_04_04_quality_testing/GOLDEN_FILES_MANAGEMENT.md - Manage reference audio

Quick Reference¶

Build & Run (Copy-Paste)¶

# Windows - Full workflow
cd "C:\AudioDev\audio-lab\2 - FOUNDATION\03_INFRA\03_04_testing_framework"
cmake -S . -B build -DCMAKE_TOOLCHAIN_FILE=C:/vcpkg/scripts/buildsystems/vcpkg.cmake
cmake --build build --config Debug
.\scripts\run_tests.ps1

# macOS/Linux - Full workflow
cd ~/audio-lab/2\ -\ FOUNDATION/03_INFRA/03_04_testing_framework
cmake -S . -B build -DCMAKE_BUILD_TYPE=Debug
cmake --build build -j 4
./scripts/run_tests.sh

Test Filtering¶

# Unit tests only (fast)
./test_example_processor "[unit]"

# Integration tests
./test_example_processor "[integration]"

# Benchmarks
./test_example_processor "[benchmark]" -s

# Real-time safety
./test_example_processor "[rt-safety]"

# Specific test case
./test_example_processor "SimpleProcessor processes audio correctly"

CTest Commands¶

cd build

# Run all tests
ctest -C Debug

# Run with verbose output
ctest -C Debug --verbose

# Run only failed tests
ctest -C Debug --rerun-failed

# Run in parallel (4 jobs)
ctest -C Debug -j 4

# List all tests
ctest -N

Time Budget Summary¶

Task	Time
Install Catch2 (if needed)	2-5 min
Configure with CMake	30 sec
Build tests	1-2 min
Run all tests	5-10 sec
TOTAL	5-10 min

Resources¶

README.md - Complete testing framework guide
TEST_CHEAT_SHEET.md - Command reference
Catch2 Docs - Official Catch2 documentation
CTest Docs - Official CTest documentation

You're now ready to run and write AudioLab tests! 🎧