ML Subsystem Integration Guide

Version: 1.0 Last Updated: 2025-10-15

---

🎯 Purpose

This guide explains how to integrate the Machine Learning subsystem (05_26_MACHINE_LEARNING) with other AudioLab subsystems and external applications.

---

🏗️ Integration Architecture

High-Level Overview

┌─────────────────────────────────────────────────────────────┐
│                    APPLICATION LAYER                         │
│  (JUCE Plugins, Standalone Apps, CLI Tools)                 │
└───────────────────────────┬─────────────────────────────────┘
                            │
┌───────────────────────────▼─────────────────────────────────┐
│              05_11_GRAPH_SYSTEM (Audio Graph)               │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌──────────┐   │
│  │ DSP Node │─→│ ML Node  │─→│ DSP Node │─→│  Output  │   │
│  └──────────┘  └──────────┘  └──────────┘  └──────────┘   │
└───────────────────────────┬─────────────────────────────────┘
                            │
┌───────────────────────────▼─────────────────────────────────┐
│           05_26_MACHINE_LEARNING (ML Subsystem)             │
│  ┌────────────────┐  ┌──────────────┐  ┌────────────────┐  │
│  │ ML Framework   │  │ Noise Reduce │  │ Classification │  │
│  │ (ONNX/TFLite)  │  │ (RNNoise)    │  │ (VGGish)       │  │
│  └────────────────┘  └──────────────┘  └────────────────┘  │
│  ┌────────────────┐  ┌──────────────┐  ┌────────────────┐  │
│  │ Source Sep     │  │ Audio Gen    │  │ Preset Gen     │  │
│  │ (Spleeter)     │  │ (DDSP)       │  │ (Recommender)  │  │
│  └────────────────┘  └──────────────┘  └────────────────┘  │
└───────────────────────────┬─────────────────────────────────┘
                            │
┌───────────────────────────▼─────────────────────────────────┐
│              05_04_DSP_PROCESSING (DSP Core)                │
│  FFT, Resampling, Audio Buffers, Signal Processing         │
└─────────────────────────────────────────────────────────────┘

---

🔌 Integration Points

1. Graph System Integration (05_11_GRAPH_SYSTEM)

ML models can be added as processing nodes in the audio graph.

Creating an ML Node

#include "AudioGraph/AudioGraph.h"
#include "MLFramework/MLFramework.h"
#include "MLNoiseReducer/MLNoiseReducer.h"

// Create ML node
class MLNoiseReductionNode : public AudioGraphNode {
public:
    MLNoiseReductionNode() {
        reducer_.initialize("models/rnnoise.onnx");
        reducer_.setReductionStrength(0.8f);
    }

    void processBlock(const float* input, float* output, int numSamples) override {
        reducer_.denoiseRealTime(input, output, numSamples);
    }

private:
    MLNoiseReducer reducer_;
};

// Add to graph
auto graph = std::make_unique<AudioGraph>();
auto ml_node = graph->addNode<MLNoiseReductionNode>("ml_noise_reduction");

// Connect: Input → ML Node → Output
graph->connect(input_node, ml_node);
graph->connect(ml_node, output_node);

Complete Example: ML Processing Chain

// Create graph
AudioGraph graph;

// Add nodes
auto input = graph.addInputNode("audio_input");
auto classifier = graph.addNode<MLClassifierNode>("classifier");
auto noise_reducer = graph.addNode<MLNoiseReductionNode>("denoiser");
auto compressor = graph.addNode<CompressorNode>("compressor");
auto output = graph.addOutputNode("audio_output");

// Connect chain
graph.connect(input, classifier);
graph.connect(classifier, noise_reducer);
graph.connect(noise_reducer, compressor);
graph.connect(compressor, output);

// Conditional routing based on classification
classifier->setCallback([&](ClassificationResult result) {
    if (result.label == "speech") {
        noise_reducer->setReductionStrength(0.9f);
        compressor->setRatio(4.0f);
    } else if (result.label == "music") {
        noise_reducer->setReductionStrength(0.3f);
        compressor->setRatio(2.0f);
    }
});

// Process audio
graph.process(audio_buffer, num_samples);

---

2. DSP Processing Integration (05_04_DSP_PROCESSING)

ML models often need DSP operations for feature extraction and post-processing.

Feature Extraction for ML

#include "DSPCore/FFT.h"
#include "DSPCore/SpectralOps.h"
#include "MLFramework/MLFramework.h"

class AudioFeatureExtractor {
public:
    // Extract mel-spectrogram for ML input
    Tensor extractMelSpectrogram(const std::vector<float>& audio) {
        // FFT
        auto stft = fft_.computeSTFT(audio, window_size_, hop_size_);

        // Mel filterbank
        auto mel_spec = spectral_ops_.applyMelFilterbank(stft, num_mels_);

        // Log scale
        for (auto& val : mel_spec) {
            val = std::log(val + 1e-6f);
        }

        // Convert to Tensor
        return Tensor(mel_spec, {1, num_frames_, num_mels_});
    }

private:
    FFT fft_;
    SpectralOps spectral_ops_;
    int window_size_ = 2048;
    int hop_size_ = 512;
    int num_mels_ = 128;
};

ML + DSP Hybrid Processing

// DDSP: Differentiable DSP combines ML and classical DSP
class DDSPProcessor {
public:
    std::vector<float> process(const std::vector<float>& input) {
        // 1. ML inference: Extract interpretable parameters
        auto params = ml_encoder_.extractParameters(input);

        // 2. Classical DSP: Synthesize audio
        auto harmonics = harmonic_synth_.synthesize(
            params.harmonic_amplitudes,
            params.harmonic_frequencies
        );

        auto noise = noise_gen_.generate(params.noise_filter_coeffs);

        // 3. Combine
        std::vector<float> output(harmonics.size());
        for (size_t i = 0; i < output.size(); ++i) {
            output[i] = harmonics[i] + noise[i];
        }

        return output;
    }

private:
    MLEncoder ml_encoder_;            // ML component
    HarmonicSynthesizer harmonic_synth_;  // DSP component
    NoiseGenerator noise_gen_;            // DSP component
};

---

3. Preset System Integration (05_14_PRESET_SYSTEM)

ML-generated presets integrate with the preset management system.

AI Preset Generation

#include "PresetSystem/PresetManager.h"
#include "MLPresetGenerator/PresetGenerator.h"

class AIPresetManager {
public:
    // Generate preset from reference audio
    Preset generatePreset(
        const std::string& reference_audio_path,
        const std::string& effect_type
    ) {
        auto reference_audio = loadAudio(reference_audio_path);

        // Use ML to generate parameters
        auto params = preset_gen_.generateFromAudio(
            getProcessor(effect_type),
            reference_audio,
            input_audio_
        );

        // Create preset
        Preset preset;
        preset.name = "AI Generated - " + getTimestamp();
        preset.category = "AI";
        preset.parameters = params;
        preset.metadata["generated_by"] = "ML";
        preset.metadata["reference"] = reference_audio_path;

        // Save to preset system
        preset_mgr_.savePreset(preset);

        return preset;
    }

    // Recommend presets based on audio content
    std::vector<Preset> recommendPresets(const std::vector<float>& audio) {
        return preset_gen_.recommendPresets(audio, 5);
    }

private:
    PresetGenerator preset_gen_;
    PresetManager preset_mgr_;
    std::vector<float> input_audio_;
};

---

4. Performance Variants Integration (05_16_PERFORMANCE_VARIANTS)

ML inference can be optimized using SIMD and GPU acceleration.

SIMD-Optimized Inference

#include "PerformanceVariants/SIMDVariants.h"
#include "MLFramework/TensorOps.h"

// Use SIMD for tensor operations
class OptimizedMLProcessor {
public:
    void processTensor(Tensor& tensor) {
        float* data = tensor.data<float>();
        int size = tensor.size();

        // Use AVX2 for vectorized operations
        #ifdef __AVX2__
        AVX2TensorOps::applyActivation(data, size, ActivationType::ReLU);
        #else
        ScalarTensorOps::applyActivation(data, size, ActivationType::ReLU);
        #endif
    }
};

GPU Acceleration

// Configure ML engine to use GPU
ModelConfig config;
config.model_path = "models/source_separation.onnx";

#ifdef __CUDA_AVAILABLE__
config.provider = ExecutionProvider::CUDA;
#elif defined(_WIN32)
config.provider = ExecutionProvider::DirectML;
#elif defined(__APPLE__)
config.provider = ExecutionProvider::CoreML;
#else
config.provider = ExecutionProvider::CPU;
#endif

auto engine = createInferenceEngine(ModelFormat::ONNX);
engine->initialize(config);

---

5. AI Orchestrator Integration (05_25_AI_ORCHESTRATOR)

The AI Orchestrator manages multiple ML models and coordinates their execution.

Model Orchestration

#include "AIOrchestrator/ModelOrchestrator.h"

class MLPipeline {
public:
    void setupPipeline() {
        orchestrator_.loadModel("classifier", "models/classifier.onnx");
        orchestrator_.loadModel("noise_reducer", "models/rnnoise.onnx");
        orchestrator_.loadModel("source_separator", "models/spleeter.onnx");

        // Define execution order
        orchestrator_.addDependency("noise_reducer", "classifier");
        orchestrator_.addDependency("source_separator", "noise_reducer");
    }

    void processAudio(const std::vector<float>& audio) {
        // Orchestrator handles execution order and resource management
        auto result = orchestrator_.execute("pipeline", {
            {"input", audio}
        });

        auto stems = result["stems"];
    }

private:
    ModelOrchestrator orchestrator_;
};

---

🛠️ Build Integration

CMake Configuration

# Your project's CMakeLists.txt

# Find ML subsystem
find_package(AudioLabML REQUIRED)

# Link ML libraries
add_executable(my_audio_app main.cpp)

target_link_libraries(my_audio_app
    PRIVATE
        AudioLab::ML::Framework
        AudioLab::ML::NoiseReduction
        AudioLab::ML::Classification
        AudioLab::ML::SourceSeparation
)

# Copy model files to output directory
add_custom_command(TARGET my_audio_app POST_BUILD
    COMMAND ${CMAKE_COMMAND} -E copy_directory
        ${CMAKE_SOURCE_DIR}/models
        $<TARGET_FILE_DIR:my_audio_app>/models
)

Model Deployment

models/
├── rnnoise.onnx               # Noise reduction
├── vggish.onnx                # Classification
├── spleeter_4stems.onnx       # Source separation
├── ddsp_violin.onnx           # Audio generation
└── preset_recommender.onnx    # Preset generation

---

🧪 Testing Integration

Unit Tests with ML Models

#include <gtest/gtest.h>
#include "MLNoiseReducer/MLNoiseReducer.h"

TEST(MLNoiseReductionTest, BasicFunctionality) {
    MLNoiseReducer reducer;
    ASSERT_TRUE(reducer.initialize("models/rnnoise.onnx"));

    // Generate test signal
    std::vector<float> noisy_audio = generateNoiseSignal(16000);

    // Denoise
    auto clean_audio = reducer.denoise(noisy_audio);

    // Verify output
    EXPECT_EQ(clean_audio.size(), noisy_audio.size());
    EXPECT_GT(computeSNR(clean_audio), computeSNR(noisy_audio));
}

---

📊 Performance Monitoring

Profiling ML Inference

#include "MLFramework/Profiler.h"

MLProfiler profiler;
profiler.enable();

// Run inference
auto result = ml_engine->run(inputs, outputs);

// Get performance stats
auto stats = profiler.getStats();
std::cout << "Inference time: " << stats.inference_time_ms << " ms\n";
std::cout << "CPU usage: " << stats.cpu_usage_percent << "%\n";
std::cout << "Memory: " << stats.memory_mb << " MB\n";

---

🚀 Deployment Checklist

• All required ML models included in distribution
• ONNX Runtime / TFLite / LibTorch libraries packaged
• GPU drivers detected (CUDA, DirectML, Metal)
• Model loading tested on target platform
• Real-time performance validated
• Memory usage within limits
• Error handling for missing models
• Fallback to CPU if GPU unavailable

---

📚 Additional Resources

• ML Framework API Reference
• Model Training Guide
• Performance Optimization Guide
• Troubleshooting Guide

---

Last Updated: 2025-10-15 Version: 1.0