260115--AudioMidiController

Status

Audio: Initializing...

MIDI: Initializing...

Audio Visualization

Audio Data

Volume: 0.00

Bass: 0.00

Mid: 0.00

Treble: 0.00

MIDI Data

Active Notes: None

Velocity: 0

Pitch Bend: 0.00

CC Values:

None

MIDI Devices

Controls

Documentation

How It Works

AudioMIDIController uses the Web Audio API to capture audio input (microphone or line-in) and the Web MIDI API to receive MIDI messages from connected devices (keyboards, controllers, etc.). The audio is analyzed using FFT (Fast Fourier Transform) to extract frequency bands (bass, mid, treble) and overall volume. MIDI messages are parsed to extract note events, control changes, and pitch bend.

Basic Setup

import AudioMIDIController from "/dailies/AudioMIDIController.js";

const controller = new AudioMIDIController({
  enableAudio: true,
  enableMIDI: true,
  smoothing: 0.8,
});

await controller.init();
controller.start();

Getting Data

// In your draw/animation loop
const audioData = controller.getAudioData();
// { volume: 0-1, bass: 0-1, mid: 0-1, treble: 0-1 }

const midiData = controller.getMIDIData();
// { activeNotes: [], velocity: 0-127, cc: {}, pitchBend: -1 to 1 }

Audio Visualization

import AudioVisualizer from "/dailies/AudioVisualizer.js";

const canvas = document.getElementById("audio-canvas");
const visualizer = new AudioVisualizer(canvas, {
  showOscillator: true,
  showBars: true,
});

// In animation loop
const audioData = controller.getAudioData();
visualizer.update(audioData);
visualizer.draw();

Reacting to Audio

function draw() {
  if (controller.isAudioReady()) {
    const audio = controller.getAudioData();
    
    // Map volume to parameter
    sketchConfig.noiseScale = 0.05 + (audio.volume * 0.5);
    
    // Map bass to size
    sketchConfig.dotSize = 1.0 + (audio.bass * 2.0);
  }
}

Reacting to MIDI

function draw() {
  if (controller.isMIDIReady()) {
    const midi = controller.getMIDIData();
    
    // Change size based on note count
    if (midi.activeNotes.length > 0) {
      const sizes = [4, 8, 16, 32, 64];
      sketchConfig.matrixSize = sizes[midi.activeNotes.length - 1] || 4;
    }
    
    // Use velocity for speed
    sketchConfig.speed = 0.002 + (midi.velocity / 127) * 0.05;
    
    // Use CC values
    if (midi.cc[1] !== undefined) {
      sketchConfig.brightness = midi.cc[1] / 127;
    }
  }
}

Browser Compatibility

Troubleshooting