F1 MIDI Web Controller
Web-based sound board mapped to Traktor Kontrol F1 with audio synthesis, waveform visualisation, and 3D reactive graphics.
The F1 MIDI Web Controller turns a browser into a full-featured sound board that responds to a physical Traktor Kontrol F1 controller via the Web MIDI API. Each pad triggers audio samples through Tone.js with real-time waveform visualisation, while Three.js renders reactive 3D graphics that respond to audio frequency data. A creative coding project that pushes the boundaries of what's possible in a browser.
Vite + Web MIDI API + Tone.js + Three.js
Hardware-Connected Audio Synthesis in the Browser.
A creative coding project connecting physical MIDI hardware to browser-based audio synthesis with real-time 3D visualisations that react to sound.
Project Overview
Bridging physical controllers with web audio and 3D graphics.
The Web MIDI API opened up a world of possibility for connecting hardware controllers to browser-based applications. This project maps every button, knob, and fader on the Traktor Kontrol F1 to a custom sound board built entirely in the browser. Tone.js handles audio synthesis and sample playback with low-latency scheduling, while a custom analyser node feeds frequency data into Three.js scenes that pulse, morph, and react to the music in real-time. The Vite build system enables hot-reload during development so tweaking audio parameters and visual responses is instantaneous. MIDI mapping is configurable through a visual interface, letting you remap any controller to any sound or visual parameter without touching code.
Web MIDI API and Hardware Integration
Real-time control of browser-based audio from physical controllers.
The Web MIDI API provides browser-level access to MIDI input and output devices connected via USB or Bluetooth. This application requests MIDI access on load, then listens for incoming MIDI messages from the Traktor Kontrol F1 — each button press, knob rotation, and fader movement generates a note-on/off or control change (CC) event. A configurable mapping layer translates these MIDI messages into application actions: pressing pad 1 might trigger a synth note, twisting knob 2 might adjust filter cutoff, moving fader 8 might control reverb depth. The mapping interface lets you build these connections visually, storing them in localStorage so your configuration persists across sessions.
Latency is critical for live performance. Unlike traditional DAW systems, the browser's audio context introduces variable latency depending on the operating system and browser implementation. To minimize perceived lag, the application uses Tone.js's scheduling system which quantizes incoming MIDI events to the audio clock, ensuring beat-aligned timing. Audio samples use short buffers (256 samples at 44.1kHz = 5.8ms latency), and the visualization updates are decoupled from audio processing so visual frame drops don't affect sound. The result is responsive, sub-20ms latency comparable to hardware synthesizers, making the system usable for live performance.
Audio Synthesis and Waveform Visualization
Real-time audio synthesis with visual feedback.
Tone.js provides high-level audio synthesis primitives — synths, samplers, effect racks — that simplify music programming compared to the Web Audio API's lower-level nodes. The F1 controller features multiple synth modes: a subtractive synth with filter and envelope controls, a wavetable synth with morphable waveforms, and a drum machine with drum sample playback. Each synth is fully assignable from the MIDI mapping interface, so you can wire the F1's pads to play different octaves or samples, and the knobs to modulate filter cutoff, attack/release envelopes, or delay time. The effect rack includes reverb, delay, compression, and EQ, each with MIDI-assignable parameters.
Audio-Reactive 3D Visualization
Three.js scenes responding to frequency and amplitude data.
The audio visualization uses the Web Audio API's analyser node to extract real-time frequency and waveform data. This data feeds into Three.js scenes where geometries morph, colors shift, and particle systems emit based on what's being played. A frequency spectrum display shows the classic equalizer-style bars, while a waveform display scrolls horizontally showing the raw audio output. The 3D scene includes a central geometry that rotates based on frequency content — bass frequencies spin it slowly, mid-range frequencies accelerate it, and high frequencies cause rapid rotation. Particle emitters burst on drum hits (detected via threshold analysis of audio amplitude), creating the sense that sounds are physically manifesting in the 3D space.
The visualization code runs on a separate animation loop from the audio processing, allowing smooth 60fps visuals even if audio processing occasionally stutters due to garbage collection or system load. The analyser node provides two data sources: the frequency spectrum (via FFT analysis) and the raw waveform. The spectrum data drives the height of visualization elements and color shifts, while waveform data drives more subtle animations like geometry vertices following the wave. Custom shaders apply bloom and glow effects to make the visuals feel energetic and modern, and the whole scene is reactive enough to feel like the music is driving the visuals rather than vice versa.
Creative Coding as a Development Tool
Using creative projects to explore web platform capabilities.
The F1 MIDI controller was as much about exploring the boundaries of what's possible in a browser as it was about building a functional tool. By pushing the Web MIDI API, Web Audio API, and WebGL through a single cohesive project, the learning goes deep. You discover latency trade-offs, browser differences, optimization techniques, and new mental models for how to architect audio applications. This kind of exploratory work sharpens your engineering skills across the full stack — from low-level audio processing through to high-performance graphics. For portfolio purposes, a project that combines multiple cutting-edge web APIs into a polished, functional tool demonstrates mastery-level capability to potential clients and employers.
Creative Coding and Artistic Expression
Using web technologies for music and art production.
The F1 MIDI controller project exists at the intersection of software engineering and artistic expression. Web technologies are typically thought of as tools for business applications — e-commerce, dashboards, marketing sites. But they're equally capable of enabling creative tools and instruments. The Web MIDI API, Web Audio API, and WebGL together provide a complete toolkit for interactive music and generative art. This project demonstrates that a browser can be a legitimate creative workspace, not just a consumption device. For musicians and producers, having access to professional-grade synthesis, effects processing, and visualization without leaving the browser is liberating. The latency performance is comparable to hardware synthesizers, making it genuinely useful for performance rather than just a novelty.
The broader lesson is that technology is a medium for expression, not an end in itself. Whether you're building a CRM, designing a portfolio, or creating a music synthesizer, the underlying principles are the same: understand the user (or artist) needs, design the interaction thoughtfully, optimize for performance, and iterate based on feedback. The F1 MIDI controller demonstrates these principles applied to creative work rather than business applications. For a full-stack developer on the Gold Coast looking to differentiate beyond typical commercial work, exploring creative applications of web technology opens up opportunities for gallery exhibitions, festival installations, and collaborations with musicians and artists that pure software engineering might not offer. This versatility — the ability to build business tools AND creative tools with the same technical foundation — is increasingly valuable in specialized markets.