Canvas 2D to PixiJS: A Migration Story

A few months ago I asked myself a question I genuinely couldn't answer with confidence: is Clypra's editor actually rendering through the GPU pipeline I built, or have I been staring at a Canvas 2D fallback this whole time?

I'd built a whole PixiJS-based effects system for Clypra Studio — filters, transitions, video effects, all running as real GLSL shaders. It worked. It looked good in demos. And then I sat down to trace whether the Desktop Editor — the actual product people would use to cut and export video — was using any of it.

It wasn't. Not fully. Not even close, in places I was most confident about.

What followed was months of tracing render paths, naming a bug pattern I kept tripping over, studying how Premiere and Resolve actually solve the problems I was hand-rolling badly, and slowly earning the right to say Clypra's preview is GPU-accelerated end to end. This is the long version of that story — the bugs, the research, the dead ends, and where it's headed next.

Why Canvas 2D stops working the moment you have more than one thing on screen

Canvas 2D is fine for a single overlay. It falls apart the moment you're compositing multiple video layers, each with their own effects, at 1080p or higher, sixty times a second.

Here's the mechanism, concretely. ctx.drawImage(videoElement, ...) requires the CPU to copy the current decoded video frame into the canvas's backing store, every tick, per layer. Any per-pixel effect — a glitch filter, an RGB channel split — usually means getImageData/putImageData, which forces a full GPU-to-CPU pixel readback. That's one of the most expensive individual operations you can perform in a browser, and my early architecture was doing it in a loop, once per effect, per clip, per frame.

PixiJS flips this entirely. VideoSource uploads the decoded frame straight into GPU memory as a texture — many browsers have an accelerated path that never touches the CPU at all. A GLSL fragment shader runs the same per-pixel math in parallel, across potentially thousands of GPU cores, instead of a JavaScript loop touching every pixel sequentially. PixiJS also batches sprites that share a texture into a single draw call, collapsing what would be many separate CPU-side operations into one GPU submission.

None of this is a novel insight. It's why Premiere's Mercury Playback Engine, Resolve's Fusion pipeline, and Final Cut's Metal-based renderer all do GPU compositing natively rather than through a 2D canvas. I was just late to admit my own architecture hadn't caught up to that reality yet.

The bug that actually took the app down

The scariest bug wasn't a rendering bug at all — it was a lifecycle bug that froze the entire browser tab, and finding it taught me more about async ownership than anything else in this project.

I'd fixed one WebGL context-loss issue by making a renderer's destroy() method more aggressive about releasing GPU resources — calling WEBGL_lose_context's loseContext() explicitly, rather than trusting garbage collection to eventually free the context. That fix introduced a worse bug. React Strict Mode, in development, synchronously mounts a component, tears it down, and remounts it — a deliberate stress test for exactly this kind of lifecycle bug. My "more aggressive" destroy call fired during the phantom teardown, calling loseContext() on a canvas element that the second, surviving mount was about to reuse. A canvas only ever hands out one real WebGL context for its entire lifetime — forcing that context to die didn't just clean up the phantom instance, it permanently poisoned the canvas for the real one too.

The console told the story clearly once I knew what to look for: WebGL context lost — attempting recovery, fired twice, right alongside two duplicate "Font system initialized" logs — unambiguous proof that two Application instances had briefly existed against one canvas. Chrome enforces a hard cap on simultaneous WebGL contexts per page. Cross it, and the browser forcibly reclaims the oldest one to free GPU resources. Once that happens, shader compilation starts failing, gl.getProgramInfoLog() returns null, and the render loop hangs trying to recover from a context that is never coming back — because there's nothing left to recover to.

The actual fix wasn't "destroy more carefully." It was recognizing that destruction was happening on the wrong instance entirely. I built a canvas-keyed singleton registry — a WeakMap from canvas element to renderer instance — so a phantom Strict Mode remount recognizes an existing owner and reuses it instead of racing to create a second one. Only a genuine final unmount, confirmed by generation-counting across the synchronous replay, is allowed to actually call destroy(). Once I had that pattern, I found myself wanting to apply the exact same discipline everywhere a GPU resource got created: mount once, update every frame, tear down exactly once, on purpose, never by accident.

What professional editors already knew that I had to learn the hard way

One bug in particular sent me down a genuinely useful research rabbit hole. A user sent a screenshot: a video clip rendering as a tiny box in the middle of the frame instead of filling it, with the selection handles drawn at a completely different size than the actual pixels.

The root cause was two compounding bugs. First, sizing was computed from videoWidth/videoHeight read live, at whatever moment the code happened to run — sometimes before the video's loadedmetadata event had even fired, meaning the dimensions were still zero. Second, an anchor mismatch: PixiJS sprites default to a top-left anchor point, matching old Canvas 2D drawImage semantics, but the transform-handle system needed a center anchor to support rotation-around-center. Somewhere in the migration, the anchor got changed to center without the position math being updated to compensate — so every sprite silently shifted up and left by half its own dimensions.

Before writing a fix, I went and read how Premiere, DaVinci Resolve, and Final Cut Pro actually solve this, because I suspected I was reinventing a wheel badly. I was right. Every one of them treats a clip's size relative to its canvas as an explicit, named, stored property — never something recalculated live from a possibly-unready source during rendering. Final Cut calls it Spatial Conform: Fit, Fill, or None, chosen once and stored, editable later but never implicitly guessed. DaVinci Resolve's node-based compositor makes the rule explicit at the architecture level — in a Merge node, whichever input is connected as the background defines the output resolution, and the foreground gets fit into that space, full stop. Resolve's Transform node is deliberately resolution-independent: it always references a clip's defined source resolution, captured once, never re-read live — which is exactly the discipline my bug was missing.

That research turned into an actual system: a ClipConform record storing mode (fit, fill, or none), sourceWidth/sourceHeight captured exactly once via a loadedmetadata listener, and user-adjustable scale and offset layered on top. One function, resolveConform(), became the single place fit math is allowed to happen — no more ad hoc position calculations scattered across render call sites, each one a fresh opportunity for the same bug to reappear in a new shape.

The hang that taught me to stop trusting my own assumptions about "cheap"

Stacking two video clips on overlapping tracks — a completely ordinary edit — started making the whole preview stutter and occasionally hang outright. I assumed it was a rendering complexity problem. It was actually a caching problem, hiding in code I'd written to fix a caching problem.

The compositor rebuilt every filter and effect from scratch, every single frame, for every clip — new AdjustmentFilter(), new BlurFilter(), fresh GPU shader construction, sixty times a second, with the previous frame's instances never explicitly destroyed. Shader compilation is one of the most expensive synchronous operations WebGL can perform. Doing it once a frame for one clip stayed just under the threshold of "obviously janky." Doing it for two simultaneously active clips doubled that cost in the exact frame budget that was already tight — which is precisely why stacking clips, and only stacking clips, triggered visible stutter.

Layered on top of that: a media-sync function was firing unconditionally on every single animation frame during playback, regardless of whether the set of active clips had actually changed. There was already a signature computed elsewhere — a hash of exactly which clips were currently active — that existed specifically to answer "did anything actually change." It just wasn't being used to gate this call at all.

The fix for the first problem was a structural key: cache filter instances per clip, keyed on which effects are structurally active, not their live parameter values — rebuild only when an effect is added or removed, update uniforms in place every other frame. The fix for the second was using the hash that already existed instead of computing a new one nobody asked for. Neither fix was clever. Both were things I'd have caught immediately if I'd asked "what does this actually cost, every single frame" before writing the first version, instead of after a user reported the app hanging.

Building toward something bigger than PixiJS

Here's the honest part: PixiJS is the right tool for where Clypra is right now. It's not a compromise — CapCut's own web tooling runs on comparable pipelines, and for an editor competing at that tier, general-purpose WebGL compositing is genuinely sufficient, not a placeholder I should feel uneasy about.

But it has a ceiling, and I want to be upfront about where it sits. Real professional-tier tools — Resolve, Premiere, Avid — don't render through a browser-embedded GPU context at all. They talk directly to the native graphics driver: Metal, DirectX, sometimes Vulkan. That buys them things a browser context structurally can't give you for free: 10-bit and HDR color pipelines, ten-plus simultaneous 4K layers without breaking a sweat, and rock-solid multi-hour session stability that doesn't have "the browser tab decided to reclaim your WebGL context" as a category of failure.

The clearest proof this ceiling is real, and crossable: Figma. It's the best-known case of a creative tool built on browser GPU rendering that actually scales to genuinely professional use — and they didn't reach for an off-the-shelf library to get there. They built a fully custom WebGL renderer, specifically because general-purpose scene-graph engines don't have the domain-specific memory management a massive-scale creative tool needs.

That's the honest long-term plan for Clypra. Not "replace PixiJS because it was wrong" — grow a native Rust compositing layer, most likely via wgpu, that handles production-grade rendering and export once Clypra's ambitions genuinely push past what a general-purpose library can give. And the part I'm most relieved about: none of the GLSL effect work is wasted if that day comes. GLSL translates to WGSL far more cleanly than rebuilding an entire effect library from first principles. The shader logic is portable, even if the rendering backend underneath it eventually isn't permanent.

What I'd actually tell someone starting this migration

Audit before you assume. I thought I knew what my own renderer was doing. I was wrong about several load-bearing pieces of it, including the one — transitions — I'd been actively designing new features against for weeks. The only way I found out was tracing real call paths, not trusting file names, import statements, or my own memory of what I'd built.

Name the pattern the second time you see it, not the eighth. Split-brain became a search query, not just a diagnosis, the moment I had a name for it. I stopped waiting for the next screenshot to hand me the ninth instance and started actively hunting for it with a taxonomy in hand.

Don't build the safety net you don't need yet. I spent real effort designing a dual-pipeline architecture — feature flags, gradual rollout, automatic fallback to Canvas 2D if the WebGL path failed — before remembering Clypra doesn't have real production users yet. That entire apparatus was solving a problem I didn't have. The moment I admitted that, the plan simplified from "careful phased rollout" to "just finish the migration properly, all at once." The more cautious-looking plan isn't always the right one — sometimes it's just the more comfortable one.

A shader recompiled every frame will always cost more than you think it should. Twice, separately, the actual bug was "this is more expensive per-frame than it looks," not "this logic is wrong." I've started asking that question first now, before writing the first version of anything that runs inside a render loop.

If you're mid-migration on something similar — Canvas to WebGL, or any "two systems quietly disagreeing" bug of your own — I'd genuinely like to hear what pattern you found repeating on you. Mine was split-brain. What's yours?