Doodling across disciplines
This little bird is a doodle. It’s also a sophisticated piece of tech. Under the handmade drawing there's a skeleton rig and skinning system, and when you drag the bird across the canvas it walks, with every curve bending along with the bones.
The bird hatched in tldraw flash, the animation app we've been building for the past few months. Unlike most animation and motion graphics tools, tldraw flash has no keyframes and no frame-by-frame drawing. You grab your drawing, move it and record it. The interaction model is part whiteboard, part game, part video editor. Because few, if any, animation apps focus on direct manipulation of shapes we had to pull from a range of influences. Nearly every idea that makes tldraw flash work came from copying, combining, and remixing ideas across different fields.
Thinking across fields
Outside of tldraw, I'm an artist. The drawings and paintings I exhibit in galleries and museums are all created with the help of custom software I’ve built for use in my studio. Often, I’ll use the tldraw SDK as a starting point. I’m also a professor at the Rhode Island School of Design, where my Art & Computation courses have students invent their own bespoke software. BBuilding your own tools, you will create the most innovative art, animations, games and products.
This past year, Steve invited me to help his team continue to innovate on the infinite canvas. We started with experiments focused on agentic workflows, then began creating games and game-like interactions.
The first game-related experiment I built was a rigged character based on one of Steve's drawings. A rig is an invisible skeleton, the thing that lets you move an arm without redrawing it. I wired it up like a game character: WASD keys to walk, space to jump, and a few more keys for things like waving.
The most important decision was one you couldn't see. The rig didn't draw anything itself. It moved real tldraw shapes, which are just React components rendering SVG in the DOM. So the character never stopped being a drawing. You could still select him and edit him like anything else on the tldraw canvas. Everything that follows depends on that.
What if the skeleton was insanely sophisticated?
If a hidden rig could move shapes on the canvas, did the rig need to be 2D at all? To find out, I swapped in a three.js 3D skeleton. It worked. The skeleton could walk and turn in 3D space while puppeteering flat SVG shapes, and those shapes stayed ordinary tldraw shapes throughout. One of our interns, Leo, built some incredible custom rigging tools as well. These were all fun tricks without an obvious use, so we set them aside in the end. Remember it, though. It comes back.
Direct manipulation
In many ways, this project began with an observation about animation apps that, in my decades of experience, I hadn’t really thought about before. Why is the learning curve for animation software so high? And why are there no tools for making quick back-of-the-napkin animations to help tell stories and explain ideas?
One reason is that most animation tools are indirect. For instance, in After-Effects, or any timeline-based tool, you animate a walk by setting keyframes and tuning curves between them. You describe the motion from the outside, then scrub back to see whether you got it right.
Steve wanted nothing to do with that! tldraw flash has no keyframes and no conventional frame-by-frame animation. Like everything in tldraw, we focused instead on user actions such as click and drag. You don't describe the motion; you perform it. In fact, the record button doesn’t even record shape properties – it records the user’s actions (diffs) on the canvas (like undo/redo systems).
We started with techniques to record shape movement, and then realized we could use the rig. The rig was already a performance, just played on a keyboard, so I handed it to the cursor instead. Drag the character across the canvas and he walks along with you. Hit record, and that's your animation. The number keys trigger extras like waving, a technique carried over from game days.
Nobody draws a person in pieces
That left the hardest part: letting people draw their own characters.
Until recently, you had to draw each limb and the head as a separate shape, and tldraw flash snapped each one to the nearest bone. It often picked the wrong one. Worse than that it asked people to draw in a way nobody draws. When you doodle a person, you don't make a kit of parts. You just draw a character.
The fix was hiding in the 3D detour.
This is where the shelved 3D experiment paid off. AAA 3D game characters aren't assembled from parts snapped to bones. Their surface is one continuous mesh skin stretched over the skeleton, each point tied to the bones around it with different degrees of pull. The technique is called skinning, and it's how all characters bend in games, animated films, and special effects.
So we brought skinning to the tldraw primatives. Every point along every path gets pulled by the nearby bones. The closer to a joint, the stronger the pull. A point near the elbow doesn't have to choose between the upper arm and the torso. Both pull on it, so the line curves through the bend instead of breaking.
Scribble a figure however you like, even in one messy stroke, and it bends where it should.
Game engine bones, notebook doodles
Developing tldraw flash required thinking across fields, which is exactly what I teach at RISD: take a tool built for one thing, remix it with another and point it somewhere else. A game rig, combined with a whiteboard app, turned into an animation tool. A shelved 3D experiment taught a 2D doodle how to bend. And the technique that moves characters in games and films now works on the kind of guy you'd draw in the margin of a notebook. Serious machinery, charming drawing. That mismatch is my favorite thing about it.
David Epstein's book Range makes the case better than I can: the people who solve strange problems are often the ones who've wandered across a few fields and can carry an idea from one to another. Skinning only looked like the answer to a doodle problem because I'd spent years using tools for things they weren't built for.
What's changed is that artists and designers can now act on those instincts directly. LLM’s like Claude Code and Codex have collapsed the gap between having an offhand idea and building it. I felt this shift building the rig and skinning system, I'm watching it with the interns on our team, and I see it in my students in RISD's new Art and Computation major. That's a lot of cross-disciplinary minds heading into tech with the ability to build. The tech industry will be better for it.
Maybe it’s Steve’s background as an artist that has put tldraw at the cutting edge of this industry-wide shift in how products are engineered. Prototyping is not relegated to just one team, it is the core of the business, and tldraw flash is one particularly cool example of this.
Go try it at flash.tldraw.com. Make a doodle, drag it around, and show us what you get up to on Discord or X.
