Volume 01 · A public learning log · 2026—
I read the ideas, rebuild the small parts, and write down what actually changed my mind.
A note is not finished when it sounds clear. It is finished when an experiment leaves evidence behind.
The register
A 28-byte header, 58 MB of unnamed floats, one C inference loop, and the contract that holds them together.
Flash Attention, mixed precision, DDP, shards, checkpointing, and HellaSwag without pretending I trained GPT-2.
Tiny Shakespeare ablations for context, depth, and width—and the caveats that matter more than the leaderboard.
The complete Tiny Shakespeare baseline as an instrument: diagnostics, curve, checkpoint, and one revealing broken specimen.
Pre-LN, identity gradients, residual initialization, and why a block is two writes into one persistent stream.
From a loop mean to scaled causal self-attention, following every tensor shape through the implementation.
A shifted batch, 4,096 simultaneous next-character bets, and the first numerical check that tells me whether the model is alive.
PicGen Studio now has Prompt Examples that fill prompts with one click, and a proper ‘Add Reference’ button that lets you upload local images or pick from history. No more staring at a blank input box.
PicGen’s image generation used to be a simple form: type a prompt, get an image. I just shipped a Studio that turns the whole experience into a ChatGPT-like conversation — where you talk to an AI artist, iterate on images, and build a creative thread.
What started as a simple Docker rebuild turned into a full-day adventure — fighting uv venv permissions, freeing 13GB of disk, and debugging a Cloudflare Turnstile secret that refused to work.