Bits All The Way Down

Despite calling myself a systems programmer, I never really understood how CPUs worked. I'd made dozens of contributions to the Rust compiler, so I wasn't clueless. But the steps after compilation? Vague at best. That gap made me feel like a fraud.

The problem was, I can't learn technical topics without getting my hands and mind dirty. I have to visualize an idea, turn it into a three-dimensional object, rotate it, stretch it, reverse it. Until a topic reaches that point, I don't truly feel comfortable with it. I didn't know this about myself when I started this project.

I obsess over details, I get sidetracked. That's why I hesitated to start learning about CPUs. Then one day I had to deal with SIMD instructions. That was two years ago.

One thing about me is I like software to be fast, as fast as it can reasonably be. If you're the kind of programmer who cares about performance, one day you'll meet SIMD. Single instruction, multiple data. I'd heard the name but never had a reason to look closer.

In my day job, while working on a parser for our logging system, I had to use a Rust crate called memchr, a brilliantly fast crate for string searching. I wanted to make it even faster, so I cracked open the code, and after finding my way through the codebase, there it was: SIMD intrinsics hundreds of lines of them.

I knew right then this wasn't going to be a simple patch. I stared at the code for a few minutes, trying to make heads or tails of it. I didn't know what I was looking at. I opened the SIMD article on Wikipedia. Nope, that didn't help. Tried Rust SIMD examples. Didn't help either.

I got up and looked outside. The sun had already set. That night I watched SIMD tutorials and talks on YouTube while trying to fall asleep.

Fake work

Excitement mixed with anxiety. I had to learn how SIMD worked.

There was no other way. I was waking up thinking about it. I browsed Wikipedia, trying to see a path through the dark enclosure I'd found myself in.

I wasn't sure how to begin, so like every productive procrastinator, I started building my personal knowledge tree. I downloaded Obsidian, watched a few tutorials on how to create the best note-taking system. I learned notes could link to each other, so I linked everything to everything. The graph view drew my notes as little stars, all connected. It looked like a map of everything I was about to understand. I felt productive. I felt I was on the right track. I was writing things down. A week in, I had a dozen pages of markdown notes with beautiful highlights of important sections.

Then I opened the memchr code again, to check. I understood it exactly as well as before, which is to say, not at all.

I hadn't written a single line of code. All I had was these high-level notes, linked to each other, connected to nothing real.

I started to have doubts about my abilities. That led to questioning my choices, which led to more doubts and more anxiety. This pattern would repeat itself many times before the project was done.

Somewhere in that spiral, an old memory surfaced. Years ago, around the time I was first learning to program seriously, I'd stumbled across someone writing an N64 emulator in Rust. I couldn't understand a single thing he was doing, but I remember thinking it was the coolest thing I'd ever seen. And here I was, a decade later, hesitating to even start my own.

Partner in Crime

The N64 lit the fire, and naturally, my brain escalated. I said to myself, if I'm doing this, I should do something difficult. It somehow made sense to me at the time: I was already on macOS with an ARM chip, so why don't I just emulate an ARM system? And of course, it should be the latest one. That sounded like a plan.

I don't remember which AI model it was. What I do remember is opening a chat window and typing something like:

I knew it sounded insane, but deep down I felt this was the surest way to learn SIMD, CPUs, and everything about computers in one go. (Yes, C. I don't know what I was thinking.)

It gave a long and very bad answer. I opened a new chat window and asked again.

It gave me a bunch of words I didn't understand. Register. Stack pointer. Program counter. Exception levels. I started scolding the AI:

Every time it explained a term, the AI introduced more unknown terms. I felt like my brain was going to burst into flames.

But I wasn't going to give up. I just embraced the situation. Constant confusion felt bad, but stopping felt worse.

So I kept going. AI tools were a great help. Sure, they were wrong sometimes, but I could ask them the same question over and over without feeling judged. We became partners in crime. Just like that, I managed to create a CPU struct.

I needed something to actually start the engine. But how?

The ELF Detour

I said to the AI:

It told me, more or less:

Bootloader. ELF files. Two more terms I didn't know, and apparently I needed both before I could even think about Linux. I remember sighing and complaining back:

Looking back, what the AI said was wrong. Booting Linux directly would have been easier, and it would have saved me weeks. But it doesn't matter. This wasn't the first time I'd been bitten by an AI, and it wouldn't be the last.

I couldn't really wrap my head around ELF files, and the problem was I was looking at them from the wrong perspective. Since I'd never worked with things this low-level, I had this sense (until very recently) of something magical going on somewhere underneath. Things wouldn't connect.

The missing link was memory. Without understanding memory, nothing about any of this made sense. So I started asking questions like a nine-year-old. What even is a file? What even is a program? What, what am I even doing.

There Is No Magic

I began asking AIs again, trying to make sense of what this ELF thing was. They were talking about segments, program headers, sections. My own samsara of confusion → frustration → enlightenment started once again.

I was starting to notice a pattern about myself. Every time I tried to learn something difficult and pushed through it, I had to go through this cycle. There was no skipping it.

Eventually one of these conversations went like this:

That answer finally connected the pieces.

Bits are bits. It doesn't matter where they are or what they are. They can only be zero or one. We programmers decide how to interpret what they mean.

The word abstraction suddenly made sense.

Road Back

Realizing that bits were bits felt like a huge enlightenment. I assumed it would stop me from making the same kind of mistake again. It did not.

What followed was confusion → frustration → enlightenment, applied to one thing after another. Virtual memory. The MMU. The instruction set. Instruction encoding collisions. A lot of off-by-one errors. After the first twelve months I got used to the difficulty spikes, used to the cycle. I knew I was bound to make mistakes. This wasn't a project I could rush.

The harder lesson was about speed. For a very long time I thought this project was a race. I had to finish it as soon as possible, I was hurrying, because that's what I learned from a certain genre of online essay. You gotta go fast, they said. They were kind of right, but I completely misunderstood them. They meant: move fast, walk the longest road, there are no shortcuts, move.

I came back to memchr eventually. The SIMD code that used to look scary now looked like what it was: bits, just bits, doing data operations. It was still hard. But I could see what it was doing and why.

The project started because of SIMD, but it grew bigger than SIMD. I understood more about CPUs.

Almost two years later, the emulator boots a recent Linux kernel, and I'm writing a device driver for it. It runs neural networks now, on an accelerator I made up.

cepu-now - github.com/ouz-a/cepu