Adam's Perspective

For the last month, I’ve had unlimited access to OpenAI Codex. It’s been such a weird experience! This post is about my experience, learnings, and reflections.

What I ended up shipping

Quite a lot. I’m sharing this early in the post to put the rest of it in perspective.

• PHP ports of libsqlite, LightningCSS, gitoxide, and markerPDF (Pandoc still in progress).
• WordPress Playground stuff: Playground Autosave, custom PHP extensions, next PHP version preview (8.6), Symfony in Playground example, <php-snippet> web component for interactive PHP code examples, profiling and perf improvements, a simpler Preview this PR in Playground GitHub Action.
• Native PHP extension for sqlite-database-integration and php-toolkit.
• PushMd to support git pull and git push from and to your site. It exports and imports your content as markdown.
• Measured the SQLite performance impact on WordPress core for a 5GB WooCommerce site.
• A bunch of unfinished or archived explorations: PHP to native binary compiler, ForkPress, pure PHP full-text indexer, static site generator, Blueprints v2 runner in TypeScript, SVN client in PHP, ~500 exploratory redesign of Playground, WordPress museum design for exploring old releases, and some more.

It took 500 billion tokens in total, most of them from GPT-5.5 xhigh fast. By the middle of the month, I was running around 100 Codex sessions 24/7.

But it didn’t start that way at all.

Here’s the story from the beginning:

At first, I wasted days running the LLM treadmill

At the beginning, I’ve been round-robin talking to 10 parallel sessions. That was counter-productive.

Usually, I choose a specific goal and keep a sharp perspective. With Codex, however I’ve kept switching between projects maybe every 3 minutes, losing my entire train of thought. I felt engaged. I felt productive. I was shipping code. And yet, I wasn’t achieving anything.

I’ve also felt disconnected from my colleagues. Codex kept me so engaged I’ve turned to human interaction much less than before.

I played a lot of Heroes of Might and Magic 3 as a teenager. Afternoons turned into evenings and then into nights as I kept telling myself just one more turn. Codex hooked me the same way.

It’s always one more prompt. Each prompt—a pull of the lever in the AI casino—carries the promise of a wish fulfilled. Most pulls only grant a slight bump in blood pressure, but every so often the algebraic genie conjures what I desire, and all the struggle seems worthwhile. Just like in a real casino.

Every pull of the lever made me feel more rushed, more isolated, more miserable. AI was supposed to save work, but I was working more than ever. And why? No one even asked me to do that. Still, my brain couldn’t resist. Something had to change.

Then, I’ve delegated larger tasks

Most of my Codex interactions amounted to micromanaging: it doesn’t work, you’ve built the wrong thing, yes it is possible, no, it’s not finished yet. So I thought: what if Codex could micromanage itself for me?

I was exploring a PHP to assembly compiler at a time. I’ve asked Codex to work for a week, use sub agents, and never stop until this huge project is finished. It did a great job. For about 55 minutes. Then it stopped. The next few prompts bought me ~30 minutes each. Okay, so that wasn’t going to work.

I’ve then switched to a Ralph loop. It’s a really simple device: while(true) { codex "the same prompt every time"; }. It worked fine for a short while, but most of my attempts drifted away from impactful work and towards building smoke tests, documenting, adding low-impact code. Bummer.

Next, I’ve tried an autonomous-loop skill I’ve created. It was a little bit better, but that’s it.

Then came the /goal feature. That one stays on track much more often than Ralph! I’ve successfully ran 8 hour, 24 hour, and even 72 hour goal sessions. This was cool! A lot of ForkPress got built that way. I still couldn’t get parts of it right with /goal, e.g. it just wouldn’t expand the conflict resolution UI to the extent that I’ve asked it, even with specific examples, screenshots, and creative threats.

Then, I’ve used cli-agent-orchestrator. It works well enough and I still use it today. And yet, I still couldn’t get it to follow directions over the long run. I wanted 10 concurrent sub-agents implementing prioritizing the most challenging features, reviewer agents, and a single integrator.

That worked for two weeks. Then the progress regressed, grew again, and finally got stuck. I’ve tried to get it unstuck for the next two weeks, to get just one more test to pass, but nothing worked. I discarded all that progress and started over.

By then, I’ve figured out Codex is incapable of managing itself and needs a deterministic harness. It needs a daemon to keep it from idling and dying. It must be reminded to do the most difficult work first. It needs frequent high-level feedback to prevent drifting. It needs to be constantly nudged to restart and re-engage sub-agents. And it needs a specific success metric from the beginning—I’ve used the number of passed PHPT tests from the upstream suite.

I’ve tried building my own harness scripts with sqlite as a data store, but Codex was very creative in defying my will. Very soon the harness became a complex full-time side-project that still failed—so I’ve abandoned it.

Most recently, Jon Surrell inspired me to pick up Gastown. It has worklanes, roles, message passing, reviewers, an integrator, watchdogs. My results are mixed. It is consistently porting popular libraries to PHP without my involvement. Great! But with PHP to assembly compiler, most sessions quickly become idle and I need to intervene.

Right now, I am exploring the great ideas Dan Luu shared with me recently. When the harness fails, I ask codex to figure out why and put a system in place to prevent the next failure. Then it fails again, and I do the same thing. And then again and again. So far it’s been extending the lifetime of the PHP compiler swarm pretty successfully, so maybe with a few more adjustment that swarm will finally become autonomous?

What worked well

• Tmux. Codex lies about using subagents. With tmux I’m always ctrl+b,w away from seeing what’s actually happening.
• Specific progress metrics. I’ve initially asked Codex to write progress to the PROGRESS.md in the repo. It did. The bars were growing. I was happy. But those progress figures were completely made up. Since then, I’ve used the number of ported, passed, and total upstream tests as progress metrics and got more useful figures.
• Addressing hardest work first. I am now giving Codex a list of the hardest tasks in those bigger projects, and asking it to only work on them. Otherwise, it drifts to other work, simpler work, mocks, long if/else lists that return exact values without doing the actual computational work, and other workarounds. I then monitor it for a bit and nudge if needed.
• Implementing the same thing n times. I’ve ran the same task 4 times or 500 times and chose a single best solution. Sometimes it made a difference between degrading the program’s performance and improving it by 25%. The models are probabilistic. It takes multiple samples to get both the good and the bad results.
• Parallelizing the work using n always-active lanes. Otherwise it drifts to doing work in a single session. That’s slow!
• Very fast prototyping. In our ForkPress work, Jan Jakeš and I explored git, building ZFS to WASM, Dolt, Doltlite, Building MySQL to WASM, forking SQLite, patching Turso, copy-on-write support in ReFS, XFS, and APS, bundling it all as a static binary, as a PHP extension, and shipping Windows, Linux, and Mac versions. We’ve picked up ideas every day, found their limitations, and discarded them looking for new ones. We would have never been able to go through all these ideas so quickly without LLMs.
• Brainstorming. I’ve asked Codex a few dozen variations of What copy-on-write techniques could I use on Windows, Mac, and Linux to clone a WordPress site running on SQLite? I’ve learned so much! Pressing it about minor inconsistencies was especially rewarding, as it often admitted to misrepresenting the truth, and then provided a fuller picture and links I could fact-check.
• Parallelizing the work using n always-active lanes. Otherwise it drifts to doing work in a single session. That’s slow! Having one git worktree per lane helps. Splitting the work into independent, zero-coordination lanes really helps..
• Using Codex for easy and mundane work:
  • Once the CI checks are green, tag a new release
  • Tell me if this CI failure is a known flake
  • Install WordPress Studio, Profile new site creation in this Windows VM. Measure each step, be granular, don’t accept a slow library as an answer. Keep drilling down to the level of system calls if needed, and report the root cause of slowness. Provide a text breakdown and a plot.
  • Split PR 3655 into a stack of smaller PRs that are be easier to review

What did not work well

• Hands-on talking with multiple Codex sessions. It’s exhausting, I can’t do any meaningful work, I lose sense of time, I stop talking to people. And, even though I know these things, it still sucks me in momentarily if I start.
• Hands-on talking to just one Codex session. All the waiting for response, micromanaging, frustration, it’s not helping me much. Oftentimes I can do the work faster than Codex, but I still ask it to fix commas and rename variables for me. I think long-running task harnesses are the way to go. They achieve more, and I can use their work at my convenience so I’m less reactive and more present with others.
• Coordinating multiple agents. That seems largely unsolved. Agents can use message queues, talk, adjust behavior, and yet the swarm struggles to converge on consistent results. I’m trying to keep different sessions working on independent areas of the project, otherwise they get stuck, struggle to integrate the work back into trunk, and require a lot more of my oversight.
• Producing a lot of code without a pre-existing test suite.
  • ForkPress is ~200k lines of code. I didn’t know how to tell when it’s ready. It worked pretty well in my testing, within the limitations of what it can do, but I’ve never read and understood the entire codebase. Codex said the test suite was comprehensive, the tests were all green, and I still didn’t know if it’s ready. I couldn’t meaningfully engage with the code. Codex was my only gateway to it.
  • My port of libsqlite to PHP passes a large list of libsqlite tests that were ported to PHP alongside it. I’m much more confident in it as I know the important parts are covered by tests.
• Some skills. Popular design skills produced worse designs. Code review skills produced less useful reviews. However, my skill for documenting the code improved the docblocks. Ditto for writing PR descriptions.
• Reviewing code. Codex makes me lazy. When I read the code and process it to understand what it does, I feel I’m engaging to a lesser capacity than before. I don’t know if that’s muscle atrophy or not feeling the same obligation as I feel towards a colleague who worked hard to write that code.

What do I make of it