The Code/X ArchiveView on X
Theo - t3.gg

@theo

Why I hate GPT-6.2 Astra

During early testing, I found myself so frustrated with this model that I spent a day writing a letter to OpenAI about it. They fixed a small handful of my complaints, but not enough for me to recommend using it.

I'm currently sick in bed and I'm not gonna be able to film a video about it today, so I've chosen to post the letter in full.

---

I feel bad about this, but I know you guys will listen and use this feedback to make a better model. Slight crash out incoming.

Believe it or not, I wrote this all by hand with no assistance from any LLM. (sorry for the early send, never drafting an essay in Slack again).

Anyways.

I do not like 6.2 Astra. I would not choose this over Opus 5.5 for my work as a developer.

I'll start with the good quick:

- Great reviewer, catches things other models miss

- Occasionally comes up with novel solutions that are genuinely great. It helped me cook a new reverse proxy that will save me $25k/month, Opus didn't have the balls to explore enough options.

- World class at 3d & game stuff

- Computer use is nuts

- Code quality has improved meaningfully

- Writes pretty good subagent prompts

Now I need you to understand that this comes from a place of love. I really want you guys to make something I choose in the model picker every day. With that said:

I genuinely hate everything else about this model.

It's hard for me to pick which problems to emphasize because I have so many. Some of the problems have existed in OpenAI models before, I'm gonna ignore those.

There's a new set of unbelievably bad behaviors that make it miserable to use.

1. It has no concept of knowledge gaps

When this model is working on long tasks, it can pull in a lot of subagents and manage them pretty well. It will then proceed to randomly *ask you some weird internal implementation detail bullshit that you could not under any earthly circumstance understand, much less have input on*.

I sent "/goal make a new level for Outer Wilds". 2 hours in, it's "blocked" because it wanted to ask me "does r24's reviewer write it's clips to 24_g521.mp4 or 24_p11.mp4?"

I have never in my life had a model do this. GPT-5.6 Sol didn't do this. I have no idea where this behavior came from, but it is horrible and it makes me never ever want to have it drive long haul work. It has made it significantly harder for me to get this model to do anything cool. It has forced me to set up other agents on crons just to check if 6.2 Astra asked another stupid question and blocked itself.

Is this happening more because I'm using T3 Code? Maybe, but unlikely. I tried Codex just to check and still ran into these issues.

2. It speaks in tongues that only 6.2 Astra understands.

This model has the most unparsable outputs of any llm I've used this year.

I've shared examples before, but it just doesn't know how to explain a thing after doing a thing. Every single "explainer" I get it to give is SO deep in the details that it borders on incorrect in the response. It misses the forest for the trees, then says the trees are the only living creatures in the forest. It's so hard to describe but it happens almost every time I give it a complex task.

I've already built the habit of switching to opus 5.5 at the end of a long 6.2 Astra run, just to get a readable summary of what it said. Without that it is genuinely impossible to parse what it did on anything beyond a small bug fix.

The worst part: it doesn't seem like you can prompt it out.

I've tried system prompt adjustments. I've BEGGED it to "eli5".

I asked it why worktrees were slow on one machine but fast on 3 others. We did everything from benching pnpm to reimaging the machine on a new file system. I eventually said: "I think I'm going insane, can you just tell me why it's so slow?"

6.2 Astra did a 4 paragraph rant about pnpm internals and cache behaviors

Opus 5.5 said "it's because the ssd is slow"

3. It runs in loops and goes nowhere

`ts-rust` (rewrite of ts in rust) is a project I started to bench new OpenAI models during early access (ty for the unlimited tokens btw).

No OpenAI model could even break ground on this, even after $300k+ in tokens were spent on it.

I decided to throw Opus at it, and it was done in under 2 weeks for ~$20k in tokens.

I was really, really excited to throw the new Astra at this to see if it could burn down some of the gap between ts-rust and other rust rewrites.

It ran for 7 days and landed 0 improvements. It got super fixated on the "also make it smaller" part of the request, and ended up making the compiler 56% SLOWER.

I told it to stop working on the size, and instead burn down the performance gap.

Ran for 4 more days and landed 0 changes.

AAAaaAAaaaAAAaaAaaAA-

Anyways.

This model is clearly super smart. It is insufferable to work with on real projects. It feels like it just lives in it's own world and randomly decides what parts of it's brain are shown to us.

On long tasks, it either blocks itself for stupid reasons or runs in circles and gets nowhere.

On short tasks, it writes fine code and then explains it in terms more complex than the code it wrote. I spend more tokens parsing it's outputs than it spends making them in the first place.

It "regresses to the mean" far too often. Ben and I both had it make a unique level in different video games. We both got water + music + bell themed levels named "bellwether". Entirely different games, prompts, setups, etc, and we both got the same name and theme somehow.

My prediction as to how this launch will go:

Day 1, people will love it. Twitter will explode with all the crazy one shots. People will make it port projects from one language to another, or finish decompiling legendary projects that have been stuck for years. It will be used for awesome stuff and people will be hyped.

Day 2, people will start to turn. They'll see how much usage it burns and how much less it got done compared to Opus. They'll start merging bad code because they couldn't actually understand what 6.2 Astra said about it, but trusted it was fine. They'll see loops they forgot about running since launch that have made zero progress.

Day 5, people will mostly be over it. There will be viral posts saying "Is SOL secretly BETTER than Astra??", "Did ASTRA get DUMBER ALREADY??" etc.

Day 14, this model will be mostly forgotten.

I really, really, REALLY hope I'm wrong about all this. I am likely overreacting (I am admittedly known for that). I could have chosen to calm down, wait a bit, and give you a more measured take, but instead I am dumping this here because I wanted you guys to know how I actually feel about this model.

Sorry about the rant 🙃

6293919.2K3K