Astra ships with a 1,050,000 token context window. Two separate limits stop you from using it.
Inside: the 272K price cliff, the rate limit that blocks a full-window call on most accounts, and the five effort levels OpenAI ran its benchmarks at the top of.
Get both ceilings right and the same work costs a third as much.
Here's the full setup ��
Before we dive in, I share daily notes on AI & vibe coding in my Telegram channel: https://t.me/zodchixquant 🧠
Ceiling one: the price cliff at 272K
From the model page: prompts above 272,000 input tokens are priced at 2x input and cache rates and 1.5x output, for the full request.
Not for the overage. For everything.
Output gets repriced on that same request too.
A 30k answer goes from $1.50 to $2.25 because your input crossed a line you never saw.
Ceiling two: your tier can't send the window
This one is in the rate limit table. Tokens per minute by usage tier:
- Tier 1: 500,000
- Tier 2: 1,000,000
- Tier 3: 2,000,000
- Tier 4: 4,000,000
- Tier 5: 40,000,000
The window is 1,050,000 tokens. On Tier 1 and Tier 2 a single full-window request exceeds your entire minute. Tier 3 is the first tier where the advertised window physically fits in one call.
Three numbers, and the smallest one is your actual ceiling.
The preflight check
The character heuristic is deliberately rough. It exists to catch the request that would have cost you double, not to be exact. Exactness comes from usage after the call, below.
The cost function
This is the piece worth keeping. It turns any response into dollars, with every multiplier the price sheet applies.
Log that dict on every call for one day. Two fields tell you everything:
- over_cliff true on any request means you paid double for that call, and penalty says how much
- cache_hit_rate near zero across a session means your prefix is changing every request, usually a timestamp, a session id, or a system prompt you rebuild from a template
Caching here is automatic and prefix-based. There are no breakpoints to set, which also means it fails silently. This function is how you find out.
Effort has five levels, not three
reasoning.effort takes low, medium, high, xhigh, max.
The launch evals carry a footnote: unless stated otherwise, models ran at maximum effort.
FrontierMath Tier 4 at 98%, ARC-AGI-3 at 99.9%, ExploitBench at 100%, all at the slowest and most expensive setting.
Baseline at low. Raise to high for the step that needs it. Reserve xhigh and max for work you would otherwise do yourself.
Two tools that are cost levers
In the Responses API, Astra supports hosted shell, apply patch, computer use, skills, tool search, MCP, web search, file search and code interpreter. Two of those change your bill directly.
Apply patch. Whole-file rewrites bill as output at $50 per million. A structured diff on the same edit is a fraction of it, every time the agent touches a file. This is the single largest output-side saving available.
Tool search. Tool definitions live in your prefix and get sent on every call. Past roughly a dozen tools, loading them on demand takes them off the per-request bill entirely.
Batch and Flex at half, Fast at double
- Batch and Flex are 50% of standard. Anything with no human waiting belongs here
- Fast mode is 2x. You're buying latency, not quality
- Batch queue limits are separate from TPM and much larger. Tier 3 gets 2M TPM but a 100M batch queue, so deferred work never competes with live traffic
Pass mode="batch" to price() and it applies the discount, so your logs show what the same work would have cost either way.
Common mistakes
- Planning around the full context window. 1.05M in the docs, repriced at 272K, unreachable in one call below Tier 3.
- Copying the benchmark effort into production. The evals ran at max. That's a purchase decision, not a default.
- Letting the agent rewrite whole files. Output is $50 per million and apply patch exists for exactly this.
- Assuming the cache works. It's automatic, so it fails without an error. cache_hit_rate is the only proof.
The 10-minute setup
- Paste price() into your logging layer (3 min)
- Add preflight() before every call with your tier's TPM (2 min)
- Set reasoning.effort to low as baseline (1 min)
- Turn on apply patch for anything that edits files (2 min)
- Run one real session and read the dict (2 min)
If over_cliff is false and cache_hit_rate climbs turn over turn, you're done. If either fails, you now know exactly which one and what it cost you.
A million token window reads like permission to stop thinking about retrieval. The price doubles at 272K and the rate limit stops you earlier than that on most accounts, which means the architecture question was never optional.
Thanks for reading!
I share daily notes on AI & vibe coding in my Telegram channel: https://t.me/zodchixquant ��
