Pricing: $150 / 1M input tokens, $600 / 1M output tokens. (Not a typo.)

Very expensive, but I've been using it with my ChatGPT Pro subscription and it's remarkably capable. I'll give it 100,000 token codebases and it'll find nuanced bugs I completely overlooked.

(Now I almost feel bad considering the API price vs. the price I pay for the subscription.)

Is your prompt {$codebase} find bugs?

Typically something like:

  Look carefully through my codebase and identify any bugs/issues, or refactors that could improve it.

  
  …
  

Doesn't have to be anything overly complicated to get good results. It also does well if you give it a git diff.

It has a 2023 knowledge cut-off, and 200k context window... ? That's pretty underwhelming.

On the positive side, the cutoff date probably makes it a lot more upbeat.

Seems underwhelming when openai's best model, o3, was demoed almost 4 months ago.

Assuming a highly motivated office worker spends 6 hours per day listening or speaking, at a salary of $160k per year, that works out to a cost of ≈$10k per 1M tokens.

OpenAI is now within an order of magnitude of a highly skilled humans with their frontier model pricing. o3 pro may change this but at the same time I don’t think they would have shipped this if o3 was right around the corner.

If you start paying someone and give them some onboarding docs, to a first approximation they'll start doing the job and you'll get value.

If you attach a credit card to o3 and give it some onboarding docs, it'll give you a nice summary of your onboarding docs that you didn't need.

We're a long way from a model doing arbitrary roles. Currently at the very minimum, you need a competent office worker to run the model, filter its output through their judgement, and act on it.

At first I thought, great, we can add it now to our platform. Now that I have seen the price, I am hesitant enabling the model for the majority of users (except rich enterprises) as they will most certainly shoot themselves in the foot.

> they will most certainly shoot themselves in the foot

...and then ask you for a refund or service credit.

> $150/Mtok input, $600/Mtok output

What use case could possibly justify this price?

Synthetic data generation. You can have a really powerful, expensive model create evals so you can tune a faster, cheaper system with similar performance.

You could do that, but OpenAI specifically doesn't want you to: https://openai.com/policies/row-terms-of-use/

What you cannot do. You may not use our Services for any illegal, harmful, or abusive activity. For example, you may not: Use Output to develop models that compete with OpenAI.

Presumably you run the risk of getting banned if they realize what you're doing.

Screw their TOS.

OpenAI trained on the world's data. Data they didn't license.

Anyone should be able to "rip them off" and copy their capabilities on the cheap.

I suspect they have no way to enforce that without risking false positive hurting their rich customers (and their business).

It enables obscene unnatural things at a fraction of most SWE hourly rates. One win that jumps to mind was writing a complete implementation of a Windows PCM player, as a flutter plugin, with some unique design properties and emergent API behavior that it needed to replicate from existing iOS/Android code

Does it really? Your average software engineer is like £20-30 an hour, for the cost of 1m output tokens you can get a Dev for a full week.