This picture summarizes it well:
Here on my blog, for this popular blog post I get a lot of comments. 28k blog comments over the years. Some of them are terribly written and hard to understand, so I let AI suggest a rewrite. That code that sends the blog post comment to AI, I actually fire off three times: once with OpenAI gpt-5, once with OpenAI gpt-5-mini, and once with Claude claude-opus-4.8. I use my human eyes and judgement to evaluate the results, and I can tell you they do equally well. Only the slightest differences.
The surprising thing is how amazingly slow OpenAI's gpt-5 is! It's nearly 10x slower than claude-opus-4.8. What's up with that!?
It's also clear that the latency difference between gpt-5-mini and gpt-5 is significant. At the time of writing, the input token price difference between gpt-5.4 and gpt-5.4-mini is $2.50 compared to $0.75! That's a 3x difference.
Conclusion
-
If you're constructing a prompt the API, use Claude.
-
If you have to use OpenAI, consider the
minimodel because it's both cheaper and faster.
Bonus
Before I added Claude, I used to use litellm to wrap OpenAI's models. The code looks like this:
response = litellm.completion(
model="openai-gpt-5",
api_key=settings.OPENAI_API_KEY,
messages=my_prompt_messages,
)
Unlike, if you use the native OpenAI Python SDK the invocation looks like this:
client = openai.OpenAI(api_key=settings.OPENAI_API_KEY)
response = client.responses.create(
model="gpt-5,
input=my_prompt_messages,
)
I measured the difference, in speed, where I compare using the OpenAI SDK versus the litellm wrapper and the difference looks like this:
Granted, in June I "only" did a bit over 30 of these calls, but strangely there's a difference!
I don't have the intricate knowledge to understand why the litellm makes the total time different from using the SDK. (not sure I care either!)
Either way, I'm moving away from litellm and only use the SDKs provided by Claude and OpenAI. Feels safer given the CVEs we've seen this year on litellm.