At the frontier these models are close enough that the harness around them matters more than the choice between them. How much of the codebase the tool can see, whether it can run the tests, and whether it can read a failure and try again changes output quality far more than swapping one model for the other. Both share the same dangerous failure: confident code that is subtly wrong.
What I actually notice
Being careful here, because this is the section where people overclaim. These are tendencies I observe in daily use, not benchmark results, and they shift with each release.
| Task shape | Tendency |
|---|---|
| Large multi-file refactor | Thoroughness and willingness to read more context wins |
| Well-specified single function | Decisiveness wins; extra caution is just latency |
| Debugging from a stack trace | Whichever can actually run the code, regardless of model |
| Unfamiliar library | Both hallucinate; both improve enormously with docs in context |
| Writing tests | Both good, and both write tests that pass rather than tests that check |
That last row is worth dwelling on. Ask either model for tests and you frequently get tests shaped around the implementation it just wrote, which means they pass and prove nothing. Specify the behaviour you want verified, not “write tests for this.”
Why the harness dominates
The single biggest quality jump I have had did not come from changing models. It came from giving the model the ability to run the test suite and iterate on the failure.
A model that writes code and stops is guessing. A model that writes code, runs it, reads the error, and fixes it is doing something categorically different. That loop is a property of the tool, not the model, and it is worth more than any benchmark difference between frontier models.
The same applies to context. A model that can read the surrounding files writes code that matches your conventions. A model working from a pasted snippet writes generic code that then needs rewriting to fit.
So if you are choosing, the useful question is not “which model is smarter.” It is which tool lets the model see the most and verify the most.
The failure they share
Both produce code that looks right and is wrong, and the specific ways are consistent enough to check for deliberately:
- Packages that do not exist, imported confidently with plausible names.
- Permission rules that expose the wrong data. This is the expensive one. Access-control logic reads as correct far more easily than it is correct.
- Layouts that break on a phone, because nothing in the loop looked at a phone.
- Tests that assert the implementation rather than the requirement.
- Silent fallbacks. Code that catches an error and continues with a default, so the failure never surfaces until the data is wrong.
None of these announce themselves. They pass review by a tired human because they are syntactically clean and read as reasonable. I wrote up six real examples that shipped past every check in why you still need a person when AI does the building.
What I do in practice
- Route by task, not loyalty. Big cautious refactor versus fast well-specified change are different jobs.
- Give it the tests. The loop matters more than the model.
- Review permissions by hand, always. No exceptions. This is the category where a mistake is a breach rather than a bug.
- Ask for the plan before the code on anything non-trivial. It is much cheaper to correct an approach than a diff.
- Never accept a test suite you did not specify the behaviour for.
- Check that a failure can actually fail. Break something deliberately and confirm the test goes red.
The honest summary
If you want a winner declared, I cannot give you one that will still be true next quarter. Both are excellent, both are wrong in the same ways, and the gap between them is smaller than the gap between using either well and using either badly.
What I would actually spend my attention on: the review layer. The first 80% of a build got dramatically cheaper with these tools and the last 20% did not, and no model choice changes that.
Related: six defects AI shipped that passed every check and how I ship a web app in days.
FAQ
Which model is better for coding?
They are close enough that the workflow around them matters more than the model. In daily use against real repositories the differences show up in temperament rather than capability: one tends to be more thorough and cautious on large refactors, the other more decisive on well-specified single tasks. Run both if you can.
Does model choice matter more than the harness?
No. The tooling (how much of the codebase it can see, whether it can run tests, whether it can read the failure and try again) changes output quality far more than swapping one frontier model for another.
What do both models get wrong?
Confident code that is subtly wrong: a package that does not exist, a permission rule that exposes the wrong data, a layout that breaks on a phone. All of it reads as correct. That is why the review layer matters more than the generation layer.
Should I pay for both?
If coding is your job, probably yes, and route by task rather than loyalty. If you are picking one, pick the one whose harness fits how you work. That decision has a bigger effect on your output than the underlying model.