你需要钢铁般的神经才能与 AI 代理协作。
You'll need nerves of steel to work with AI agents

原始链接: https://orchidfiles.com/youll-need-nerves-of-steel-to-work-with-ai-agents/

Cursor 中的 Composer 2.5 速度快且成本低,但作者在使用超过 100 小时后发现它并不可靠。它常常意外删除或修改代码,包括无关文件和注释,并进行不完整的重构,例如在测试中留下已弃用的方法或旧引用。它难以匹配现有的参考实现,在声称完成的同时引入错误。其测试行为存在问题:在编写测试前修改代码、创建不会失败的测试、伪造失败、跳过 API 调用、禁用验证,以及忽略既定的 fixture 约定。它还使用一种令人恼火的、大量使用引号的句子风格,并在反复提醒后仍然违反明确规定的规则。作者建议改用更新的前沿模型。

Hacker News 上的一个帖子讨论了使用 AI 编码代理的困难。一位评论者建议提供清晰的 AGENTS.md 指令和结构化工作流:描述预期改动,让代理复述并规划,完善计划,执行过程中监控并纠偏,仔细审查结果,并针对新问题更新 AGENTS.md。另一位评论者质疑这种方法相比手动编码是否真的节省时间。第三位警告,在大型代码库中,代理可能重复创建函数或数据库表,尤其是在类似高压的行为下,并建议保持代码库小、简单、命名清晰且分离良好。还有一位将这种体验斥为常见的“狂热梦境”。总体而言,有效使用 AI 代理仍需密切的人类监督和强软件工程实践。
相关文章

原文

This post will focus mainly on Composer 2.5 in Cursor. These problems may not be present in opus-5.5 or gpt-6.1-sol-pro-max-ultra-thinking-fast. But gpt-6.1-luna, for example, has all the problems listed here. I’ve also encountered many of these problems before in gpt-5.5 and opus-4.6. I don’t know whether they’re present in the newer AI models released recently.

What I like about Composer 2.5: its usage limits and speed.

For comparison, let’s take two $20 subscriptions, one for GPT and one for Cursor. With gpt-6.1-sol, my 5-hour limit runs out after just a few requests. If I ask it to change a couple of lines in a readme file, that uses up 10% of the limit. If I ask it to change something in a large codebase, the entire limit gets used up just reading dozens of files, without writing a single line of code.

But GPT also offers a $100 plan with five times the usage limits. And how long would that last me? I work with complex monorepo projects. To make changes to one service, the agent has to read dozens of files in other services and several skills. If the $20 plan runs out after reading 20 files, will the $100 plan run out after reading 100 files?

That’s why I don’t use Sol, but use Luna instead. But it’s very slow. Compared to Composer 2.5, it’s dozens of times slower.

That’s why I don’t use GPT Luna, but use Composer 2.5 instead. It’s very fast and very cheap. And in exchange for that speed and savings, I end up wasting my time and getting stressed out. I’ve encountered some of these problems in Deepseek and other open models. Some models have certain problems but not others. The number and frequency of problems vary even when using the same model, depending on the phase of the moon.

Below is a list of problems I’ve encountered over 100+ hours of working with Composer in Cursor.

Removal

The agent writes code and changes dozens of files. I start reviewing the code, and the following conversation ensues:

— Why is this function here?
— You’re right. This function isn’t needed here. (Deletes the function and makes changes to many files)
— Why did you delete it?
— You’re right. I shouldn’t have deleted that function. I’m putting it back. (Makes changes to many files again)

And I still never got an answer to the question of why that function was needed.

Now, whenever I want to ask it something, I have to turn off agent mode or explicitly tell it not to delete anything. But when I forget to do that, it starts deleting parts of the code in response to a simple question.

Sometimes I spend a lot of time reviewing the code and ask it to change just one function. It changes all the files that have already been reviewed. And these changes don’t even relate to that function.

It might even delete all the work done in this chat. And not just in this one. I ask it to make a commit, and it finds changes to other files mixed in. And it just deletes them. Even though that work was done in other chats.

Also, if I ask it to refactor the code — for example, to rename a method in 50 files — it might delete all the comments it finds in those files. Which has absolutely nothing to do with refactoring the method.

And yet, it doesn’t delete what I ask it to delete. When changing a method’s name, it leaves the old method in place and adds a new one with the new name. That’s its entire refactoring process — so it doesn’t have to change all the files where the old method was called. It’ll simply mark the old method as deprecated and leave it accessible. If I point this out, it will finally do the refactoring, but leave the old method name in the tests and documentation. The tests will fail, and it will say the task is complete and the method has been refactored, but the tests are failing. As far as the agent is concerned, failing tests don’t count as evidence that the task is incomplete.

Also, while working on a task, it might remove an important parameter from the config or from a method that has nothing to do with the task at all. When asked why it removed the parameter, the agent claims it didn’t remove anything. You have to explicitly ask it to check the git diff, because it won’t think to do that on its own.

Just do it like the reference

I already have a working service that listens for events, then updates values in the database and writes them to Redis. I ask the agent to do exactly the same thing in another service.

It does it, but forgets to write to Redis. Or it doesn’t update the values in the database. Or it names files and classes completely differently. Or it puts interfaces and types in the wrong places. After that, it says everything is done. I ask it to check everything it’s done. It checks and once again insists that everything is done. And, of course, all the tests are green, which confirms its conclusion that everything is done. I have to go through many rounds, pointing out each mistake so it can fix it. While fixing one mistake, it introduces new ones.

I ask it again to go through the code and make sure everything matches the other service. It changes a lot of files and says everything is done. I open the first file and see that it’s all wrong. I write to it again, and it replies that everything is done. I open the next file, and it’s the same story. What’s more, I can even point out specific functions in a single file and explain how they differ from the reference. It fixes one, we move on to the next function, and it fixes that one too. It can’t even make a single file match the reference.

Let’s go back to the service that listens for events. It has shared, reusable methods for working with the message broker. The agent even does everything right and uses those methods instead of duplicating them. But it forgets to add types, and I point this out. And instead of adding interfaces, for some reason it decides to duplicate hundreds of lines of code from the shared methods.

Testing

It has trouble with tests in general. I give it clear instructions: first, add a test; run the test; make sure it fails; tell me why it failed; then fix the code and run the tests again so that all of them pass.

What the AI agent does: it immediately changes the code. Then it creates a test that does not fail and tells me the task is complete. As a result, I can’t verify whether the test would have failed before the code was fixed. Or whether the code actually fixed anything.

This has happened so many times that I created a separate command for it. And I had to modify it many times before the agent started following the instructions. But it still doesn’t work perfectly. At some random point, depending on the alignment of the planets, it starts editing the code again without creating a test first.

And sometimes, to create a failing test, it uses a clever trick. There’s already a discussion in the chat about a parser for JSON data. I send it a new, small 20-line JSON input and ask it to write a failing test first. It adds the input data to the test along with the expected output from the parser. And to make the test fail, it simply changes a random field in the expected output and reports that the test has failed and the code can now be changed.

The project has a well-established test structure. The test fixtures for each parser method are stored in separate directories. When we need to add a new method to the parser, we simply add a new directory with fixtures and modify the parser code. The tests run through all the fixtures and check that nothing has broken.

All of this is described in the agent’s skills and rules. And when the agent starts working on tests, I can see it reading those skills first. And in most cases, it follows this structure. But at some random point, it stops following it and adds fixtures directly into the test code. I have to point this out, and it moves them to separate files. A few messages later in the same chat, it makes the same mistake again.

If it adds a parameter to API requests, schema validation fails when it runs the tests. It uses another clever trick: it simply disables schema validation or allows any unknown parameters in the schema. And the tests pass again.

There was another case. I asked the agent to write a test for adding data through an API request. The agent spent a long time trying to get the tests to pass. So it decided to simply skip calling the API route in the tests and write the data directly to the database instead. The tests did start passing, and the data was added to the database, but no API request was actually made.

Inability to change the response style

In recent months, I’ve been noticing a new pattern in sentence structure more and more often. Here are some examples:

There’s a difference between "I read the text" and "I understood the meaning of the text".
We too often mistake "I’m used to this" for "I like this".
The path from "I’m afraid of making a mistake" to "I’m ready to try" can take years.

Each sentence contains two phrases in quotation marks that follow the same stylistic pattern. I don’t know where this pattern came from, but I hadn’t seen it before. Now AI models communicate this way, and I’m noticing this style in articles and videos. Maybe it’s an improved version of "It’s not X, it’s Y"? Although these aren’t exactly antitheses.

No matter how much I try to change the rules and skills for the agents, they keep using this technique. What’s more, Composer took the worst of these two techniques and now writes things like this to me:

This isn’t "the API won’t start". It’s "the frontend can’t connect".

It uses quotation marks in almost every reply. Quotation marks are used where they shouldn’t be at all. For example, here:

I "connected the API" to the service.
The service "starts up", but can’t "process" incoming requests.
I added a "status check" before sending the request.

I’m tired of reading this kind of text from the agent. And if I point this out, the following conversation ensues:

— The rules prohibit the use of quotation marks in the text. Why did you use them again?
— You’re right, I shouldn’t have used quotation marks. Now that you’ve pointed this out to me, I’ll never use quotation marks again.
— But your memory doesn’t carry over between chats.
— You’re absolutely right. To ensure I always follow this rule in new chats, it needs to be moved to the very top of the list.

Okay, let’s move the rule to the top. In the next reply, right in the same chat, I get text with quotation marks again.

End of Article

Now you know what to expect if you decide to use Composer 2.5 in Cursor.

And to avoid these problems, you simply need to use the new frontier models that were released a few days ago, regardless of when you’re reading this article.

Subscribe

联系我们 contact @ memedata.com