"Plausible-looking, locally coherent, globally wrong." That's how developer Daniel Doubrovkine described the AI-generated code that knocked out his Slack application, slack-sup2, after it blew through API rate limits and took down every API call in the system.
The task seemed simple enough: close old group DM conversations in Slack. The AI wrote code that looped through hundreds of conversations, calling conversations_close() on each one sequentially. Clean-looking code. One problem: Slack enforces a global rate limit of 1 request per second on that endpoint. When the job ran against workspaces with hundreds of stale DMs, it didn't just fail on the close calls - it triggered a global rate limit that broke every other API call the app was making.
The "Fix" That Made It Worse
When Doubrovkine asked the AI to handle the rate limiting, it generated retry logic with sleep() calls. In the async framework the app uses, sleep() blocks entire fibers (lightweight threads that handle concurrent operations). So the "fix" didn't just fail to solve the rate limit problem - it froze the app's ability to do anything else while waiting.
This is a pattern worth recognizing. The AI produced code that works perfectly in isolation. Close a conversation? Sure, here's the API call. Handle a rate limit? Sure, just wait and retry. Each piece makes sense on its own. But in a distributed system where one endpoint's rate limit affects all endpoints, and where blocking calls freeze concurrent operations, the combined result is a cascading failure.
What Actually Fixed It
Doubrovkine's real solution required understanding the system as a whole:
- Moved the cleanup into a 30-minute cron job instead of running it all at once
- Put the feature behind a disabled-by-default setting so it wouldn't surprise existing users
- Built a separate script to slowly drain thousands of unclosed DMs for current customers
- Refactored to auto-close only a limited number of DMs at a time, staying well under rate limits
None of these decisions are things a code generator would suggest unprompted. They require knowing that Slack's rate limits are global (not per-endpoint), that the framework's async model doesn't tolerate blocking calls, and that existing customers had thousands of accumulated DMs that needed a migration path.
The Real Lesson for AI-Assisted Development
This isn't an argument against using AI for code. It's a precise illustration of where AI code generation fails: at system boundaries. The AI can write correct Slack API calls. It can write correct retry logic. What it can't do is understand how those pieces interact with rate limits, concurrency models, and deployment realities it has never been told about.
For anyone using tools like Cursor, Claude Code, or GitHub Copilot for production work, the takeaway is specific: AI-generated code needs the most scrutiny exactly where it looks the most confident - at the boundaries between your code and external systems with their own constraints.