How fast is GPT-4o compared to Claude Sonnet right now, not in a press release, but at this moment?
AI Speedometer is a free web tool that measures real-time token generation speeds across major AI models. Token generation speed - how many words or word-fragments a model can output per second - is one of the most practical performance metrics for everyday AI use, and one that rarely appears in official model announcements. The tool runs live tests rather than displaying static benchmark reports, which go stale fast in an industry where model infrastructure changes weekly.
For daily AI users, speed has a real impact that capability scores don't capture. A model that scores well on reasoning tests but takes three seconds per sentence will feel frustrating in any workflow where you're waiting on output repeatedly - coding assistants, customer-facing chatbots, live document drafting. The difference between 40 tokens per second and 80 tokens per second is the difference between a tool that feels responsive and one that feels like it's thinking too hard.
This appears to be a solo developer project rather than a commercial product, which removes the vendor incentive to favor any particular provider's numbers. The tradeoff is that real-time benchmarks are inherently noisy - a model can be fast at 9am and throttled at peak hours. A single data point tells you less than a pattern over time, and the tool's staying power depends entirely on whether it gets maintained as new models ship.
For now, it fills a gap. If you're choosing between models for a latency-sensitive use case and want a live read rather than a marketing claim, it's a reasonable place to start. Available at ai-speedometer.oliveowl.xyz.