At ultra-fast token rates (~750 tok/s), the binding constraint in an agent loop migrates from model inference to CPU-bound tool execution, so the harness/tool-runner becomes the critical path that must be scheduled and parallelized rather than the model call itself.
It relocates optimization effort: teams that keep tuning model latency/throughput will plateau, while tool fan-out, sandbox provisioning, and host CPU contention become the dominant latency contributors in end-to-end agent runs.
Whenever one stage of a pipeline is accelerated by orders of magnitude, the adjacent stage inherits the bottleneck; architecture should be re-profiled, not just re-tuned, after a step-change in component speed.
Ultra-fast speeds (750+ tokens per second) will become the default within a year or two.
High token speeds change the bottleneck from bandwidth to CPU tool calls.