Agentic AI Token Growth Raises Memory Pressure in AI Data Centers
Редакция ЦОД:Главное
AI agents were consuming roughly five times as many tokens as human users by August 2026, according to OpenRouter data cited in a secondary report. The reported pattern points to context reuse and KV-cache activity—not only accelerator capacity—as a growing infrastructure constraint.
- Agentic token consumption reportedly rose from about 0.51 trillion to 7.3 trillion tokens after February.
- AI-related token use was reported at roughly five times human usage by August 2026.
- Cached prompts accounted for more than 85% of agent tokens in the cited account, highlighting KV-cache and context-memory demand.
- Memory capacity, bandwidth and component sourcing could become constraints alongside GPU availability if the trend continues.
- The cited outlook for tighter RAM and storage supply in 2027–2028 is a forecast, not confirmation of an existing shortage.
Reported OpenRouter data indicate that AI-related token use had reached roughly five times human usage by August 2026. Following February, agentic token consumption reportedly rose from about 0.51 trillion to 7.3 trillion tokens—approximately a 14-fold increase—while human token consumption increased about 2.8 times to 1.4 trillion tokens.
The infrastructure significance is in the composition of that traffic. More than 85% of agent tokens were described as cached prompts, according to a16z as cited in the report. In one cited Claude Code usage example from September, 96% of input data was attributed to rereading earlier conversations. Such workloads can place increasing emphasis on the capacity and bandwidth required to retain and retrieve context, including KV-cache memory, rather than on new compute alone.
For AI data-center operators, this could shift server and cluster design priorities toward HBM, system RAM, context storage and the associated memory fabric. A continued rise in agentic workloads may increase the value of memory-oriented configurations and make component procurement a more material planning issue alongside GPU sourcing.
The report also said that Micron expects RAM and storage-device shortages to worsen in 2027–2028, with HBM allocated preferentially to AI data centers. That is a forward-looking assessment, rather than evidence of confirmed shortages during those years. The source provides no absolute HBM or RAM shipment volumes, memory prices, installed memory capacity, or data-center power figures, so the scale of any resulting effect on AI data-center CAPEX cannot be quantified from the available information.
If the reported trend persists, AI data-center operators and server designers may need to place greater weight on HBM, system RAM, KV-cache capacity, context storage and memory bandwidth. Memory supply and pricing could become constraints alongside GPU availability, although the available material does not quantify the resulting impact on infrastructure investment.
- TechNews 科技新報 / КитайскийПервоисточник →
Материал подготовлен редакцией ЦОД: Главное на основе указанных первоисточников.