Not only is hardware memory scarce these days, but context memory, the conversational data exchanged with AI models, can be an issue too.<\/p>\n
Cloudflare’s answer to this particular problem is Agent Memory, a managed service for siphoning AI conversations when space is scarce, then injecting the data back on demand.<\/p>\n
“It gives AI agents persistent memory, allowing them to recall what matters, forget what doesn’t, and get smarter over time,” said Tyson Trautmann, senior director of engineering, and Rob Sutter, engineering manager, in a blog post<\/a>.<\/p>\n AI models can accept a limited amount of input, referred to as context. Measured in tokens, the amount varies by model.<\/p>\n Anthropic’s Claude Opus 4.7, for example, has a 1M token context window<\/a>, which can accommodate ~555,000 words or ~2.5 million Unicode characters. Claude Sonnet 4.6 also has a 1M context window, but it holds ~750,000 words or ~3.4 million Unicode characters because it relies on a different tokenizer.\u00a0<\/p>\n Google’s Gemma 4 family of models has context windows of 128,000 for the smaller models and 256,000 for the larger ones.<\/p>\n That may seem like an ample amount of space for model prompts, but there’s a lot of extra text that accompanies every prompt \u2013 the system prompt, system tools, custom agents, memory files, skills, messages, and the auto-compact buffer<\/a>. So the actual context space available might be 10 to 20 percent less.<\/p>\n Storing prompts and responses as “memories” makes the most of available space by providing a place to offload useful chat details that may not be needed for every conversational turn (prompt).<\/p>\n At the same time, more context isn’t always better<\/a> \u2013 there may be times when AI models provide better results when given less context. So memory is potentially useful for pulling data out of a conversation as a quality enhancement as well as a storage management option.<\/p>\n There are already various software<\/a> projects<\/a> and integrated memory tools<\/a> available to help remember AI conversations. Cloudflare is proposing that AI memory should be a managed service.<\/p>\n “Agents running for weeks or months against real codebases and production systems need memory that stays useful as it grows \u2014 not just memory that performs well on a clean benchmark dataset that may fit entirely into a newer model’s context window,” wrote Trautmann and Sutter, arguing that this can be done quickly at a reasonable per-query cost, in a way that doesn’t block the conversation.<\/p>\n Basically, they’re talking about an asynchronous CRUD operation. For example, after storing a memory about the user’s preferred package manager (e.g. pnpm), that memory could be recalled via the following commands:<\/p>\n \n const results = await profile.recall(“What package manager does the user prefer?”);\n<\/p>\n \n console.log(results.result); \/\/ “The user prefers pnpm over npm.”\n<\/p>\n