If you use OpenAI’s dot, don’t be surprised if it consumes a lot of tokens on its own. According to AI agent security and governance outfit Geordie, one dot agent produced 155 separate activity sequences and consumed more than 300,000 tokens – all without further human interaction, and all in the course of a day.<\/p>\n
Geordie has been testing the \u201calways on\u201d agentic assistant<\/a>, dot, since OpenAI announced<\/a> it in September, following in the footsteps of\u00a0Meta Muse<\/a> and\u00a0Grok Bot<\/a>.\u00a0<\/p>\n Compared to other personal agents, however, OpenAI\u2019s dot \u201clooked a little bit different,\u201d Hanah-Marie Darley, Geordie chief AI officer, told The Register. \u201cOne of the things that we discovered really quickly was that they were really, really active without much user interaction.\u201d<\/p>\n For its testing, Darley gave dot read-only permissions, and she didn\u2019t connect it to her personal apps, email, or other communications apps like Slack. The bot quickly went into research mode, collecting as much information as possible and surveying systems.\u00a0<\/p>\n \u201cYou can think about it very much like a new project manager coming in and taking a look at what’s there, looking around to see what accesses you’ve given it, what accesses you have, and also what kind of systems you’re running,\u201d Darley said.\u00a0<\/p>\n The agent collects all of these data points into something akin to a daily journal entry, and continually updates those throughout the day.\u00a0<\/p>\n \u201cIf you’ve given it any tasks, it will effectively check on its own tasks, too,\u201d Darley said. \u201cBut what was really unique was that the dot was spinning up agents without interacting with the user at all. It was all driven by the dot rather than a persistent or scheduled task from the user, or any user prompting.\u201d\u00a0<\/p>\n ‘Hyper-personalized,’ and ‘always-on’ \u2026 what could possibly go wrong?<\/p>\n Most enterprise AI agent usage tends to be custom-built bots, tasked with specific business projects, or agent harnesses like Claude, Codex, or Hero for AI-assisted engineering and coding, Darley said.\u00a0<\/p>\n \u201cThose tend to be really driven by employees who are setting goals,\u201d she told us during an interview. \u201cBut interestingly, personal agents set a lot more of their own tasks – and I don’t think many people realize that as they start to play with them.\u201d<\/p>\n This is the whole value proposition of personal agents. Rather than passively waiting for user prompts, bots like Muse and dot aim to be proactive digital workers that perform multi-step tasks and connect to multiple apps, services, and accounts on the users\u2019 behalf.\u00a0<\/p>\n \u201cThey\u2019re trying to go hyper-personalized and collect as much context and information about you as possible to recommend things to you, and to be proactive,\u201d Darley said. \u201cBut what that means is that they’re effectively always on, operating in the background, and they have these active periods,\u201d she added.<\/p>\n Dot, in particular, is more proactive than other personal agents like Muse or Grok Bot, according to Darley. This doesn\u2019t necessarily translate into higher token usage, she added. \u201cThey just operate differently.\u201d<\/p>\n For example, dots always operate with at least two agents running: The primary agent interacts with the user, and the secondary agent continuously runs in the background, collecting data. Dots and other agents can also spin up subagents for specific tasks.<\/p>\n 50 traces in 2 minutes<\/p>\n Of the dot-in-testing\u2019s 155 agent traces, at least 50 of those happened within a two-minute period, and the majority occurred within a couple of hours, according to Darley.<\/p>\n \u201cWithin one of the traces that we analyzed, the agent spun up multiple sub-agents that were short-lived, they would just do specific tasks, and fed the information from the existing accesses that had already been granted to the dot – and none of this was interacting with the user,\u201d Darley said.\u00a0<\/p>\n Part of the evaluation involved testing a development system, and one of the dots spun up a sub-agent that checked all of the code repositories in the system. Meanwhile, another sub-agent continuously inventoried and tallied up all the data that it found while operating.<\/p>\n \u201cThe user would have no idea that there were sub-agents working on their behalf,\u201d she added. \u201cThey would have no idea how those agents were given access, or what their tasks were, which is really, really different from how most agents have been operating in business contexts.\u201d<\/p>\n The 155 traces cost Geordie about 307,000 tokens. Other requests cost anywhere from a couple hundred tokens to several thousand, depending on how much research the dot performs. \u201cThat\u2019s really interesting because it’s self-directed research,\u201d Darley said.\u00a0<\/p>\n The user isn\u2019t asking it to perform these tasks – or use a set number of tokens – but rather the dot does it on its own.<\/p>\n Tokenomics<\/p>\n Token costs vary widely depending on the user’s location, subscription tier, discounted pricing arrangements, computing costs, and other terms and conditions.<\/p>\n But for the purposes of our discussion, roughly 200,000 tokens consumed by a dot based on OpenAI\u2019s Astra model cost between $5 and $10. \u201cSo if you think about what that\u2019s going to look like every day, that could mean spending what you pay for a latte, depending on the city – but without any interaction with the user,\u201d Darley said.<\/p>\n \u201cOne of the things that a lot of people struggle with, especially when it comes to understanding their AI spend, is: How does token usage connect out to work product, or how does it connect out to results? And it’s a hard one because it’s not necessarily one to one,\u201d she noted.\u00a0<\/p>\n One request to an LLM could cost 500 tokens and still be more valuable, from a business perspective, than another single request that cost 800,000 tokens. \u201cIt’s really about how much compute does it cost you to get to a good answer, and that becomes a really subjective judgment,\u201d Darley said.\u00a0<\/p>\n When it comes to dot and other always-on agents, however, she suspects \u201ca lot of people aren\u2019t going to be prepared for the background spend.\u201d<\/p>\n Do you know what your agents are doing?<\/p>\n Just like employees did with chatbots, and then with coding agents, and now with personal agents: assume that people will bring dot and friends from home into their work life, Darley said.\u00a0<\/p>\n \u201cBusinesses are surprised often by the number of agents that are actively in their systems, even if they have a smaller rollout or a more controlled rollout in mind,\u201d she said.\u00a0<\/p>\n \u201cThe second thing is not all agents act the same,\u201d she added. \u201cThey operate with varying degrees of autonomy, with varying degrees of user input, and they’re also really context driven. The best agents are the ones with the narrowest context because that means that they can pursue their goal with the right information at the right time.\u201d<\/p>\n Darley suggests organizations try the technology. She\u2019s a self-described \u201cmassive AI optimist,\u201d and says she believes agents can improve productivity and transform the way people work. AI isn\u2019t going away, and, as such, businesses should familiarize themselves with agents. But she has one final warning before letting agents loose.\u00a0<\/p>\n \u201cDon\u2019t give them any personal information that you wouldn’t be comfortable sending in an email,\u201d Darley said. \u201cAnd don\u2019t put real money on it – like your personal cards. Buy it a Visa gift card, if you want to give it access to make purchases for you. Oftentimes, people don\u2019t think about that until something goes wrong.\u201d \u00ae<\/p>\n","protected":false},"excerpt":{"rendered":"If you use OpenAI’s dot, don’t be surprised if it consumes a lot of tokens on its own.…\n","protected":false},"author":2,"featured_media":932299,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[62,276,277,49,48,61],"class_list":["post-932298","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-artificial-intelligence","tag-artificialintelligence","tag-ca","tag-canada","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/posts\/932298","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/comments?post=932298"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/posts\/932298\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/media\/932299"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/media?parent=932298"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/categories?post=932298"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/tags?post=932298"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}