{"id":235032,"date":"2025-10-31T01:17:12","date_gmt":"2025-10-31T01:17:12","guid":{"rendered":"https:\/\/www.newsbeep.com\/uk\/235032\/"},"modified":"2025-10-31T01:17:12","modified_gmt":"2025-10-31T01:17:12","slug":"are-large-language-models-making-us-less-productive","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/uk\/235032\/","title":{"rendered":"are large language models making us less productive?"},"content":{"rendered":"

This article is an on-site version of The AI Shift newsletter. Premium subscribers can sign up here<\/a> to get the newsletter delivered every Thursday. Standard subscribers can upgrade to Premium here<\/a>, or explore<\/a> all FT newsletters<\/p>\n

Welcome back to The AI Shift, our weekly newsletter about AI and the labour market. <\/p>\n

If the threat of an entry level job-pocalypse (see last week\u2019s edition<\/a>) is many people\u2019s biggest worry about generative AI, the other side of that coin should be huge productivity gains for those of us fortunate enough to already have white-collar jobs. So, are experienced workers in the most AI-exposed industries accomplishing more in less time by using their very own intern<\/a>? Let\u2019s start with a look at the data.<\/p>\n

John writes\u2009.\u2009.\u2009.\u2009<\/p>\n

The first question we need to ask when dealing with AI and productivity is, do the touted gains originate in self-reported guesstimates, or are they directly measured?<\/p>\n

A lot of the splashiest headlines fall into the former category. The latest example came only two days ago when we were sent a press release declaring that new research<\/a> shows \u201cAI boosts productivity by the equivalent of one workday per week<\/a>\u201d.<\/p>\n

A 20 per cent jump in productivity would show up clear as day in the macroeconomic data, but we see nothing of the sort. The devil is in the detail: the figure was arrived at by asking thousands of employees how many hours per week they believe their use of AI saves them in their job. This is certainly not useless information but nor is it especially robust. In fact that 20 per cent figure rang a bell for me\u2009.\u2009.\u2009.\u2009<\/p>\n

A few months ago, AI research organisation METR ran an experiment<\/a> to measure AI\u2019s impact on productivity. It recruited 16 experienced software developers and asked them to provide examples of coding tasks on their to-do lists then randomly assigned them to complete some of these tasks using the latest AI tools and others without. The participating developers were then asked how much time they thought AI had saved them. The answer? That familiar 20 per cent. The catch? As well as self-reported estimates, METR recorded how long the tasks had actually taken. It turned out the developers were 19 per cent slower with AI than without.<\/p>\n

<\/p>\n

Some content could not load. Check your internet connection or browser settings.<\/p>\n

\"\"\/<\/a><\/p>\n

Based on follow-up conversations with the participants, one plausible explanation METR landed on was that it was precisely the developers\u2019 wealth of expertise<\/a> that produced the apparent AI slowdown. They were typically working on very large and long-running projects which reward tacit knowledge over AI\u2019s approach of starting from first principles. Their highly trained eyes made them especially fastidious when checking AI-generated code. This complements other research which finds that AI often does boost productivity<\/a> with more basic code writing and checking<\/a>. Experience, complexity and context matter. <\/p>\n

But METR\u2019s is not the only work casting doubt on some of the most bold claims about AI-enhanced productivity in software. If the modified theory is that we should see more of an impact on lower-complexity code and new projects rather than mature ones, we might expect to see an uptick in the release of new smartphone apps, websites or open-source coding projects. Yet a comprehensive analysis<\/a> by veteran software engineer Mike Judge found no such thing.<\/p>\n

It\u2019s worth re-stating that METR\u2019s was a relatively small and idiosyncratic study which made no claims of being representative of software work as a whole let alone the broader economy, and Judge\u2019s is just another of many ways of looking at this, but two things I think we learn here are 1) people are often not great judges of whether AI is making them more productive, let alone by how much and 2) it\u2019s possible that a tool can feel slick and enjoyable to use<\/a> without actually making us more efficient.<\/p>\n

Sarah writes\u2026<\/p>\n

It\u2019s good to be reminded, John, that when it comes to our own work performance, we aren\u2019t always reliable witnesses! But while you\u2019ve looked into this question at the level of individual tasks, I think the story gets even more interesting when you consider productivity at the organisational level. Because even if large language models do help individuals to race through certain tasks, that doesn\u2019t mean businesses should expect an automatic and commensurate increase in overall productivity. That\u2019s because organisations aren\u2019t just groups of individuals, they\u2019re systems.<\/p>\n

Let\u2019s stick with coding for a moment because it\u2019s a good example. I found the latest annual report<\/a> from Google Cloud\u2019s DORA research programme (which surveys software engineers and tracks performance metrics) interesting on this theme. It argues that AI\u2019s role in software development is proving to be that of an amplifier, which \u201cmagnifies the strengths of high-performing organizations and the dysfunctions of struggling ones.\u201d The research found that the best returns were generated by teams with a good \u201cunderlying organizational system\u201d whereas \u201cwithout this foundation, AI creates localized pockets of productivity that are often lost to downstream chaos.\u201d As W. Edwards Deming, the legendary management expert once said, \u201ca bad system will beat a good person every time.\u201d<\/p>\n

Jason Gorman, who trains and mentors software developers, explained it to me this way: speeding up code generation \u201cin a process with real bottlenecks\u201d can make those bottlenecks worse. \u201cThe high-performing teams have addressed the real bottlenecks \u2014 so they don\u2019t do things in big batches and phases, with handovers, testing, code review, integration and customer feedback happening continuously,\u201d he said. \u201cMost dev teams need to fix those problems before they even think about attaching a code-generating fire hose to their plumbing.\u201d<\/p>\n

I\u2019d be curious to know if this resonates with readers in other professions (do email us<\/a>). You might be able to draft a memo about a new project faster, but do you still have to wait for a committee to read and approve it? And are those decision-makers now drowning in more memos than ever? There\u2019s a link here, too, with the term \u201cworkslop\u201d, coined<\/a> in a Harvard Business Review article to describe \u201clow-effort, passable looking work that ends up creating more work for their coworkers\u201d.<\/p>\n

The example of software development is encouraging overall, though. It suggests that productivity gains should increase over time, as the tools themselves get better, workers learn how to use them more effectively and organisations begin to restructure some processes to make the most of them. The 2025 DORA report suggests that progress has been made since 2024, for example. And if it\u2019s right to say that pre-existing organisational structures can hold things back, then we should start to see some very productive start-ups which can create AI-integrated workflows from a blank slate.<\/p>\n

So what have we learned?John<\/p>\n

Always check the label on claimed productivity boosts. Self-reported estimates of time saved by AI are an unreliable metric that at best tend to produce over-optimistic exaggerations and at worst turn a negative into a positive. Direct measures of changes in output and time spent are the gold standard, though as both the METR example and your reporting make clear, Sarah, even here we should not assume that results for a particular setting or firm will necessarily apply more broadly. Much as with last week<\/a>\u2019s topic of job displacement, there is often an assumption that we will quickly see the impact of AI in the form of a broad wave visible in the macroeconomic data, but even if the technology improves smoothly and steadily, its implementation will not.<\/p>\n

Sarah<\/p>\n

For businesses wondering why they aren\u2019t seeing big productivity gains from AI yet, the best idea is probably to make a careful assessment of systems, workflows and bottlenecks, rather than chivvying individual staff<\/a> to use the new tools more often. That said, not all bottlenecks are easy to dispatch with. In high-stakes professions like accountancy and law, the requirements for quality and accuracy strike me as one example. John, you point out that AI is becoming quite adept at performing \u201ccode review\u201d in the tech sector, but in non-code contexts, human experts are still having to check AI-written documents for hallucinations and errors. We\u2019ve already seen some examples of the reputational and financial consequences<\/a> for companies which failed to invest sufficient time and expertise in verifying AI-generated output.<\/p>\n

There\u2019s still time to save your spot at the FT\u2019s Future of AI Summit on November 5-6, featuring Nvidia\u2019s Jensen Huang and 600 business leaders involved in AI application. As an AI Shift subscriber, you can get 20 per cent off an in-person pass at this year\u2019s summit using the code FT20<\/a>.\u00a0Or register for your complimentary digital pass here<\/a>.<\/p>\n

Recommended reading<\/p>\n

Wharton School professor Ethan Mollick has a thoughtful article<\/a> on improvements in agentic AI and the importance of human agency in directing those agents to do work that provides real social value (John)<\/p>\n

Our colleague Isabel Berwick has an interesting long-form video<\/a> on AI\u2019s \u201cmessy\u201d rollout in the workplace (Sarah)<\/p>\n

The Financial Times and MIT Technology Review are launching a new six-part newsletter series on AI. First up, John Thornhill and Caiwei Chen discuss the US-China battle for supremacy. Sign up here.<\/a><\/p>\n

Recommended newsletters for you<\/p>\n

The Lex Newsletter \u2014 Lex, our investment column, breaks down the week\u2019s key themes, with analysis by award-winning writers. Sign up here<\/a><\/p>\n

Working It \u2014 Everything you need to get ahead at work, in your inbox every Wednesday. Sign up here<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"This article is an on-site version of The AI Shift newsletter. Premium subscribers can sign up here to…\n","protected":false},"author":2,"featured_media":235033,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[554,733,4308,86,56,54,55],"class_list":["post-235032","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-artificial-intelligence","tag-artificialintelligence","tag-technology","tag-uk","tag-united-kingdom","tag-unitedkingdom"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts\/235032","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/comments?post=235032"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts\/235032\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/media\/235033"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/media?parent=235032"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/categories?post=235032"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/tags?post=235032"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}