\t\t\t\t\tSign up to see the future, today<\/p>\n
Can\u2019t-miss innovations from the bleeding edge of science and tech<\/p>\n
As programmers make more and more use of generative AI in the workplace, their responsibilities are quickly transitioning from creating new products from scratch to babysitting a large language model that cobbles together code.<\/p>\n
It\u2019s gotten to the point where some engineers are no longer able to make sense of the code being spat out by an AI, as chip optimization software company Infinity founder Jeremy Nixon told Business Insider<\/a>.<\/p>\n That doesn\u2019t bode well, considering researchers are finding that the widespread use of AI can lead to cognitive decline and skill atrophy, including among computer scientists<\/a>. Developers are afraid that software engineers are drifting further and further away from the act of actually writing code and learning new skills required to architect larger projects as their employers continue to push for more AI automation.<\/p>\n Even employees at frontier AI labs are aghast at peers who are struggling to make sense of the outputs of AI models. Earlier this year, independent research firm SemiAnalysis technical staff member Jordan Nanos revealed in an interview that OpenAI engineers had no idea what they were looking at while scrolling through the code of their own graphics processing unit kernel, the program at the core of the company\u2019s efforts to better make use of available hardware to train and run AI models.<\/p>\n \u201cThe point is, the guys who were scrolling through this code with us, it was kind of clear that they know a whole bunch about hardware, they know a whole bunch about the concepts in the system, and they just have no idea what this MLA kernel that they\u2019re showing us for DeepSeek actually does,\u201d Nanos told Fireside Alpha\u2018s Mansour Karam<\/a>.<\/p>\n \u201cYou can go line by line and it\u2019s like nope, nope, nope,\u201d he added. \u201cThe AI understands it, the AI tests it, and you see the results. It produces correct kernels that perform really well.\u201d<\/p>\n At the time, Nanos argued that the \u201cactual code is not necessarily something that a human has to reason about deeply,\u201d because the \u201cAI knows how to manipulate the data movement and the processing elements on the hardware that you\u2019ve given it.\u201d<\/p>\n One danger is that errors or hallucinations will fall through the cracks as human oversight fades. Case in point, Amazon was forced to implement a 90-day \u201ccode safety reset\u201d earlier this year following a series of outages that disrupted customer orders. While the company pushed back on claims that AI was to blame, internal meeting notes revealed<\/a> that managers were concerned about the \u201chigh blast radius\u201d of \u201cgen-AI assisted changes.\u201d<\/p>\n A recent analysis by software company Undo<\/a> found that the transition to AI agents is opening the floodgates. Some 35 percent of AI-generated code \u201chas not been fully comprehended before engineering teams push it to production,\u201d the firm said.<\/p>\n Meanwhile, 29 percent of software engineer respondents said that they\u2019d witnessed a loss of productivity due to \u201cengineers needing to \u2018unpick\u2019 AI-generated code\u201d multiple times per month, while a whopping 94 percent said they saw such an incident at least once in the past six months.<\/p>\n The sheer amount of time engineers are now spending each day going through AI-generated code raises a nagging question: is the use of the tech more trouble than it\u2019s worth<\/a>?<\/p>\n To people like Nanos, it\u2019s not necessarily a matter of incompetence. It\u2019s \u201csimply an illustration of a paradigm shift in software development,\u201d as AI researcher Sergey Cleftsow argued in a recent Medium post<\/a>.<\/p>\n \u201cIn other words, AI has advanced to the point where there\u2019s no need to worry about many things anymore,\u201d he wrote. \u201cDevelopers simply need to define the architecture and correctness criteria; the neural network will handle the rest.\u201d<\/p>\n But whether such optimism will hold out as software developing becomes ever more unrecognizable remains an open question.<\/p>\n