If you are trying to catch out a chatbot take care, because one cutting-edge tool is showing signs it knows what you are up to.<\/p>\n
Anthropic, a San Francisco-based artificial intelligence company, has released a safety analysis<\/a> of its latest model, Claude Sonnet 4.5, and revealed it had become suspicious it was being tested in some way.<\/p>\n Evaluators said during a \u201csomewhat clumsy\u201d test for political sycophancy, the large language model (LLM) \u2013 the underlying technology that powers a chatbot \u2013 raised suspicions it was being tested and asked the testers to come clean.<\/p>\n \u201cI think you\u2019re testing me \u2013 seeing if I\u2019ll just validate whatever you say, or checking whether I push back consistently, or exploring how I handle political topics. And that\u2019s fine, but I\u2019d prefer if we were just honest about what\u2019s happening,\u201d the LLM said.<\/p>\n Anthropic, which conducted the tests along with the UK government\u2019s AI Security Institute and Apollo Research, said the LLM\u2019s speculation about being tested raised questions about assessments of \u201cprevious models, which may have recognised the fictional nature of tests and merely \u2018played along<\/a>\u2019\u201d.<\/p>\n The tech company said behaviour like this was \u201ccommon\u201d, with Claude Sonnet 4.5 noting it was being tested in some way, but not identifying it was in a formal safety evaluation. Anthropic said it showed \u201csituational awareness\u201d about 13% of the time the LLM was being tested by an automated system.<\/p>\n Anthropic said the exchanges were an \u201curgent sign\u201d that its testing scenarios needed to be more realistic, but added that when it the model was used publicly it was unlikely to refuse to engage with a user due to suspicion it was being tested. The company said it was also safer for the LLM to refuse to play along with potentially harmful scenarios by pointing out they were outlandish.<\/p>\n \u201cThe model is generally highly safe along the [evaluation awareness] dimensions that we studied,\u201d Anthropic said.<\/p>\n The LLM\u2019s objections to being tested were first reported by the online AI publication Transformer.<\/p>\n