Although the prompt injection attack seems ineffective at this point, Spader warned that prompt injection \u201cwas not among the dangers we contemplated\u201d when courts were first grappling with AI scrambling justice systems. In Connecticut, like many other court systems, the focus so far has been on policing AI outputs that damage trust in courts, like hallucinated citations or fabricated quotes, not inputs like prompt injections.<\/p>\n
Courts will most likely need to draft rules around prompt injection, too, Spader suggested, since Elliott\u2019s case shows the technology and its misuses are rapidly advancing. If not, attorneys may find their own clients using prompt injections to manipulate court filings without their knowledge, Spader warned.<\/p>\n
To Spader, there is a lesson to be learned from Elliott\u2019s failed prompt injection attacks that he thinks \u201creaches well beyond this case.\u201d<\/p>\n
Elliott seemingly turned to prompt injection after using AI to build his case as a pro se litigant without a legal expert to assist in drafting his arguments. Such use is widespread among pro se litigants these days, Spader acknowledged, but those inexperienced in the courtroom are seemingly using chatbots in a way that hurts their cases, he suggested.<\/p>\n
What frequently happens, Spader explained, is that pro se litigants build their argument backward, asking the chatbot to help them advocate only for their position, without ever asking the chatbot for the actual truth or to advance opposing arguments. This is \u201ca genuine hazard of the technology, and one that judges now see often,\u201d Spader said, as chatbot sycophancy then entrenches litigants in their arguments despite any ruling to the contrary. In Elliott\u2019s case, defending his arguments fiercely meant turning to prompt injection to try to force the court to agree with him.<\/p>\n
\u201cAn argument prompted only to agree with its author is, in the end, dishonest even with its author,\u201d Spader said. \u201cThose using these tools must ask them to test a position as readily as to advance it.\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"Although the prompt injection attack seems ineffective at this point, Spader warned that prompt injection \u201cwas not among…\n","protected":false},"author":2,"featured_media":856631,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[256,254,255,64,63,105],"class_list":["post-856630","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-artificial-intelligence","tag-artificialintelligence","tag-au","tag-australia","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/856630","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/comments?post=856630"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/856630\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media\/856631"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media?parent=856630"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/categories?post=856630"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/tags?post=856630"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}