AI-Generated Passwords Are Apparently Quite Easy to Crack

The era of AI has not been particularly great for cybersecurity. We know that vibe-coded websites and apps have been a hotbed of security flaws that leave the platforms vulnerable to attacks. It turns out that vibe-generating your passwords also puts you at risk. New research from cybersecurity firm Irregular found that passwords generated by large language models appear strong but are “fundamentally insecure” and shockingly easy to guess.

To determine the capability of popular AI models to act as your password generator, Irregular asked Claude, ChatGPT, and Gemini to generate 16-character, secure passwords that include special characters, numbers, and letters—and in some cases, passphrases. The models are capable of spitting out strings of characters that appear like any auto-generated password from your password manager or built-in password tools provided by Google or Apple. They even scored as being strong passwords according to online checkers like KeePass.

And yet, they proved quite crackable. Why? Because large language models are not particularly good at randomization. For instance, when the researchers asked Anthropic’s Claude Opus 4.6 model to generate 50 unique passwords, it actually used a very predictable pattern. Every password it generated started with a letter, most of which were an uppercase “G.” The second character was almost always the digit “7.” The characters “L,” “9,” “m,” “2,” “$” and “#” appeared in all 50 passwords, and most of the alphabet never appeared in any of the 50 options.

Other models had similar issues. OpenAI’s ChatGPT started nearly every single password with the character “v” and nearly half of all passwords used “Q” as their second character. Like Claude, ChatGPT stuck with a narrow subset of characters when generating a password, instead of making use of the full alphabet. Google’s Gemini had the same patterns, with most of its passwords starting with either an uppercase or lowercase “K.” The characters that followed were almost always some variation of “#,” “P,” or “9.”

The researchers even noted that the LLMs seemed to make choices that would make passwords appear more random but actually reveal a lack of randomness. The researchers noted that there were zero instances of repeating characters throughout the generated passwords. While that makes the outputs look more randomized at a glance, Irregular noted, “Probabilistically, this would be very unlikely if the passwords were truly random.”

Password strength is generally measured by bits of entropy, which is meant to measure how many guesses it would take to crack a password. For instance, if you could only choose between two passwords—let’s say “11111” or “12345”—there is a 50% chance of someone guessing your password. That means there is 1 bit of entropy. If your password can be any one of 1,000 words, it would take a person up to 1,000 tries to guess it, which is about 10 bits of entropy. By having a high variance of options for each character within a password, you create more bits of entropy, which makes it harder to brute-force the password. A password with 20 bits of entropy generates about one million possibilities, but that can be cracked in a matter of seconds if the attacker is using modern, high-end GPUs to generate guesses. Meanwhile, a password with 100 bits of entropy would take trillions of years to crack.

So just how bad are LLM-generated passwords? According to the researchers, if a truly secure password would produce 6.13 bits of entropy per character, LLM-generated outputs are closer to 2.08 bits of entropy. If a standard, truly secure 16-character password would have about 98 bits of entropy, the LLMs were only able to spit out passwords with an estimated 27 bits of entropy, making them extremely susceptible to brute-force attacks.

It’d be easy enough to avoid this as an individual: Just don’t use an LLM to generate your password. Gemini even offers a prompt telling you that you shouldn’t use passwords it creates for sensitive accounts. But increasingly, people are offloading coding and other tasks to AI agents. And even those agents are prone to relying on LLMs to do password creation. The researchers said they were able to find common LLM-created patterns in the wild by searching GitHub and other technical documents, meaning there are password-protected apps and services out there just waiting to be cracked.

Irregular doesn’t seem to think this is a problem that can be addressed with a simple update, either. “People and coding agents should not rely on LLMs to generate passwords. Passwords generated through direct LLM output are fundamentally weak, and this is unfixable by prompting or temperature adjustments: LLMs are optimized to produce predictable, plausible outputs, which is incompatible with secure password generation,” the company said.

Gizmodo reached out to Anthropic, OpenAI, and Google for comment on the findings of the researchers, but did not receive a response at the time of publication.