I asked Claude for 37,500 random names, and it can't stop saying Marcus
An intriguing experiment revealed that Claude, an advanced AI, has an unusual fondness for the name "Marcus" when asked to generate random names, often repeating it with surprising frequency. This phenomenon highlights a fundamental limitation of large language models (LLMs) in truly mimicking randomness, sparking widespread discussion on their internal biases and deterministic nature. The findings challenge common assumptions about AI capabilities and prompt users to consider how LLMs derive their 'creative' outputs.
The Lowdown
Benji Smith conducted an extensive experiment asking Claude, an AI, to generate 37,500 random names across various models and prompts. The surprising and amusing discovery was Claude's overwhelming preference for the name "Marcus", demonstrating a significant bias in what was expected to be a random process. This study delves into the underlying mechanisms of LLMs and their limitations in generating truly unpredictable outputs.
- The name "Marcus" appeared 4,367 times, accounting for 23.6% of all male names generated.
- Claude Opus 4.5, in particular, returned "Marcus" in 100 out of 100 simple prompt attempts, indicating near-perfect determinism.
- Nine specific parameter combinations resulted in zero entropy, producing identical outputs every time.
- While more elaborate prompts did increase the diversity of names, they often introduced new, different biases.
- Using random word seeds proved more effective than random noise in boosting the diversity of generated names.
- The full experiment involved significant API calls, costing $27.58.
Overall, the experiment serves as a compelling illustration that LLMs, despite their apparent creativity, operate on statistical probabilities and patterns learned from their training data, rather than genuine randomness or intuition.
The Gossip
Randomness Revealed: LLMs' Lack of True Luck
Many commenters quickly pointed out that LLMs are inherently deterministic systems and not designed for true randomness. They produce tokens based on learned probabilities, not a random number generator. Some jokingly suggest LLMs are just emulating human inability to pick random numbers (like always picking '7'). The discussion delves into the technical aspects of temperature settings and sampling distributions, explaining that while outputs aren't truly deterministic (temperature > 0), they are highly biased towards specific, high-probability tokens. This predictable behavior, some argue, is a feature, not a bug, for systems that learn patterns.
Bias Busters: Unpacking the 'Marcus' Mystery
Commenters speculated on why 'Marcus' specifically, or other names like 'Dorian,' become recurring favorites for LLMs. Theories range from specific names appearing frequently in training data (e.g., Marcus Aurelius, Marcus Crassus), to their being distinct tokens in the LLM's vocabulary (like TikToken). Personal anecdotes of LLMs obsessively repeating certain names (like 'Dorian') underscored the shared experience of encountering these biases. There's a humorous element too, with suggestions that famous figures like Gary Marcus might be 'living rent-free' in the AI's 'head'.
Mitigating Monotony: Strategies for Diversity
Given the LLM's inherent biases, users shared strategies to achieve more diverse or truly random-like outputs. Suggestions included asking the LLM to write and execute code for random selection from a predefined list, rather than relying on its internal 'randomness'. Others proposed more sophisticated prompting techniques, like asking for a long array of names and sampling from later elements, or prepending prompts with random English words to 'seed' more varied attention. These workarounds highlight that while LLMs aren't naturally random, they can be prompted to leverage external or internal mechanisms to simulate better diversity.