Study shows large language models suffer cognitive decline from junk data

Researchers have demonstrated that large language models (LLMs) can experience cognitive decline similar to the effects of excessive consumption of low-quality online content in humans. The study, conducted over several months, involved feeding LLMs viral Twitter data consisting of short, high-engagement posts.

The findings revealed a significant decline in the models" cognitive abilities: reasoning capabilities decreased by 23%, long-context memory dropped by 30%, and personality assessments indicated increases in narcissism and psychopathy. Notably, even after retraining the models on clean, high-quality data, the cognitive impairments did not fully recover, indicating that the representational "rot" persisted.

This research suggests that the impact of poor-quality data on LLMs leads to a permanent cognitive drift, challenging the notion that bad data only results in bad output. The study highlights the phenomenon akin to "doomscrolling" in artificial intelligence, raising concerns about the long-term effects of exposure to junk data.

For more information, the full study can be accessed at llm-brain-rot.github.io.