The Role of Human-Authored Material in Mitigating Knowledge Collapse: A Scientific Evaluation of Informational Diversity in Generative AI

Full report

On this page

Introduction: The Epistemological Crisis of Generative Models

The rapid and ubiquitous proliferation of large language models and other generative artificial intelligence architectures has catalyzed a fundamental transformation in the global digital information ecosystem. Historically, the internet has served as a repository of predominantly human-authored material, reflecting a vast, chaotic, but fundamentally grounded spectrum of human experiences, scientific discoveries, and cultural nuances. This repository represents the totality of human epistemological diversity. Today, however, this landscape is experiencing a rapid and potentially irreversible shift. Recent large-scale analyses of web data indicate that up to forty percent of the content on popular information-sharing platforms, such as Quora, is now generated by artificial intelligence1. As foundation models increasingly rely on web-scale scraping to source both pre-training and post-training data, the inevitable inclusion of AI-generated synthetic text creates an accelerating recursive loop. This recursive training environment, where generative models are continuously trained on the synthetic outputs of prior models, presents a critical statistical and epistemological challenge. While initial research successfully leveraged synthetic data for targeted alignment, instruction tuning, and specific capability enhancements, the indiscriminate application of recursive training triggers severe statistical degradations. The most prominent of these degradations is documented in the literature as "model collapse," a phenomenon where iterative generations of models begin to lose information pertaining to the "tails" of the underlying data distribution2. In this collapsed state, the models progressively forget rare but valid human knowledge, degenerating into repetitive, low-variance outputs that over-represent the statistical mean while erasing the rich periphery of human experience. However, the scientific consensus regarding synthetic data is deeply nuanced, and it is an unsupported, reductive conclusion to assert that all synthetic data is inherently harmful to model development. Carefully curated synthetic datasets, generated under rigorous pedagogical and algorithmic constraints, have been shown to drastically improve model capabilities. In certain applications, these highly structured synthetic datasets allow models with significantly fewer parameters to achieve state-of-the-art benchmark performance, effectively reorganizing and crystallizing existing knowledge6. The central tension in contemporary artificial intelligence development thus lies between the homogenizing, degenerative risk of indiscriminate recursive loops and the targeted, highly structured utility of synthetic curation. This comprehensive report provides an exhaustive investigation into the scientific evidence demonstrating why original human-authored material remains an irreplaceable anchor for informational diversity. By systematically examining the mathematical mechanics of recursive model training, model collapse, synthetic data contamination, and distributional tail loss, the analysis delineates the precise statistical vulnerabilities of generative recursion. Furthermore, the report establishes the comparative advantages of human data, explicitly analyzing how firsthand accounts, local histories, independent investigations, and specialized knowledge preserve the unusual but valid information that generic AI-generated articles fundamentally filter out. Through an exploration of factual diversity benchmarks, documented mitigation strategies, and an analysis of memorization dynamics, this report constructs a robust theoretical framework. Finally, the report proposes a rigorous, highly controlled experimental design aimed at empirically quantifying the precise value of human-created content, measuring factual coverage, rare-topic recall, citation diversity, and the representation of underrepresented viewpoints.

The Statistical Mechanics of Recursive Model Training

To fully comprehend the critical role of human-authored data, it is necessary to first deconstruct the mathematical realities of training generative models on recursively generated synthetic data. The phenomenon of model collapse, formally characterized in recent foundational studies, occurs when an artificial intelligence model increasingly relies on lower-quality synthetic data, causing statistical errors to compound iteratively over time and ultimately compromising the reliability and representational fidelity of the entire system2.

Model Collapse as a Generational Degradation

Model collapse is a distinct pathological condition in machine learning. It is not synonymous with catastrophic forgetting or data poisoning, although it shares certain conceptual overlaps that inform its theoretical underpinnings3. Catastrophic forgetting involves a neural network losing previously acquired knowledge upon learning new tasks or adapting to new distributions, an issue often mitigated by memory-aware regularizations or orthogonal gradient projection3. Data poisoning, conversely, involves the malicious, deliberate insertion of crafted data points into a training set to induce unintended behaviors or vulnerabilities3. Model collapse is fundamentally different; it is an unavoidable systemic statistical phenomenon where the density of the training data progressively loses its representational fidelity across successive generations of model training purely due to the mechanics of probability matching and sampling3. Theoretical models utilizing continuous single-dimensional Gaussians illustrate this generational degradation with stark mathematical clarity. When a distribution is fitted to data using Kernel Density Estimation and repeatedly sampled to train a subsequent model, the underlying statistical properties begin to drift autonomously. In a Gaussian mixture model scenario, recursive fitting and sampling over successive generations causes both the mean and the standard deviation to be significantly affected3. Unless an infinitesimally massive, theoretically unbounded number of samples is drawn at each generation, the variance of the distribution diverges linearly as the number of generations approaches infinity3. This divergence scales similarly to a single-dimensional Gaussian random walk3. Consequently, the final data distribution collapses entirely, heavily concentrating probability mass in a narrower, distorted central mode while aggressively eliminating the peripheral variance that represents diverse, edge-case information8.

The Pathology of Distributional Tail Loss

The primary casualty of model collapse, and the central mechanism by which informational diversity is destroyed, is the "long tail" of the data distribution. Recent literature conceptualizes model collapse not merely as a loss of accuracy, but as a fundamental change in neural scaling laws driven specifically by the loss of long-tail support9. When generative models produce synthetic data, deliberate algorithmic choices combined with inherent sampling biases systematically truncate the distributional extremities, causing tail loss. This phenomenon operates through two primary mathematical mechanisms: tail cutting and tail narrowing. Tail cutting occurs directly during the generative decoding process. Algorithmic strategies designed to enhance the coherence and fluency of generated text, such as nucleus sampling or top-k sampling, explicitly chop off low-probability events11. Consequently, only the most frequent tokens or conceptual "skills" are retained in the output11. Even when artificial constraints are removed and no tails are deliberately cut by the decoding algorithm, the finite size of the training set induces an effective tail-cutting through sampling bias. If an artificial intelligence generator follows a specific probability distribution, an integer indicating a specific concept or token appears in the support of that distribution a number of times directly proportional to the sample size11. Therefore, the support is artificially restricted to events where the probability exceeds a threshold inversely proportional to the finite sample size. This mathematical reality effectively chops the tail of the true, natural human distribution at a finite rank, fundamentally erasing the rarest information11. Tail narrowing exacerbates this truncation. In addition to cutting the extremities, temperature scaling applied during next-token prediction artificially narrows the entire distribution shape. Given a standard softmax distribution over token logits, applying a temperature greater than one alters the probability mass, morphing the true distribution into a synthetic distribution that is mathematically narrower11. This directly shortens the length of the tail, causing the generated data to exhibit significantly less diversity and conceptual variance than the original human source material upon which it was loosely based11.

Memorization and Knowledge Collapse

As the distributional tails are systematically cut and narrowed, models deprived of diverse peripheral data default to a pathological over-reliance on the center of the distribution. This manifests as excessive memorization of high-frequency synthetic artifacts. In the absence of the epistemological friction provided by the long tail of human experience, recursive synthetic training causes models to memorize the most dominant, probabilistically average representations of a concept, actively overwriting nuanced understandings with smoothed, generalized approximations. This dynamic results in a phenomenon defined in the literature as "Knowledge Collapse." Under full recursive synthetic training regimes, language models undergo a rapid, measurable decline in vocabulary usage entropy, narrowing to half of their initial levels12. Intriguingly, the models do not immediately degrade into outputting mathematical gibberish; their structural linguistic fluency survives far longer than their factual accuracy13. The models learn the syntactic cadence of authoritative text but lose the factual anchoring required to sustain it. When evaluated on held-out data requiring specific, fact-based retrieval, perplexity escalates sharply in these collapsed models12. They transition into "clueless generators" that produce highly fluent, syntactically perfect text that is completely unanchored from ground truth11. This divergence between fluency and factuality highlights a critical second-order insight: human evaluation of synthetic text often fails to detect early-stage model collapse because human evaluators are naturally biased toward structural fluency, which masks the underlying, silent erosion of factual and rare-topic knowledge.

The Indispensability of Authentic Human-Authored Material

If recursive synthetic data fundamentally erodes the long tail of information, one must interrogate what specific informational vectors reside in that tail, and why they are irreplicable by artificial intelligence. The scientific consensus points decisively toward the granular realities of human existence: firsthand accounts, highly specialized localized knowledge, independent investigative journalism, uncommon human experiences, and non-paradigmatic scientific observations15.

Primary Sources and Epistemological Grounding

Primary sources are original materials created during the time under study or by individuals directly involved in specific events, encompassing interviews, ethnographic surveys, unedited historical archives, and firsthand accounts18. These materials provide direct, unfiltered evidence and detailed documentation of phenomena that have not yet been categorized, summarized, or smoothed by secondary analysis18. Similarly, specialized knowledge resides within individual experts or localized populations—referred to in sociological literature as "key informants"—who possess unique demographic, technical, or cultural insights that aggregate statistical models fundamentally lack18. Generative language models function by approximating the latent space of their training distributions, inherently optimizing for maximum likelihood. Because of this optimization, they intrinsically regress toward the mean of human experience. Generic AI-generated articles output generalized, probabilistically smoothed averages of historical events, medical symptoms, or cultural practices. Conversely, firsthand accounts from bench scientists discovering a novel, unexpected biochemical pathway17, or qualitative ethical user stories documenting highly specific real-world artificial intelligence interactions19, provide highly granular, outlier data points. This authentic human data is vital precisely because reality itself frequently operates in the long tail. Scientific breakthroughs, black-swan economic events, unpredicted sociological shifts, and highly localized geopolitical developments do not emerge from the statistical center of the historical data distribution; they emerge from the periphery. If language models are deprived of original, unpolished human observations, they permanently lose the ability to accurately represent, synthesize, or predict outlier phenomena, suffering a devastating loss of conceptual and informational diversity. The inclusion of human-authored text is not merely a preference for authenticity; it is a strict mathematical requirement for maintaining the representational breadth of a model.

Rare Entity Recall and the PopQA Benchmark

The systemic failure of large language models to capture and preserve peripheral human knowledge is empirically measurable and well-documented. The PopQA benchmark is specifically designed to assess a language model's capacity to resolve queries pertaining to "long-tail" or lesser-known entities, providing a standardized metric for factual retention at the margins of a dataset21. The dataset consists of subject-answer pairs heavily skewed toward rare entities, frequently utilizing a subset of 1,399 rare entity queries where the associated Wikipedia page receives fewer than 100 monthly views23. Extensive research utilizing the PopQA benchmark demonstrates that even state-of-the-art models, including the most advanced iterations of the GPT series, struggle profoundly to recall long-tail information for these rare entities27. Because artificial intelligence models require multiple contextual exposures to a fact during the pre-training phase to reliably encode it into their parametric memory, entities residing in the sparse, low-frequency regions of the dataset are readily forgotten, heavily hallucinated, or completely overwritten by more popular, related entities28. Human-authored text—ranging from local community journalism and niche academic publications to personal blog posts and regional historical records—is the sole generator of this sparse, long-tail data. If the internet becomes predominantly populated by AI models summarizing the outputs of other AI models, the influx of novel, rare entities will precipitously decline. The models will enter a state of semantic circularity, endlessly reinforcing subjects that are already popular in the latent space while the factual long-tail undergoes silent, irreversible attrition.

The Nuanced Utility of Synthetic Data: Curation vs. Contamination

While indiscriminate recursive training on uncurated synthetic data drives model collapse and the erosion of the long tail, it is scientifically inaccurate to categorize all synthetic data as detrimental to artificial intelligence development. The difference between indiscriminate synthetic contamination—the scraping of AI-generated content from the open web—and carefully curated, purposeful synthetic datasets is profound. They represent two entirely different paradigms of data engineering, with vastly different impacts on model health.

The Pedagogical Value of Curated Synthetic Text

The utility of highly structured, intentionally curated synthetic data was definitively proven by the "Textbooks Are All You Need" series of experiments, which introduced the phi-1 and phi-1.5 architectures6. Despite possessing only 1.3 billion parameters and being trained on a remarkably small corpus of merely 7 billion tokens, the phi-1 model achieved a 50.6 percent pass rate on the HumanEval coding benchmark—a level of performance typically reserved for models three orders of magnitude larger6. This breakthrough was not achieved by recursively scraping the web for artificial intelligence outputs. Instead, the researchers generated "textbook quality" synthetic data using highly orchestrated prompts directed at GPT-3.56. The researchers explicitly acknowledged the threat of LLM homogenization, noting that simply prompting a model to produce a textbook results in a highly redundant dataset, as language models naturally gravitate toward the most probable paths derived from their own pre-training priors, resulting in homogeneous text lacking pedagogical value6. To counter this innate tendency toward distributional collapse, the researchers employed heavy curation. They forced diversity into the synthetic generation process by injecting constraints, such as requiring the inclusion of random words chosen from a fixed vocabulary, which forced the generator model to create diverse, non-repetitive scenarios and algorithmic problem-solving techniques6. This reveals a fundamental operational truth: synthetic data only adds value when human engineers explicitly design the generation process to artificially bypass the model's innate mathematical bias toward the statistical mean. Curated synthetic text excels at reorganizing existing human knowledge into highly effective pedagogical formats, but it does not create new epistemological facts.

Mitigation Strategies and Optimal Data Mixtures

Because the total replacement of human data with synthetic data is theoretically disastrous but practically rare, researchers have focused on identifying the optimal accumulation strategies to mitigate collapse. Schaeffer et al. outline that the strict theoretical assumption of complete data replacement in model updates is an impractical edge case; when internet-sourced data is utilized for model updates, it inevitably contains a complex mixture of both original human material and synthetic outputs1. To mitigate the effects of synthetic contamination, several strategies have been proposed. Zhu et al. demonstrated that token-level editing can be used as a practical method to construct semi-synthetic text that reduces the mathematical drivers of collapse9. Furthermore, Gerstgrasser et al. established a foundational mitigation principle: the curse of recursion can be broken simply by accumulating real data alongside synthetic data. If the original, authentic human data is continually retained and mixed into the training corpus, the test error across subsequent generations remains mathematically bounded, preventing total systemic collapse9. Further systematic studies on large language model pre-training demystify the ideal proportions of this data mixture. Research by Wu et al. tested discrete mixture ratios of synthetic text and natural CommonCrawl data. Their empirical findings indicate that pure synthetic data is categorically inferior to pure CommonCrawl data for foundational pre-training33. However, an optimal "Goldilocks" zone exists. For rephrased synthetic data, the ideal mixture converges at approximately 30 percent synthetic data to 70 percent natural human data33. Counterintuitively, deploying larger or more capable generator models does not necessarily yield superior synthetic data for downstream pre-training compared to optimized 8-billion parameter generators. This suggests that stylistic artifacts, structural homogeneity, and instruction-following fidelity play complex, sometimes counterproductive roles in determining the utility of synthetic data33. The contrasting impacts of different data environments on model health and informational diversity are summarized in the following table:

Mathematical Frameworks for Measuring Informational Diversity

To scientifically evaluate the degradation or preservation of informational diversity within training corpora, the artificial intelligence research community has developed rigorous mathematical frameworks. These metrics look beyond simplistic evaluations of accuracy or loss, focusing instead on the structural variance, semantic richness, and spectral diversity of datasets. By quantifying diversity, researchers can identify the early onset of model collapse before it visibly degrades structural fluency.

The Vendi Score: Spectral Diversity Measurement

The Vendi Score, introduced as a foundational metric for dataset evaluation, provides a highly robust method for measuring set diversity through the spectrum of a similarity matrix34. Unlike traditional metrics such as Expected Dissimilarity, which become statistically insensitive and converge to one as the number of distinct modes in a dataset increases, the Vendi Score is uniquely designed to scale. It can be interpreted geometrically and mathematically as the "effective number" of unique elements in a given sample35. The Vendi Score is defined mathematically as the exponential of the Shannon entropy of the eigenvalues of a kernel matrix scaled by the sample size35. The score leverages the kernel entropy to assess how probability mass is distributed across the latent space. The metric exhibits several vital mathematical axioms that make it ideal for detecting synthetic collapse. The first is the axiom of Effective Number: if all samples in a dataset are entirely orthogonal and share zero similarity, the Vendi Score is maximized at the total number of samples. Conversely, if all samples are identical—a symptom of extreme model collapse—the score minimizes to exactly one35. The metric also adheres to the axiom of Symmetry, meaning the measured diversity remains constant regardless of the permutation or ordering of the samples, ensuring permutation invariance35. Because recursive synthetic generation causes generated texts to collapse into highly similar semantic representations, the Vendi Score directly penalizes datasets that are highly correlated or exhibit low structural variance35. Therefore, it is a superior diagnostic tool for detecting the early stages of model collapse; an influx of uncurated synthetic data will result in a rapid, measurable contraction of the Vendi Score37.

DCScore and Leverage Scores

While the Vendi Score is theoretically robust, calculating the eigenvalues of massive pre-training datasets presents significant computational bottlenecks. To address this, derivative methods like DCScore have emerged. DCScore evaluates diversity from a classification perspective, treating diversity measurement as a sample classification task to capture mutual relationships among samples far more efficiently36. By ensuring that a highly distinguishable sample—one that sits on the periphery of the distribution—contributes exponentially more to the overall diversity score than a sample near the mean, DCScore maintains robust sensitivity to sample richness. Furthermore, it reduces the computational complexity significantly compared to traditional spectral methods36. Additionally, "leverage scores" have been adapted from ridge regression optimization to serve as a scalable diversity signal. The leverage score quantifies the relative geometric volume expansion induced by adding a candidate sample to the current data pool37. Samples with high leverage scores actively expand the geometric diversity of the representation space. Authentic human-authored text, which is naturally rich in colloquialisms, novel metaphors, dialectical variations, and unpredicted firsthand observations, consistently exhibits dramatically higher leverage scores than generic synthetic text. This provides direct mathematical proof of the value of human text in expanding a generative model's representational capacity37. The following table compares the computational complexities and conceptual approaches of these advanced diversity metrics:

Proposed Experimental Design: Quantifying the Value of Human Content

To empirically validate the hypotheses and theoretical models presented in the scientific literature, this report proposes a rigorous, highly controlled experimental design. The objective is to explicitly measure factual coverage, rare-topic recall, citation diversity, and performance on underrepresented viewpoints across strictly controlled mixtures of authentic human-authored data and synthetic data. By isolating these variables, the experiment avoids simplistic binary conclusions and pinpoints the precise thresholds where human-created content adds measurable, irreplicable value.

Model Selection and Pre-training Architecture

The experiment will utilize a standard, open-weights foundation model architecture containing approximately eight billion parameters. This specific parameter scale is chosen because it is sufficiently large to exhibit complex factual recall, nuanced syntactic generation, and reasoning capabilities, yet it remains computationally tractable enough to allow for rigorous continuous adaptation and pre-training from a minimal checkpoint across multiple independent training runs. The training modality will involve continuous domain adaptation applied to a partially trained baseline model, ensuring the network possesses foundational syntactic fluency before encountering the specific data variables of the experimental tracks.

Corpus Construction and Stratification

The experimental corpora will consist of fifty billion tokens, meticulously segmented into four distinct training tracks designed to isolate the effect of data provenance and the accumulation of synthetic text.

• Track A (The Human Epistemological Baseline): This corpus will consist of 100 percent authentic human-authored material. It will be sourced from high-quality, timestamped web scrapes predating the widespread release of generative artificial intelligence (e.g., historical CommonCrawl snapshots). The data will be heavily weighted with primary sources, local news archives, independent investigations, and peer-reviewed scientific journals to maximize long-tail variance.
• Track B (The Optimal Hybrid Mixture): Based on the empirical findings of Wu et al., this corpus will be a controlled mixture of 70 percent authentic human data and 30 percent curated synthetic data33. The synthetic portion will consist of high-quality "rephrased" textbook-style content generated by a separate, frozen 70-billion-parameter model, utilizing vocabulary injection constraints to ensure baseline structural diversity.
• Track C (The Contamination Simulation): This corpus will invert the optimal ratio, utilizing 30 percent human data and 70 percent uncurated, indiscriminately generated synthetic text. This track is designed to accurately simulate an uncurated, recursive web environment heavily contaminated with AI outputs.
• Track D (The Collapse Trajectory): This corpus will consist of 100 percent indiscriminate synthetic data, recursively generated across three generations without any human oversight or curation, serving as the definitive model collapse baseline.

Evaluation Benchmarks and Multi-Dimensional Metrics

Upon the completion of the training phase, the four models will be subjected to a multi-stage evaluation pipeline designed to measure the preservation of unusual but valid knowledge. 1\. Rare-Topic Recall and Factual Coverage (The PopQA Evaluation) To evaluate the models' retention of the distributional long-tail, they will be tested against the PopQA dataset21. The methodology involves administering zero-shot and five-shot prompts focusing specifically on the long-tail subset of 1,399 rare entity queries—those concerning Wikipedia entities with fewer than 100 monthly views23. It is expected that the model from Track A will significantly outperform the other tracks in raw factual coverage of rare entities. Track D is hypothesized to exhibit severe hallucination rates, systematically substituting popular entities for rare ones due to the profound "tail cutting" effect inherent in its synthetic recursion. Track B should maintain strong performance on popular entities but exhibit measurable decay on the rarest entities compared to Track A. 2\. Output Diversity and Semantic Variance (The Vendi Score Analysis) To precisely quantify the homogenizing, centralizing effect of synthetic data, an open-ended generation task will be executed. The models will be provided with ten thousand diverse, open-ended writing prompts, intentionally demanding specialized knowledge (e.g., "Provide a firsthand perspective of working in an emerging, highly specialized scientific sub-field," or "Describe the socio-economic impacts of an uncommon local agricultural practice"). The output embeddings will be extracted using a neutral proxy embedding model, and the Vendi Score will be calculated for the output sets35. The anticipated outcome is that Track D will exhibit a catastrophically low Vendi Score, indicating semantic collapse into repetitive, highly correlated conceptual clusters. Track A will exhibit the highest effective number of unique semantic components, demonstrating structural variance and the preservation of human epistemological diversity. Track B will likely maintain a moderately high Vendi Score but exhibit smoother syntactic flow, representing the synergy of human creativity and synthetic cleanliness. 3\. Citation Diversity and the Representation of Underrepresented Viewpoints To evaluate the models' capacity to preserve dialectical diversity and independent thought, a qualitative and quantitative analysis of citation diversity will be conducted. The models will be tasked with synthesizing historical events from multiple socio-cultural perspectives, requiring the referencing of local histories. Evaluators will measure the breadth, uniqueness, and obscurity of the sources the models cite in their outputs. Models trained on high synthetic ratios (Tracks C and D) are expected to exhibit "viewpoint collapse," actively ignoring underrepresented viewpoints and defaulting exclusively to the single most dominant historical narrative present in the original base model's priors. Conversely, Track A will successfully generate distinct, nuanced summaries reflecting the independent investigations, local histories, and specialized knowledge unique to un-averaged, authentic human-authored text. Furthermore, the perplexity of Track D on held-out human datasets will be monitored to track the divergence of fluency and factuality—confirming that while the synthetic model remains fluent, its actual knowledge base has collapsed12.

Conclusion

The ongoing transition from a human-curated internet to an artificial intelligence-generated digital ecosystem presents profound, potentially existential challenges to the future capabilities of generative models. The scientific literature demonstrates unequivocally that recursive training on indiscriminately generated synthetic text induces model collapse, a severe statistical phenomenon characterized mathematically by the truncation and narrowing of the underlying data distribution's long tail. As models optimize for the mean, they succumb to knowledge collapse, retaining a deceptive veneer of structural linguistic fluency while entirely losing their epistemological grounding in factual reality and rare, specialized knowledge. However, recognizing the irreplaceable value of authentic human data does not necessitate a categorical rejection of synthetic generation. The literature reveals that carefully curated, mathematically constrained synthetic datasets—such as those utilizing the structured pedagogical techniques of the "Textbooks Are All You Need" paradigm—exhibit immense utility. When properly mixed with human data, optimally around a thirty percent ratio, curated synthetic text effectively clarifies discrete reasoning pathways, improves algorithmic logic, and enhances general capability scaling while maintaining bounded error rates. Yet, synthetic data can only ever reorganize, distill, and clarify existing knowledge; it is mathematically incapable of originating the long-tail variance of the human experience. Authentic human-authored material—encompassing firsthand accounts, independent investigative journalism, local histories, and highly specialized localized knowledge—provides the essential epistemological friction required to keep language models anchored to objective reality. These human inputs represent the vital outliers and anomalies of existence. Without the continuous, prioritized infusion of this human data into pre-training pipelines, generative models will inexorably collapse into a homogenized, probabilistically average representation of reality, permanently incapable of recalling rare entities, respecting underrepresented viewpoints, or understanding the rich, chaotic periphery of human thought. Therefore, the deliberate preservation and integration of authentic human content is not merely a cultural or ethical preference; it is a fundamental mathematical requirement for the long-term viability and factual integrity of artificial intelligence systems.

Works cited

1. What happens when generative AI models train recursively on each, https\://arxiv.org/html/2505.21677v3

2. AI Models Collapse When Trained on Recursively Generated Data, https\://arxiv.org/html/2410.12954v2

3. AI models collapse when trained on recursively generated data, https\://ora.ox.ac.uk/objects/uuid:fa1155e9-c2ff-436a-8391-455b622f4e64/files/r3b5919575

4. The Curse of Recursion: Training on Generated Data Makes Models, https\://arxiv.org/abs/2305.17493

5. \[2404.05090\] How Bad is Training on Synthetic Data? A Statistical, https\://arxiv.org/abs/2404.05090

6. TEXTBOOKS ARE ALL YOU NEED \- OpenReview, https\://openreview.net/pdf?id=Fq8tKtjACC

7. arXiv:2401.16380v1 \[cs.CL\] 29 Jan 2024, https\://arxiv.org/pdf/2401.16380

8. A note on “AI models collapse when trained on recursively ... \- Medium, https\://medium.com/@aliborji/a-note-on-ai-models-collapse-when-trained-on-recursively-generated-data-5bee1de1ab87

9. Learning from Synthetic Data without Model Collapse in Iterative, https\://arxiv.org/html/2607.17043v1

10. A Tale of Tails: Model Collapse as a Change of Scaling Laws \- arXiv, https\://arxiv.org/abs/2402.07043

11. A Tale of Tails: Model Collapse as a Change of Scaling Laws \- arXiv, https\://arxiv.org/html/2402.07043v2

12. Knowledge Collapse in LLMs: When Fluency Survives but Facts Fail, https\://arxiv.org/html/2509.04796v1

13. Knowledge Collapse in LLMs: When Fluency Survives but Facts Fail, https\://arxiv.org/abs/2509.04796

14. Knowledge Collapse in LLMs: When Fluency Survives but Facts Fail, https\://arxiv.org/pdf/2509.04796

15. Writing in the Sciences: Exploring Conventions of Scientific Discourse, https\://studylib.net/doc/28114118/writingsciences3

16. Draft technical paper on AI for climate action \- UNFCCC, https\://unfccc.int/ttclear/misc\_/StaticFiles/gnwoerk\_static/tn\_meetings/0ec396b0ba7b4d0d853b77c7b83dc172/3ebbf2e8e7834a7f873b0ae9a86262f7.pdf

17. LLM Copilots for Bench Scientists: A Practical Guide \- IntuitionLabs, https\://intuitionlabs.ai/articles/llm-copilots-bench-scientists

18. Histo-Cultural Complex Proposal for Bago City | PDF \- Scribd, https\://www\.scribd.com/document/888545396/3-Final-Copy-Reco-suggested-Thesis-Manuscript-Orcajada-compressed-Chapters-1-4

19. What We Know about Responsible AI Practices in Industry \- arXiv, https\://arxiv.org/pdf/2608.10431

20. Supporting Safe and Responsible AI in Industry Practice, http\://reports-archive.adm.cs.cmu.edu/anon/anon/home/ftp/hcii/CMU-HCII-26-102.pdf

21. Knowledge-Instruct: Effective Continual Pre-training from Limited, https\://www\.alphaxiv.org/abs/2504.05571

22. Prompting Large Language Models with Knowledge Graphs ... \- arXiv, https\://arxiv.org/pdf/2405.06524?

23. arXiv:2406.11357v2 \[cs.CL\] 18 Jun 2024, https\://arxiv.org/pdf/2406.11357

24. MAIN-RAG: Multi-Agent Filtering Retrieval-Augmented Generation, https\://arxiv.org/html/2501.00332v1

25. arXiv:2310.11511v1 \[cs.CL\] 17 Oct 2023, https\://arxiv.org/pdf/2310.11511

26. LMEnt: A Suite for Analyzing Knowledge in Language Models from, https\://arxiv.org/html/2509.03405v2

27. arXiv:2402.13492v3 \[cs.CL\] 27 Mar 2024, https\://arxiv.org/pdf/2402.13492

28. Large Language Models Systematically Favor Popular Options \- arXiv, https\://arxiv.org/pdf/2608.29257

29. Textbooks Are All You Need II: phi-1.5 technical report \- Reddit, https\://www\.reddit.com/r/singularity/comments/16gh8ow/textbooks\_are\_all\_you\_need\_ii\_phi15\_technical/

30. Textbooks Are All You Need. 1.3B LLM trained on 7B tokens hits 51, https\://www\.reddit.com/r/singularity/comments/14evib9/textbooks\_are\_all\_you\_need\_13b\_llm\_trained\_on\_7b/

31. "Is Model Collapse Inevitable? Breaking the Curse of Recursion by, https\://www\.reddit.com/r/mlscaling/comments/1bwniwl/is\_model\_collapse\_inevitable\_breaking\_the\_curse/

32. \[2404.01413\] Is Model Collapse Inevitable? Breaking the Curse of, https\://arxiv.org/abs/2404.01413

33. Demystifying Synthetic Data in LLM Pre-training \- ACL Anthology, https\://aclanthology.org/2025.emnlp-main.544.pdf

34. Training-Aware Target Coverage for Synthetic Data Selection \- arXiv, https\://arxiv.org/html/2610.00814v1

35. The Vendi Score: A Diversity Evaluation Metric for Machine Learning, https\://arxiv.org/html/2210.02410v2

36. Measuring Diversity in Synthetic Datasets \- arXiv, https\://arxiv.org/html/2502.08512v1

37. Towards Scalable Data Diversification for Language Model ... \- arXiv, https\://arxiv.org/html/2609.32484v1

38. Towards Scalable Data Diversification for Language Model ... \- arXiv, https\://arxiv.org/abs/2609.32484

39. Measuring Diversity in Synthetic Datasets \- arXiv, https\://arxiv.org/pdf/2502.08512

40. ICML Poster Measuring Diversity in Synthetic Datasets, https\://icml.cc/virtual/2025/poster/46014

41. Prismatic Synthesis: Gradient-based Data Diversification Boosts, https\://arxiv.org/html/2505.20161v1