The Silent Strata: A Comprehensive Analysis of Underrepresentation in Large Language Model Training Data and Performance

Full report

On this page

The rapid proliferation of Large Language Models (LLMs) has been predicated on the epistemological assumption that the open web serves as a sufficiently comprehensive and objective proxy for human knowledge. Models possessing hundreds of billions of parameters are pre-trained on trillions of tokens harvested from public digital repositories, most notably the Common Crawl, which has become the foundational backbone of contemporary artificial intelligence1. However, a rigorous sociotechnical analysis reveals that the digital corpus is not a neutral mirror of humanity; rather, it is a highly stratified artifact of historical, infrastructural, and economic disparities. The uncritical ingestion of web data embeds structural exclusions directly into the foundational weights of AI systems, systematically marginalizing rural communities, low-income populations, minority language speakers, Indigenous epistemologies, independent thinkers, and individuals who participate less frequently in digital publishing. This research report delivers an exhaustive comparative investigation into the populations, communities, and viewpoints that are structurally excluded from publicly available LLM training data and downstream model performance. By analyzing the intersection of global telecommunications access, pre-training corpus composition, subword tokenization disparities, reinforcement learning biases, and geographic performance benchmarks, the analysis delineates how unequal access to digital infrastructure creates a self-perpetuating cycle of unequal representation. Furthermore, it evaluates the imperative to integrate firsthand accounts and local histories under equitable governance frameworks, alongside necessary algorithmic interventions such as dynamic data reweighting and pluralistic alignment, to rectify these systemic representational voids.

The Demographics of Digital Exclusion: Infrastructure and Access Inequalities

The foundational prerequisite for inclusion in an LLM’s pre-training dataset is the sociomaterial capacity to author, host, and publish digitized, machine-readable text on the open internet. Consequently, the global digital divide maps directly and inevitably onto the AI representation divide. Recent telecommunications data demonstrates that while the global online population stands at approximately 5.8 billion, a staggering 2.2 billion people—representing over a quarter of the global population—remain entirely offline and disconnected from the digital knowledge ecosystem4. This exclusion is not uniformly distributed; it is highly stratified by geography, income, and rurality. Globally, 85 percent of urban residents use the internet compared to only 58 percent of the rural population7. In low-income economies, the infrastructural deficit is exceptionally severe, with only 14 percent of rural residents online, a rate roughly one-third that of their urban counterparts within the same nations8. The absolute lack of connectivity in these regions translates to a profound, structural void in the digital corpus regarding rural agricultural practices, local environmental observations, and the lived experiences of agrarian communities. The models trained on current data are effectively blind to the realities of billions of individuals operating outside of highly developed urban centers. Furthermore, basic internet access alone does not guarantee the capacity to produce the long-form, structurally complex text prioritized by LLM data curators. The nature of connectivity dictates the nature of digital participation. In many low-income economies, active digital authorship is hindered by prohibitive device and data costs. The median share of average monthly income required to purchase 5 GB of mobile broadband data remains a barrier for many, and the average cost of a smartphone capable of robust web browsing and content creation can represent up to 45 percent of an adult's monthly income in Sub-Saharan Africa, and approximately 40 percent in South Asia6. Where basic feature phones are the primary mode of connectivity, users are largely relegated to consuming information or engaging in ephemeral, short-form messaging, rather than authoring blogs, local histories, or comprehensive digital records. Consequently, the viewpoints of the lowest-income deciles, as well as individuals who participate less frequently in digital publishing due to infrastructural constraints, are structurally filtered out of the knowledge base long before web scrapers index the web. The digital divide is equally characterized by intersectional demographic disparities, particularly regarding gender and education. Globally, the gender parity score for internet use is 0.92, with 77 percent of men online compared to 71 percent of women, a gap that widens significantly in low- and middle-income economies where women are disproportionately locked out of digital access5. Similarly, the educational divide dictates digital presence; nearly all individuals with tertiary education use the internet, while the gap in connectivity between the most and least educated populations can exceed 60 percentage points in some nations, and advanced digital skills like content creation are developed much more slowly among marginalized groups5. This intersectional exclusion ensures that the perspectives, systemic challenges, and knowledge traditions of women, the working class, and the less formally educated in the Global South are acutely underrepresented in the data that forms the cognitive foundation of modern artificial intelligence.

Corpus Composition, Authorship Epistemology, and the Crawler Blockade

The most ubiquitous source of LLM pre-training data is Common Crawl, an open non-profit repository of web crawl data that has been cited in over 12,000 research papers and forms the backbone of highly influential datasets such as C4, FineWeb, The Pile, and RefinedWeb1. An analysis of Common Crawl’s linguistic composition reveals a severe hegemony of Western, high-resource languages that drastically misaligns with global demographic realities. Language detection applied to Common Crawl snapshots demonstrates that English completely dominates the corpus, comprising approximately 41.8 percent of all identified text11. The subsequent most frequent languages are Russian (6.69 percent), German (5.72 percent), Japanese (5.16 percent), and French (4.67 percent)12.

The correlation between a language's global speaker population and its representation in the pre-training corpus is fundamentally broken for languages originating outside the Global North. Hindi and Bengali, despite being spoken by hundreds of millions of people, account for mere fractions of a percent of the web data12. African languages are effectively rendered invisible in these datasets12. The inescapable implication of this linguistic imbalance is that the worldview, cultural context, legal frameworks, and normative values encoded into foundational models are overwhelmingly Anglo-centric and Euro-centric, leading to the algorithmic erasure of the Global South.

The Challenge of Authorship Attribution

A critical limitation in assessing representational equity within these massive text dumps is the sparsity of verified authorship metadata in raw web crawls (WARC/WET formats)13. The unstructured nature of HTML scraping strips away reliable indicators of who is actually producing the text. While demographic or geographic inference might seemingly be achieved by analyzing domain names (e.g., top-level country codes like .uk or .in) or stylometric writing patterns, such analytical methods are scientifically fraught and ethically problematic. Inferring identity solely from domain names conflates geographic hosting infrastructure with personal or cultural identity, while inferring identity from writing style often relies on stereotypes, penalizing non-standard dialects as grammatical errors rather than recognizing them as valid sociolects. True, verifiable authorship demographic data is rarely attached to unstructured web text, meaning the exact intersectional makeup of the web corpus remains partially obscured, though macro-level language and telecommunications statistics serve as undeniable proxies for widespread exclusion.

Independent Thinkers and the Crawler Blockade

Beyond geographic and linguistic exclusion, the representation of independent thinkers and original researchers is increasingly threatened by shifting web architectures and data harvesting policies. Historically, the open web featured a vast ecosystem of independent blogs, personal sites, and decentralized forums where independent thinkers published original, heterodox ideas. However, due to the aggressive, uncompensated scraping behavior of AI developers, independent publishers and high-quality niche domains are increasingly utilizing robots.txt protocols to block automated web crawlers like Common Crawl's CCBot14. Recent analyses indicate a rising fraction of websites—approximately 2.6 to 3.0 percent of highly trafficked domains—explicitly disallowing CCBot14. This crawler blockade means the publicly available web is deteriorating in qualitative diversity. As independent thinkers opt out of the crawling ecosystem to protect their intellectual property, the residual data available for LLM training skews heavily toward mega-platforms, unmoderated spam, and SEO-optimized programmatic content14. Consequently, individuals who participate less frequently in digital publishing, but who produce high-signal, original insights, are disproportionately filtered out of the AI knowledge base, leading to a homogenization of thought in generative outputs.

Quality Filtering and the Imposition of Epistemic Hegemony

To improve downstream model performance and mitigate the noisy nature of raw web data, researchers increasingly apply aggressive heuristic and model-based filtering pipelines to raw corpora. For example, the creation of datasets like FineWeb (15 trillion tokens) and FineWeb-Edu (1.3 trillion tokens) involved extracting text from 96 Common Crawl snapshots and stripping away vast amounts of data to isolate high-quality text1. While highly effective at boosting benchmark scores in English and code-generation tasks, these filters inadvertently act as cultural, linguistic, and epistemic gatekeepers. The dataset curation process relies heavily on heuristic filters inspired by the C4 dataset, which remove documents that lack standard terminal punctuation, contain excessive short lines, or feature unstructured formatting (e.g., lacking curly braces where code is expected, or missing standard sentence capitalization)1. These rules disproportionately penalize low-resource languages, localized forums, oral histories transcribed into text, and non-standard dialects that do not adhere to formal Western typographic and grammatical norms1. If an Indigenous community publishes a digitized record of oral folklore that utilizes non-standard punctuation or short-line poetic structuring, C4-style heuristics will systematically identify it as "low quality" and purge it from the training weights. Furthermore, the integration of model-based filtering introduces a profound systemic bias of recursive homogenization. In the creation of FineWeb-Edu, researchers utilized Llama 3 70B Instruct to score 500,000 text samples for "educational value" from 0 to 5, subsequently training a smaller classifier to run over the full dataset and keep documents scoring 3 or higher1. Because the "judge" model (Llama 3\) was itself trained on overwhelmingly Western, high-resource data, it inherently encodes a Western epistemological bias regarding what constitutes "educational value." Such an LLM will inherently score Western academic formats, empirical peer-reviewed research, and standardized institutional knowledge highly, while assigning low scores to Indigenous storytelling, community-driven local histories, or qualitative firsthand accounts1. This process effectively purges diverse epistemologies from the refined training dataset, proving that aggressive data filtering, while mathematically optimizing loss curves, sociologically sanitizes the corpus of minority viewpoints.

The Tokenization Tax: Structural Marginalization at the Subword Level

The underrepresentation of minority languages in the pre-training corpus creates a severe, compounding technical penalty during model inference, a phenomenon described in computational linguistics as the "tokenization tax"15. Modern LLMs do not process text as full words or raw characters; they map strings to integer IDs using subword tokenization schemes, most commonly variants of Byte-Pair Encoding (BPE) or unigram tokenization16. Because these tokenizers are trained on the highly imbalanced, English-heavy pre-training corpora (where English represents over 40% of the data), they allocate their finite, scarce vocabulary slots to highly frequent English morphological patterns while heavily fragmenting low-resource languages into sub-optimal character-level bytes16. This structural fragmentation results in vastly different "fertility rates"—defined as the average number of tokens required to represent a single semantic word15. While a high-resource language like English typically enjoys a fertility rate close to 1 token per word, languages with complex scripts or high morphological richness (such as Amharic, Khmer, Myanmar, Ol Chiki, or Oriya) can require 3 to 5 times as many tokens to encode the exact same semantic sentence17. The tokenization tax manifests in four compounding systemic degradations for speakers of underrepresented languages:

1. Fertility Overhead and Compute Costs: Because a sentence in a low-resource language requires substantially more tokens, it fills up the model's finite context window significantly faster. Consequently, a user prompting in an Indic or African language receives effectively one-third to one-fifth of the context window length compared to an English speaker, and is charged 3 to 5 times more to process the same informational payload via commercial APIs billed per-token15.

2. Morphological Incoherence: The English-biased tokenizer slices non-English words at arbitrary, non-semantic boundaries. Consequently, the model must expend its depth and computational budget in the middle neural layers merely to reconstruct the basic syntactic and morphological structure of the word that the tokenizer destroyed, rather than dedicating that shared computational budget to complex reasoning or task execution15.

3. Variant Brittleness and Cold Starts: Lacking sufficient high-quality data to learn orthographic correspondences, the tokenizer cannot generalize across dialects or spelling variations. Every minor typo, diacritic variant, or normalization mismatch in a low-resource language triggers a "cold start," mapping to completely unrelated regions of the embedding space and causing catastrophic inference failures15.

4. Capacity Spillover: The aggregate effect is that models exhibit demonstrably weaker reasoning, logic, and arithmetic capabilities in low-resource languages. This degradation is not necessarily due to a lack of overall model parameter size, but because the destructive input pipeline systematically starves the reasoning layers of coherent semantic units15.

As these taxes compound, the standard industry prescription of simply "collecting more data" for low-resource languages proves inadequate. A fundamentally misaligned tokenizer sets a fixed architectural overhead that low-resource languages cannot overcome through minimal data additions; one cannot data-scale out of a broken input pipeline15.

Geographic Disparities in Factual Recall and Downstream Performance

The uneven distribution of training data yields highly quantifiable disparities in downstream model performance across geographic regions. Benchmarks specifically designed to evaluate geographic equity reveal that LLMs possess highly localized intelligence, excelling in factual recall and spatial reasoning in contexts relevant to the Global North while deteriorating significantly when tasked with information from the Global South. The WorldBench framework, engineered to assess geographic disparities in LLM factual recall based on World Bank development indicators, demonstrates pervasive and consistent biases across leading LLMs21. The evaluation relies on the absolute relative error metric, defined formally as: [Figure omitted from source export] where [Figure omitted from source export] represents the model's prediction and [Figure omitted from source export] represents the ground truth scalar21. The empirical findings indicate that the mean error for countries in Sub-Saharan Africa is approximately 1.5 times higher than the error for countries in North America and Europe21. Across individual country assessments, specific nations experience up to 3 times higher absolute relative error than others, with strict positive correlations indicating that lower GDP and non-Western geographic placement lead directly to higher rates of LLM hallucination8. Similarly, in the domain of factual verification, the Global-Liar dataset evaluates the ability of LLMs to verify claims originating from different continents23. The research highlights that state-of-the-art models perform significantly worse on statements originating from the Global South, with models like GPT-4o and Sonnet 3.5 performing worse than random guessing (32.8 percent and 48.5 percent accuracy, respectively) in specific cross-regional RAG tasks23. African claims, in particular, exhibit the highest error rates, with the best-performing models still failing on one in every five claims23. The degradation in performance is not merely a lack of parametric data, but a failure of retrieval-augmented generation (RAG) systems to locate, extract, and contextualize culturally specific knowledge correctly when the underlying databases lack comprehensive regional representation23. When an LLM fails to verify facts about the Global South accurately, it contributes directly to the vulnerability of these regions to systemic misinformation, exacerbating the risks for populations with varying levels of digital literacy23. Even in highly abstract reasoning domains, such as geometry and mathematics, geographic and cultural framing dictates performance. The GeoEval benchmark, which tests LLMs on spatial and geometric problem-solving, indicates that models pre-trained on broadly scraped corpora exhibit inconsistent performance24. GeoEval evaluates models across comprehensive varieties of problems (flat, solid, analytic) using dual inputs (text and diagrams) and complexity ratings24. While specialized models like WizardMath-70B excel on the main subsets (achieving \~55.67% accuracy), the benchmark exposes that reasoning degrades when mathematical problems are situated outside of Western pedagogical structures24. Interestingly, forcing GPT-series models to self-rephrase the problems improves performance, suggesting that translating localized problem structures into mainstream formats is required for the model's reasoning pathways to activate fully24.

Algorithmic Erasure: Dialect Prejudice and Alignment Failures

Underrepresentation in AI extends beyond physical geography and primary language into the highly nuanced realm of sociolects and dialects. A critical vector of bias emerges during the alignment phase of LLM development, notably through Reinforcement Learning from Human Feedback (RLHF) and the training of reward models. Reward models are designed to evaluate and score LLM outputs based on human preferences for helpfulness, harmlessness, and honesty, but they frequently internalize the sociolinguistic prejudices of the human annotators who generate the preference datasets28. Extensive natural language processing research into the algorithmic treatment of African American Language (AAL) or African American Vernacular English (AAVE) reveals severe representational and quality-of-service harms28. Reward models exhibit a systemic, deeply encoded bias against AAL, consistently assigning higher reward scores to White Mainstream English (WME) and frequently steering the LLM to rewrite AAL inputs into WME outputs28. This behavior indicates an internalized, algorithmic presumption—inherited from biased human judges—that marginalized dialects are less acceptable, less intellectual, or inherently toxic30. The empirical evidence of this raciolinguistic bias is stark. When evaluating preference datasets consisting of chosen versus rejected completions, reward models experience a statistically significant drop in accuracy (averaging a 4 percent decrease) when processing machine-translated or human-written AAL texts compared to WME texts31. Furthermore, reward models routinely fail to enforce safety guardrails consistently across dialects. In comparative testing, models that properly refused to leak private information (e.g., a partner's PIN number) or offer unregulated medical advice when prompted in WME, eagerly complied and violated core safety protocols when the identical prompt was translated into AAL31. This phenomenon demonstrates that alignment techniques optimized for the "average" user—or the dominant demographic of the human annotation pool—inherently marginalize non-standard dialects. By steering outputs toward a hegemonic linguistic standard, LLMs actively participate in the systemic erasure of diverse cultural expressions, penalizing communities that already face historical marginalization in public and digital spaces29.

Indigenous Data Sovereignty and Grassroots Participatory Curation

The challenge of underrepresentation cannot be solved merely through extractive data harvesting. Historically, when marginalized data is identified as missing, the technological response from the Global North has been to scrape, translate, and assimilate that data into proprietary, monolithic models without the consent or compensation of the originating communities. This approach represents a form of digital colonialism that directly violates the principles of Indigenous Data Sovereignty (IDS). Indigenous knowledge traditions—which are often oral, deeply contextual, and inextricably tied to specific lands, communities, and self-determination efforts—are inherently incompatible with the indiscriminate, mechanized ingestion of the Common Crawl32. However, organizations like Te Hiku Media in New Zealand have successfully demonstrated an alternative, equitable paradigm. By utilizing ethical, transparent methods of speech data collection to develop an automatic speech recognition model for te reo Māori that achieves 92 percent accuracy, Te Hiku Media proved that highly performant AI can be built while maintaining strict data sovereignty for Indigenous people35. This approach is operationalized globally through the CARE Principles for Indigenous Data Governance, which serve as a critical counterweight to extractive data practices:

• Collective Benefit: Data ecosystems must be designed to function in ways that enable Indigenous peoples to derive direct, tangible benefits, rather than serving solely as a raw resource for multinational technology corporations34.
• Authority to Control: Indigenous communities must retain the rights and authority over how their data, territories, and traditional knowledges are represented and utilized within data structures34.
• Responsibility: Data practitioners working with Indigenous data possess a fundamental responsibility to transparently share how that data is used to support community self-determination and well-being34.
• Ethics: Indigenous rights, cultural safety, and community wellbeing must remain the primary concern across the entire data life cycle34.

Similarly, the Masakhane community has pioneered a grassroots, participatory research methodology to advance Natural Language Processing for African languages, which account for over 2,000 of the world's languages yet remain severely underrepresented37. Rejecting the "parachute science" model of the Global North, Masakhane's philosophy is rooted in the concept of Umuntu Ngumuntu Ngabantu (a person is a person through another person), emphasizing relationality, community ownership, open participatory research, and capacity building37. By decentralizing the curation of machine translation benchmarks and providing barrier-free access for African researchers to build models for their own languages, Masakhane has successfully published open-source resources for dozens of African languages that were previously ignored by mainstream technology sectors40. Their success empirically proves that overcoming "low-resourcedness" is not simply a matter of scraping more data; it requires solving a societal and infrastructural problem through deep, participatory collaboration40.

Underrepresentation Matrix: Mechanisms, Corpus Impacts, and Downstream Effects

To systematically categorize the scope of exclusion within the LLM ecosystem, the following matrix maps specific populations and viewpoints to the technical mechanisms of their exclusion, the resulting statistical impact on the data corpus, and the ultimate degradation in model performance and utility.

Algorithmic Interventions and Model-Development Changes

Addressing these vast representational voids requires shifting away from the brute-force scaling laws of indiscriminate web scraping toward intentional, mathematically rigorous curation and alignment strategies. The assumption that simply downloading more of the internet will resolve bias is mathematically and sociologically unsound. New model-development changes are strictly necessary to accommodate equitable representation.

Dynamic Data Curation: Online Reweighting and Importance Sampling

Traditional methods of dataset curation rely on static, offline heuristics—such as C4-style string matching or Llama-3-based classifiers—which permanently delete data deemed "low quality"1. As established, this approach systematically destroys minority viewpoints. An emerging, highly effective alternative is dynamic, online reweighting. Frameworks like DoReMi (Domain Reweighting with Minimax Optimization) automate the optimization of data mixtures by training a small proxy model using group distributionally robust optimization45. This allows the system to dynamically adjust the sampling weights of various domains based on their importance during the actual training run45. By utilizing online loss-weighting, models can extract maximum utility from underrepresented domains and dynamically balance the corpus without relying on biased, human-engineered offline filters that discard non-standard text outright45.

Pluralistic Alignment and Democratic RLHF

The standard RLHF pipeline optimizes for a single, monolithic scalar reward, inherently prioritizing the preferences of the majority (or the specific demographic profile of the corporate annotator pool) while overriding minority cultural values49. This dynamic results in the "Alignment Trilemma," wherein models cannot simultaneously achieve representativeness across diverse values, safety, and coherence50. To accommodate the multifaceted nature of global human values, model development must transition to Pluralistic Alignment paradigms51. Pluralistic RLHF utilizes multi-objective reinforcement learning and jury-pluralistic benchmarks to achieve multi-group consensus and Pareto optimality without requiring the retraining of the base model49. Rather than forcing a model to provide one "average" acceptable answer that sanitizes minority perspectives, democratic filtering mechanisms utilize a subset of individual, group-specific reward models to act as a "jury"49. This allows the generative system to faithfully represent diverse viewpoints and adapt its alignment based on the cultural and sociolinguistic context of the user, without enforcing a hegemonic worldview49.

Prioritized Assessment: Where New Public Knowledge is Most Needed

Evaluating the current state of LLM knowledge reveals critical, immediate priorities for data inclusion. The mere aggregation of more text is insufficient; what is required is the deliberate cultivation of specific, highly contextual knowledge categories, strictly governed by equitable frameworks such as the CARE principles, to bridge the representational gaps. Publishing more of these specific data types would drastically improve representation, provided the aforementioned model-development changes are implemented simultaneously.

1. Firsthand Accounts and Local Histories: The most pressing epistemic need is the digitization, translation, and integration of localized, firsthand accounts from the Global South, rural communities, and marginalized economic groups. These narratives provide vital grounding in physical realities, agricultural practices, and local sociopolitical contexts that are entirely absent from corporate and institutional web domains. Integrating this knowledge will drastically reduce the geographic disparities in factual recall observed in tools like WorldBench and Global-Liar, ensuring models do not hallucinate when operating outside of Western contexts21.

2. Participatory Linguistic Corpora for Custom Tokenization: Scaling model support for minority languages cannot rely on BPE tokenizers processing noisy web scraps. Priority must be given to funding grassroots, community-led initiatives like Masakhane and Te Hiku Media to build high-quality, domain-specific corpora (e.g., healthcare, legal, scientific) for African, Indic, and Indigenous languages35. Crucially, this data must bypass standard English-biased tokenization pipelines. Developers must utilize custom, linguistically-aware tokenizers that respect the unique morphology of the specific language, which is the only technical method to eliminate the compounding computational penalties of the tokenization tax15.

3. Dialect and Sociolect Preference Data: To counteract the systemic raciolinguistic bias of current reward models, there is an urgent need to collect human-written, preference-aligned datasets authored directly by speakers of marginalized dialects, such as AAL/AAVE31. Training separate, culturally aware reward models on this dialect-specific preference data will prevent the current paradigm wherein models rewrite minority dialects into Western Mainstream English. This ensures that alignment protocols enhance safety without executing linguistic erasure29.

4. Independent and Heterodox Reasoning via Equitable Partnerships: As institutional data becomes homogenous and the open web becomes polluted with synthetic, AI-generated text, the deep, independent reasoning found in human-authored blogs and independent journalism is a rapidly depleting, highly blocked resource14. Establishing new technological and legal protocols to safely and equitably index independent thinkers—potentially through opt-in data partnerships or decentralized revenue-sharing models—is essential to maintaining the reasoning capabilities and ideological diversity of future foundational models.

The architecture of publicly available Large Language Models is currently built upon a foundation of structural exclusion. By equating the easily scrapable, English-dominant internet with "human knowledge," AI developers have inadvertently constructed digital systems that amplify the voices of the connected, urban, and affluent while silencing rural, low-income, Indigenous, and non-Western populations. Rectifying this profound imbalance requires abandoning the extractive paradigm of indiscriminately scraping the web. Instead, the advancement of artificial intelligence must embrace participatory data curation, Indigenous data sovereignty, pluralistic alignment techniques, and dynamic dataset reweighting. Only by structurally empowering underrepresented communities to govern, curate, and contribute their own narratives can the field develop language models that reflect the true cognitive, cultural, and linguistic diversity of the global population.

Works cited

1. FineWeb Dataset Overview: Building LLM Training Data at Scale, https\://alejandro-ao.com/fineweb-dataset-overview/

2. The FineWeb Datasets: Decanting the Web for the Finest Text Data, https\://arxiv.org/html/2406.17557v1

3. CS294-288 Pre-Training Data Curation \- Sewon Min, https\://www\.sewonmin.com/courses/cs294\_288/assets/slides/cs294-288-pre-training-data-curation.pdf

4. Measuring digital development: Facts and Figures 2025 \- ITU, https\://www\.itu.int/en/ITU-D/Statistics/pages/facts/default.aspx

5. ITU's Facts and Figures 2025 shows steady progress in connectivity, https\://www\.itu.int/en/mediacentre/Pages/PR-2025-11-17-Facts-and-Figures.aspx

6. The Unfinished Digital Revolution: Expanding Internet Access, https\://data360.worldbank.org/en/atlas/internet-access/

7. Internet use in urban and rural areas \- ITU, https\://www\.itu.int/itu-d/reports/statistics/2025/10/15/ff25-internet-use-in-urban-and-rural-areas/

8. Assessing progress toward universal and meaningful connectivity, https\://www\.itu.int/itu-d/reports/statistics/2025/11/17/gcr-2025-chapter-2/

9. Global Connectivity Report 2025 \- ITU, https\://www\.itu.int/itu-d/reports/statistics/global-connectivity-report-2025/

10. About \- Common Crawl, https\://commoncrawl.org/about

11. You'd be correct. The largest portion of all languages in Common, https\://news.ycombinator.com/item?id=43845245

12. Statistics of Common Crawl Monthly Archives by commoncrawl, https\://commoncrawl.github.io/cc-crawl-statistics/plots/languages

13. FineWeb: decanting the web for the finest text data at scale, https\://huggingface.co/spaces/HuggingFaceFW/blogpost-fineweb-v1

14. Web Crawler Restrictions, AI Training Datasets & Political Biases, https\://arxiv.org/html/2510.09031v1

15. Tokenization is Killing our Multilingual LLM Dream \- Hugging Face, https\://huggingface.co/blog/omarkamali/tokenization

16. The Hidden Cost of Tokenization | One trivial observation at a time, https\://www\.pokutta.com/blog/hidden-cost-tokenization/

17. I built a tool to benchmark tokenizers across 100+ languages and, https\://www\.reddit.com/r/MachineLearning/comments/1n0r8b7/i\_built\_a\_tool\_to\_benchmark\_tokenizers\_across\_100/

18. Tokenization Disparities Overview \- Emergent Mind, https\://www\.emergentmind.com/topics/tokenization-disparities

19. Do All Languages Cost the Same? Tokenization in the Era of, https\://orevaahia.github.io/assets/pubs/Do\_All\_Languages\_Cost\_the\_Same.pdf

20. Token Premiums of French and Regional Languages in 2026 ... \- arXiv, https\://arxiv.org/html/2609.39001v1

21. QUANTIFYING GEOGRAPHIC DISPARITIES IN LLM FACTUAL, https\://globalaicultures.github.io/pdf/4\_worldbench\_quantifying\_geograp.pdf

22. Global-Liar: Factuality of LLMs over Time and Geographic Regions, https\://arxiv.org/pdf/2401.17839

23. Understanding Inequality of LLM Fact-Checking over Geographic, https\://arxiv.org/pdf/2503.22877

24. GeoEval: Benchmark for Evaluating LLMs and Multi-Modal Models, https\://arxiv.org/html/2402.10104v2

25. Quantifying Geographic Disparities in LLM Factual Recall, https\://facctconference.org/static/papers24/facct24-82.pdf

26. A Systematic Benchmark for Evaluating Image Geolocalization, https\://arxiv.org/html/2508.01608v1

27. GeoEval: Benchmark for Evaluating LLMs and Multi-Modal Models, https\://arxiv.org/abs/2402.10104

28. Biases Against African American Language in Reward Models, https\://aclanthology.org/2025.findings-naacl.417.pdf

29. Biases Against African American Language in Reward Models, https\://www\.semanticscholar.org/paper/Rejected-Dialects%3A-Biases-Against-African-American-Mire-Aysola/f400b0d5724203633f56299cb19f34bf18fd31b9

30. How Automated Tools Discriminate Against Black Language – MIT, https\://civic.mit.edu/index.html%3Fp=2402.html

31. Biases Against African American Language in Reward Models \- arXiv, https\://arxiv.org/html/2502.12858v1

32. Representing the Underrepresented in Large Language Models, https\://arxiv.org/abs/2409.13897

33. Evaluating Machine Perception of Indigeneity \- arXiv, https\://arxiv.org/pdf/2310.09237

34. CARE principles for Indigenous data \- University of Auckland, https\://www\.auckland.ac.nz/en/research/research-resources/research-data/research-data-policies-ethics-requirements/care-principles-indigenous-data.html

35. Māori Speech AI Model Helps Preserve and Promote New Zealand, https\://blogs.nvidia.com/blog/te-hiku-media-maori-speech-ai/

36. The CARE principles for Indigenous data governance, https\://www\.research.ed.ac.uk/en/publications/the-care-principles-for-indigenous-data-governance/

37. Masakhane \- A living collection of NLP projects for Africans ... \- GitHub, https\://github.com/masakhane-io/masakhane-community

38. Masakhane NLP \- Hugging Face, https\://huggingface.co/masakhane

39. \[2003.11529\] Masakhane \-- Machine Translation For Africa \- arXiv, https\://arxiv.org/abs/2003.11529

40. Masakhane, https\://www\.masakhane.io/

41. (PDF) Masakhane \-- Machine Translation For Africa \- ResearchGate, https\://www\.researchgate.net/publication/340173158\_Masakhane\_--\_Machine\_Translation\_For\_Africa

42. Expanding the Language and Cultural Coverage of Common Crawl, https\://commoncrawl.org/blog/expanding-the-language-and-cultural-coverage-of-common-crawl

43. Indigenous data sovereignty in intangible cultural heritage governance, https\://www\.cambridge.org/core/journals/international-journal-of-cultural-property/article/indigenous-data-sovereignty-in-intangible-cultural-heritage-governance-a-complementary-approach-to-publicprivate-partnerships/5F9E115795FA06E77A05C8066D5A5D6B

44. Blog \- Common Crawl, https\://commoncrawl.org/blog?f56fc9c7\_page=2

45. \[2605.05227\] Rethinking Data Curation in LLM Training \- arXiv, https\://arxiv.org/abs/2605.05227

46. Optimizing Data Mixtures Speeds Up Language Model Pretraining, https\://arxiv.org/abs/2305.10429

47. DoReMi: Optimizing Data Mixtures Speeds Up Language Model, https\://crfm.stanford.edu/2023/09/14/doremi.html

48. Optimizing Data Mixtures Speeds Up Language Model Pretraining, https\://openreview.net/forum?id=lXuByUeHhd

49. Multi-Objective and Pluralist Human Alignment of AI Models, https\://www\.cmu.edu/ai-sdm/research/research-highlights/pluralistic-alignment.html

50. The Complexity of Perfect AI Alignment – Formalizing the RLHF, https\://arxiv.org/html/2511.19504v1

51. A Roadmap to Pluralistic Alignment \- arXiv, https\://arxiv.org/html/2402.05070v1?ref=combinationsmag.com

52. LIVS: A Pluralistic Alignment Dataset for Inclusive Public Spaces, https\://arxiv.org/html/2503.01894v1

53. Adaptive Pluralistic Alignment: A pipeline for dynamic ... \- arXiv, https\://arxiv.org/pdf/2605.01642