Power of Participation in LLMs

Full report

On this page

Executive summary

Participation is not just a procedural nicety in AI. In the context of large language models, it is a way of allocating power: power over what problems are worth solving, whose language and knowledge count as data, which harms are recognized during evaluation, who can contest deployment, and who receives benefits from the system after release. The strongest theoretical traditions on participation all converge on this point. Arnstein’s classic framework treats participation as meaningful only when people gain actual influence over decisions that affect them. Gaventa’s power-cube lens adds that power is not only visible in formal decisions, but also hidden in agenda-setting and invisible in how social norms shape what people think is possible. Fung’s framework further clarifies that institutional design matters: who participates, how they communicate, and whether participation is linked to real public action are separate design choices, not one thing.

Applied to LLMs, this means that participation has to be mapped across the full model lifecycle. It can shape problem formulation, data collection and filtering, post-training preference learning, external red teaming, deployment monitoring, open evaluation, and governance processes. Where participation is substantive, it can improve representational coverage, surface harms earlier, reduce subgroup performance gaps, and increase legitimacy and adoption. Where it is shallow, it frequently becomes “participation-washing”: a legitimating ritual that leaves core decisions centralized.

The best current evidence is uneven. There is solid evidence that community-led data creation and curation can materially improve model usefulness for underrepresented languages and accents, as shown by Mozilla Common Voice, Masakhane, and newer low-resource speech results such as Pashto Common Voice. There is also solid evidence that external human feedback and red teaming change model behavior, as shown by InstructGPT and GPT-4o’s multi-phase external red teaming. There is growing evidence that AI systems used to summarize public participation can systematically exclude dissenting or rhetorically marginal voices unless they are audited for representational fidelity. But there is still limited causal evidence showing which participatory designs work best for frontier-model governance at scale.

The central analytical conclusion is this: in LLMs, participation is best understood as a governance layer for distributing epistemic authority, not merely as a source of user feedback. The key question is not whether stakeholders were “consulted,” but whether they could alter objectives, dataset composition, evaluation criteria, deployment thresholds, or remedies. Participation that cannot change those levers remains advisory at best and tokenistic at worst.

Your uploaded notes emphasize a closely related intuition: opting out can become a form of invisibility in both civic and algorithmic systems. That intuition is directionally consistent with the research, but the evidence supports a more precise claim: nonparticipation matters most where institutions convert presence into durable artifacts such as datasets, benchmarks, rules, model cards, and governance decisions.

Conceptual foundations

“Power” in participation research is not reducible to formal authority. Arnstein’s foundational article frames citizen participation as a ladder structured by the extent of citizens’ power in determining plans and programs, which is why mere consultation is not equivalent to shared control. Gaventa extends this by arguing that participation must be analyzed across forms of power that are visible, hidden, and invisible, and across spaces that are closed, invited, or claimed. Fung’s framework similarly separates participation into three design questions: who participates, how they communicate and decide, and how those discussions connect to policy or action. Together, these traditions imply that evaluation of participation in AI must ask not only “who was consulted,” but also “what decisions were open,” “who set the agenda,” and “what implementation power followed.”

Empowered participatory governance adds a further requirement: participation should be institutionally organized so that it addresses real problems, involves ordinary affected people, and links deliberation to implementation. Fung and Wright developed this argument as a response to the limits of representative democracy plus technocratic administration for complex problems. OECD work on deliberative institutions operationalizes this idea by identifying multiple models of deliberative participation and emphasizing that high-quality processes require clear mandates, independence, privacy protections, and explicit pathways for authorities to respond to recommendations.

These frameworks map directly onto LLMs. In LLM development, power exists in at least five distinct loci: problem definition, corpus inclusion and exclusion, preference aggregation during post-training, benchmark and audit design, and deployment/governance choices. Formal governance processes may leave all five centralized even when companies or labs solicit comments from users or civil society. Gaventa’s distinction between visible, hidden, and invisible power is especially useful here. Visible power includes published model cards and public comment opportunities. Hidden power includes private decisions about what data sources, languages, or risk categories are even eligible for review. Invisible power includes normalization of technical expertise as the only legitimate voice, which can make lived experience seem anecdotal or “non-rigorous” even when it identifies harms before formal metrics do.

UNESCO’s AI ethics framework and the OECD AI Principles push the same logic into contemporary AI governance. UNESCO treats diversity, inclusiveness, human oversight, and multi-stakeholder adaptive governance as core principles, and explicitly ties ethical impact assessment to collaboration with affected communities. The OECD AI Principles, updated in 2024, likewise center human rights, democratic values, and practical governance recommendations. These sources do not solve the institutional design question on their own, but they establish that participation is not an optional add-on in responsible AI; it is part of the normative baseline.

Participation across the LLM lifecycle

A rigorous way to think about participation in LLMs is to place it along the lifecycle rather than treat it as a single event.

flowchart LR
    A[Problem framing] --> B[Stakeholder mapping]
    B --> C[Data sourcing and curation]
    C --> D[Pretraining filters and documentation]
    D --> E[Post-training feedback and preference aggregation]
    E --> F[External red teaming and participatory audits]
    F --> G[Deployment with monitoring and appeals]
    G --> H[Governance revision and benefit-sharing]
    H --> A

This lifecycle view is consistent with participatory problem formulation research, dataset documentation work, model/documentation governance, human-feedback alignment methods, and continuous evaluation frameworks. It also aligns with NIST’s risk-management approach, which treats trustworthy AI as a full-lifecycle practice rather than a point-in-time certification exercise.

At the front end, participation can enter through problem formulation. Martin and colleagues argue that fairness failures often originate before model training, when teams decide what prediction target matters, how success is defined, and which causal dynamics are ignored; they propose community-based system dynamics to include excluded stakeholders in that phase. Zhang and colleagues’ “Deliberating with AI” case study likewise shows that when decision subjects and decision makers jointly explore model behavior, models can function as boundary objects for deliberation rather than as opaque authorities.

In data work, participation matters in sourcing, selection, validation, and documentation. “Data Statements for NLP” and “Datasheets for Datasets” were both created to make explicit whose language, contexts, and collection processes are represented, and for what intended uses. These documentation regimes do not themselves create participation, but they create the audit surface through which participatory curation becomes possible. In practice, community-led data creation efforts such as Common Voice go further by inviting contributors not simply to donate data, but to create, validate, and curate it.

In post-training, participation most often appears through human-feedback pipelines. InstructGPT used prompts written by labelers and prompts submitted through the OpenAI API, then collected ranked human preferences to improve instruction following, truthfulness, and reductions in toxic output. This is one of the clearest demonstrations that human participation can materially change model behavior. But it is also limited: the relevant question becomes whose preferences are collected, how heterogeneity is aggregated, and whether one reward model can legitimately stand in for plural publics. Recent work on RLHF from heterogeneous feedback shows this is not a trivial issue; preference diversity and strategic reporting both matter.

In safety evaluation, participation enters through external red teaming, adversarial testing, and community evaluation. OpenAI’s GPT-4o system card documents a four-phase external red teaming process involving more than 100 red teamers, 45 languages, and 29 countries, with findings feeding directly into quantitative evaluations and, in some cases, targeted synthetic-data generation. That is a meaningful form of participation because the process altered the evaluation set and mitigation design, not just the messaging around them. Community-managed leaderboards and open evaluation infrastructure on Hugging Face provide a related but distinct mechanism: they distribute some evaluative authority beyond model vendors and make results more reproducible and contestable.

In deployment and governance, participation requires monitoring, contestation, and revision mechanisms. UNESCO’s Ethical Impact Assessment is notable here because it is described as a structured process undertaken with affected communities. Consent-focused and stewardship-focused alternatives are expanding this space: collective consent assemblies propose mini-publics for settings where notice-and-consent is too individualistic to govern interlinked data harms, while data trusts and Indigenous CARE principles propose institutional stewardship, authority to control, and collective benefit as the relevant unit of governance.

Participation mechanisms compared

Power asymmetries and recurring failure modes

The most persistent asymmetry in participatory AI is that organizations often invite participation only after the most consequential decisions have already been made. Sloane and coauthors argue that participation is not a “design fix” for machine learning and warn against “participation-washing,” where communities are asked for input but have little power over infrastructure, resources, timelines, or deployment. Cooper and Zafiroglu’s work on participatory ML adds a practical mechanism for this drift: “participation brokers” often have to translate messy context into the structured inputs machine-learning pipelines can consume, and the organization’s need for scalable, standardized artifacts can subordinate participants’ actual concerns.

This creates a familiar pattern across LLMs. Expertise is treated as technical authority, while lived experience is treated as anecdotal or post hoc. Yet many of the most important LLM harms are precisely the kinds of harms that affected users identify first: accent-related performance gaps, culturally inappropriate refusals, dialect erasure, underrepresentation in training corpora, or summary systems that smooth over dissent. GPT-4o’s system card, for example, explicitly treats accent-linked performance disparity as a risk and mitigates it through diverse voice post-training and subgroup testing. That is evidence that stakeholder-relevant variation can be turned into technical measurement, but only when it is recognized as a governance priority.

Another major asymmetry is aggregation. RLHF and similar methods usually collapse many judgments into one reward model. That can be highly effective for improving instruction-following, but it creates a political problem disguised as a technical one: how should heterogeneous preferences be combined, and when is one “aligned” assistant masking value pluralism? Recent work on heterogeneous-feedback RLHF shows that personalization and aggregation systems may be needed to avoid forcing incompatible publics into a single objective.

Gatekeeping also happens through data filtering and benchmark design. A 2026 AAAI benchmark study found that common harm-reduction filtering strategies can have the side effect of increasing the underrepresentation of vulnerable groups in pretraining datasets. Relatedly, new evidence shows LLM undesirable behaviors occur disproportionately more for users with lower English proficiency, lower education, and origins outside the United States. These findings matter because they demonstrate that “safety” and “fairness” interventions are not neutral: if participation is absent from the definition of harmfulness and representativeness, mitigations can improve one dimension while worsening another.

A final asymmetry is infrastructural. Open evaluation and open-source claims can redistribute some power, but often only partially. The OSI’s Open Source AI Definition requires not just parameters, but sufficiently detailed data information and the complete source code for training and running the system. By that standard, many open-weight models are not meaningfully open enough for participatory scrutiny, reproduction, or contestation. Fully open projects like OLMo move further toward procedural transparency, but even they do not automatically solve issues of community representation, consent, or benefit-sharing. Transparency is necessary for participation, but it is not sufficient.

Case studies and empirical evidence

BigScience and BLOOM

BigScience is one of the most important participation experiments in LLM history because it treated LLM development as an open, collaborative workshop rather than a closed industrial process. Hugging Face describes it as an open and collaborative workshop gathering more than 1,000 researchers. The BigScience case-study paper describes a value-driven, year-and-a-half effort that produced the 1.6 TB ROOTS corpus and BLOOM, while also generating work on ethics, law, data governance, and modeling. The ROOTS corpus paper adds that the dataset spans 59 languages and was assembled explicitly with ethics, harm, and governance in the foreground.

What BigScience demonstrates is that participation can change the social construction of a model, not only its public relations. Its successes were interdisciplinarity, documentation, shared artifacts, and governance reflection. Its own retrospective, however, also notes challenges around participant diversity and the difficulty of coordinating large-scale participatory research. That is analytically important: even highly visible open-science projects remain vulnerable to asymmetries in who has time, compute access, institutional support, and agenda-setting influence.

Mozilla Common Voice and community-led speech data

Mozilla Common Voice is among the clearest operational examples of participatory data curation for language technology. Mozilla describes it as a free, open-source platform for community-led data creation where people can share, create, and curate text and speech datasets; the platform reports participation across 290 languages and frames itself explicitly as a response to AI systems that only work for a few languages. The original Common Voice paper described crowdsourcing for both data collection and validation as the strategy for scale and sustainability.

The empirical value of this model is visible in low-resource outcomes. A 2026 Pashto Common Voice paper reports growth from 1.5 hours and 5 contributors to 147 total hours and 1,483 unique speakers across releases, and shows Whisper Base fine-tuned on the corpus reaching 13.4% WER on the test split compared with a published zero-shot WER of 99.0% for Pashto. That is not merely inclusive process for its own sake; it is a concrete performance gain produced by community participation in data creation and validation.

Masakhane and African-language NLP

Masakhane was founded to address the severe underrepresentation of African languages in NLP by building a distributed, open-source research community across the continent. Its original paper frames the problem as not only a lack of data, but also a lack of community, funding, discoverability, and benchmarks. This is a useful reminder that participation is not only about collecting more examples from marginalized communities; it is also about shifting who gets to do the research, author the benchmarks, and define the agenda.

Human feedback and external red teaming in frontier models

InstructGPT remains a foundational example of participatory post-training. OpenAI collected labeler-written prompts and user-submitted prompts, then ranked outputs using human feedback; the resulting models were preferred by human evaluators over a much larger base model and showed improvements in truthfulness and reductions in toxic output. This is one of the best demonstrations that structured participation can directly improve LLM utility and safety.

OpenAI’s GPT-4o system card shows a later-stage extension of the same logic through external red teaming. More than 100 external red teamers speaking 45 languages from 29 countries participated in four phases, and the resulting data informed both quantitative evaluations and some targeted synthetic-data generation. The same system card also documents explicit mitigation for accent-related disparity through diverse voice post-training and subgroup testing. This case is still centrally governed by the model developer, so it is not participatory governance in a democratic sense; but it is strong evidence that broader evaluative participation changes what gets measured and mitigated.

Civic-tech infrastructure and AI-mediated participation

Civic-tech infrastructure matters because meaningfully participatory AI governance requires channels that can scale beyond ad hoc workshops. Decidim presents itself as a digital platform for citizen participation, built as free/libre software, designed for participatory processes, assemblies, consultations, and participatory budgeting, and used by hundreds of organizations. Its Barcelona case references nearly 7,000 citizen proposals in a strategic city-planning process. This is not an LLM project, but it is directly relevant as reusable institutional infrastructure for participatory AI governance.

A more directly LLM-relevant case is participatory provenance for AI-mediated public consultation. In a 2026 study of Canada’s national AI Strategy consultation, the representational-audit framework found both official summaries underperformed a random-participant baseline, degrading coverage by 9.1% and 8.0%, with 16.9% and 15.3% of participants effectively excluded; exclusion concentrated among dissenting and AI-skeptical voices. This is an important empirical warning for the future of LLM-assisted governance: if models summarize public input without participatory auditing, they can silently recode participation into exclusion.

Fully open models and scrutiny infrastructure

Ai2’s OLMo family is notable because it pushes openness past weights into training data, code, recipes, intermediate checkpoints, and evaluation suites. Ai2 describes OLMo 2 as developed start-to-finish with open and accessible training data, open-source training code, reproducible training recipes, transparent evaluations, and intermediate checkpoints. That does not create participation by itself, but it materially lowers the barrier to third-party scrutiny, reproduction, and community benchmarking. In participatory-governance terms, it expands the “invited space” for downstream actors to inspect and contest the system.

Measuring impact and scaling meaningful participation

The hardest practical problem is not whether participation is desirable, but how to tell when it changed the model or governance outcome in a meaningful way. A useful evaluation strategy separates participation into four measurable dimensions: representational impact, behavioral impact, governance impact, and distributive impact. That structure is partly synthesized here, but it follows directly from model-documentation work, fairness research, public-participation evaluation, and LLM audit research.

Suggested metrics and methods

The current evidence suggests some metrics are especially high-value for LLMs. First, subgroup performance disparity should be measured not only across conventional demographics but also across accents, English proficiency, education level, country of origin, and rhetorical style, because these are all dimensions where LLM harms have now been observed. Second, “representational fidelity” metrics are increasingly necessary when LLMs summarize or rank public input; traditional grounding and hallucination checks are not enough. Third, governance metrics must be tied to actual response and implementation behavior. A process where affected groups submit feedback that never alters thresholds, benchmark suites, or deployment decisions should score low on participatory impact even if attendance and diversity statistics look good.

Scalability requires tooling, but the tooling must support contestation rather than replace it. The most promising stack visible today combines open documentation, open evaluation, reusable civic-participation platforms, and collective stewardship mechanisms. Examples include data statements and datasheets, model cards, Hugging Face’s community-managed evaluation infrastructure, Decidim for structured democratic input, OLMo-style artifact transparency for reproducibility, and institutional mechanisms such as data trusts or consent assemblies where rights and benefits are collective rather than purely individual.

Constraints, recommendations, and open questions

Meaningful participation in LLMs faces real legal, ethical, and practical constraints. Privacy and data protection obligations run across the entire lifecycle; UNESCO explicitly ties AI governance to privacy and data protection, human oversight, and diverse stakeholder participation. NIST’s AI RMF was itself built through an open, collaborative process, which makes it a useful procedural model but also a reminder that participation is time- and capacity-intensive. The U.S. Copyright Office’s multipart AI initiative underscores that training-data governance remains legally unsettled, especially for copyrighted materials, while the EU’s GPAI Code of Practice translates AI Act obligations on transparency, safety, and copyright into a multi-stakeholder compliance instrument. These frameworks create both opportunities and limits for participatory governance.

The practical constraints are equally significant. Deep participation costs money, time, facilitation skill, translation, documentation, and compensation. It may slow iteration. It can also be captured by organized interests, over-index on highly engaged participants, or force overly simplified proxies for difficult value conflicts. None of these problems are arguments against participation. They are arguments for better institutional design, especially around recruitment, compensation, traceability, and accountability for acting on recommendations.

Actionable recommendations

These recommendations are supported by the convergence of participation theory and current AI governance practice: effectual participation requires clear linkage to decisions, independent facilitation, documentation, and monitoring of implementation.

Open research questions

The research base is improving, but several questions remain open.

First, what is the best way to aggregate heterogeneous human preferences in post-training without imposing a false consensus? Current RLHF pipelines are effective, but they remain politically thin abstractions over plural publics.

Second, which participatory structures scale best for frontier-model governance: standing panels, rotating mini-publics, data trusts, unions/cooperatives, community eval systems, or hybrid models? There is promising theory and early experimentation, but not much comparative evidence.

Third, how should LLM-assisted public consultation be audited for representational fidelity, especially when input is large-scale, multilingual, or highly unequal in literacy and rhetorical style? The new provenance work is important, but it is still early.

Fourth, what compensation and benefit-sharing schemes are fair when community participation materially improves high-value proprietary models? Current practice remains far behind the normative literature on collective benefit and authority to control.

Fifth, where should law draw the line between individual consent, collective contestability, and public-interest governance for training data? That question is no longer theoretical for LLMs; it sits directly at the intersection of copyright, data protection, and democratic legitimacy.

The bottom line is that participation has real power in LLMs, but only when it is designed as shared governance over objectives, data, evaluation, and remedy. Without that, participation improves optics more reliably than outcomes. With it, participation can make LLMs more representative, safer, more contestable, and more politically legitimate.