Make Search Engines and LLMs Accountable for What They Hide

Full report

On this page

Revised research edition — 12 October 2026 | CognitiveLiberties.com / FFTAC contextual companion

THE RIGHT TO HEAR AND BE HEARD — Publish your perspectives. Seek the perspectives of others. Refuse unjustified algorithmic invisibility. Do not shadowban yourself.

Core editorial position — expression is the point

Cognitive liberty is the right to hear and be heard. The purpose of this work is not to persuade people to withdraw from public life, mute their opinions, or disappear from the knowledge commons. It is to make lawful inquiry, human expression, original publication, disagreement, discovery, and participation easier and more durable. Do not shadowban your own thoughts. Do not assume that being unusual makes an idea unworthy of careful publication. A human perspective that is never contributed is less likely to influence shared culture, research, public records, or AI systems.

The case for contribution is an invitation, never a demand for disclosure. Some material cannot responsibly be shared by a particular person; others may need attribution or licensing choices. Those decisions are subordinate to the primary goal here: expanding the number and range of people who choose to contribute, and preserving routes by which their work can be encountered, understood, checked, and challenged. There is no guarantee a public page will be indexed, surfaced, cited, or used to train an AI model.

The institutional companion is equally direct: algorithms should not covertly or unjustifiably suppress lawful, relevant perspectives because they are unpopular or institutionally inconvenient. Not every low-ranking page has been "shadowbanned": crawling failure, language mismatch, relevance, duplicate content, user preferences, spam mitigation, and safety enforcement can also explain invisibility. The standard proposed here is verifiable transparency, independent testing, useful appeals, and pluralistic discovery—not a demand that every assertion receive equal ranking or equal credibility.

The distinction matters: visibility is not endorsement; inclusion is not proof; the right to speak is not an entitlement to someone else's agreement. We must be able to hear an argument before we can scrutinize it. Honest intellectual pluralism presents evidence strength clearly, offers competing interpretations when warranted, and resists both manipulative amplification and arbitrary exclusion.

New research priority: Challenge covert exclusion with evidence and a repair path

'Shadowban' is a useful description of the perceived effect of being unseen, but an inadequate causal diagnosis. A publisher needs a traceable sequence: page reachable? crawled? indexed? excluded by policy? ranking low? filtered by user context? cited by a model? Each stage has different tests and different legitimate explanations. Calling all absence censorship makes real problems harder to prove.

A stronger movement asks for standardized reason categories, observable test endpoints, correction/appeal routes, preservation of before-and-after evidence, and access for independent auditors. The goal is to stop unjustified, opaque viewpoint suppression while preserving legitimate defenses against fraud, malware, unlawful material, and manipulation. Avoid promises of guaranteed reach.

Build a community-led 'Visibility Clinic' that teaches small publishers Search Console basics, canonical URL checks, accessible HTML, factual documentation, appeal submissions, and longitudinal rank sampling. Publish both confirmed problems and cases where the original suspicion was unsupported.

Editorially required deliverables and first experiments

The following are practical priorities for the next release. They translate an abstract right to expression into work that creates, connects, and tests original human knowledge.

1. Action 1: Create a public diagnostic wizard for crawl/index/rank/citation stages.
2. Action 2: Establish standardized case IDs and an evidence packet format for suspected viewpoint demotion.
3. Action 3: Report corrections and cases ruled inconclusive or technical, not only confirmed exclusions.

Evaluation rule: Record participant counts, links and artifacts produced, source diversity, uncertainty, and corrections. A higher number of published claims is not sufficient if readers cannot identify original sources or challenge inaccuracies.

---

Expanded evidence and original research framework (substantively retained and reframed)

Exclusion map, regulatory comparison, auditing methodology, charter and publisher diagnostic specification · CognitiveLiberties.com · 11 October 2026

Executive finding

A missing result has many possible causes. Accountability requires forensic humility: distinguish a URL that was never crawled from one indexed but ranked poorly, a policy removal from a canonicalization error, and a model that omitted a source from one that lacked access to it. The most achievable first step is a reproducible, publisher-facing diagnostic process, paired with transparent appeal channels and independent cross-query measurements. A universal obligation to explain every commercial ranking decision does not currently exist; rights and responsibilities differ among jurisdictions and service classes.

1. The information-discovery chokepoint map

Publisher publishes → robots/access → crawler discovers → canonical/duplicate selection → indexing → retrieval/query expansion → relevance and policy ranking → displayed snippets and links → AI retrieval context → generation/summarization → citation selection → user's next click

At each arrow a viewpoint may become less visible without its text being erased from the open web. This is a diagnostic model, not a claim that filtering is always ideological. A site may be accessible directly yet absent from results because it is new, duplicated, low-quality under a service's criteria, incorrectly canonicalized, blocked by technical settings, or simply irrelevant to a query. Google Search Console URL Inspection and Bing Webmaster URL Inspection expose important parts of the crawl/index pipeline but do not supply a full internal explanation for every ranking [1, 2].

2. What law and existing tools provide

Under the EU Digital Services Act (DSA), obligations are service-specific. Article 17 imposes statement-of-reasons requirements for qualifying hosting restrictions; Article 20 concerns internal complaint handling by online platforms, subject to scope and exemptions. The same duties must not be generalized automatically to every general-purpose search result or language-model response. Very large online platforms and search engines have added systemic-risk and auditing duties, and Article 40 provides a path for vetted researchers to request qualifying data; Delegated Regulation (EU) 2025/2050 details procedures [3, 4, 5]. Competition law, privacy law and national speech doctrine add context, but do not establish a universal 'right to top-ten rank.'

Google's URL Inspection provides a view of indexed status and indexability, with limitations; Bing's inspection provides crawl/index/markup diagnostics, remedial guidance and webmaster support [1, 2]. These tools help publishers debug infrastructure but offer limited visibility into comparative ranking or generative citation decisions. NIST's voluntary AI Risk Management Framework gives a governance vocabulary for mapping, measuring, managing and governing relevant risks [6].

3. Search and AI Accountability Charter (proposal)

Information without concealed viewpoint filters. Search and AI tools should support source-first, maximally open exploration of lawful information, with meaningful visibility into ranking, exclusion and source selection. 'Unfiltered' describes an accountable research option, not a promise that every possible document can or must appear.

No secret viewpoint disappearance. Lawful, genuine and relevant publishers deserve fair indexing/retrieval consideration. Neither secret viewpoint demotion nor suppression disguised as ordinary relevance should be tolerated; significant restrictions require intelligible categories, available diagnostics and proportionate redress.

Not filtered out of AI knowledge. Developers should offer workable pathways by which a diverse range of human authors can contribute appropriately licensed original works to training-ready datasets, search corpora and model evaluations. Report what is covered, what is missing and whether the model can accurately explain underrepresented views. The demand is representational accountability, not a promise to ingest every author into every model.

Traceable originals. AI answers should disclose and link to sources where feasible and distinguish accurate citation from mere paraphrase. Give people a source-first control that can reveal disagreement and primary works.

Independent tests and remedies. Make enough general decision categories available to support external controlled audits, while protecting security-sensitive anti-spam details. Offer publisher-accessible crawl/index diagnostics, formal routes to correct material errors, and aggregate reporting on appeals and reversals.

No fabricated censorship findings. Investigators must separate verified policy restrictions from crawl failure, relevance decisions and undiagnosed omissions. A missing result cannot establish intent without corroboration.

Proportionality. Apply lawful safety and anti-abuse rules transparently and without arbitrary viewpoint discrimination, and do not confuse inclusion with endorsement. The Charter is a proposed standard and requires service- and jurisdiction-specific implementation.

4. Independent auditing protocol

Design: Build a preregistered 240-query benchmark: 60 exact-title navigational, 60 topical scholarly, 60 competing-theory and 60 multilingual translations. For each define ten eligible source URLs from diverse publishers and an independent relevance label. Compare search engines and AI assistants as different product classes, not as though rank 1 and a generated summary were interchangeable. Record outputs at three time points, search locale, logged-in state, query formulation, settings, indexability and source freshness. Annotators should not know which experimental condition generated a result.

Metrics: index coverage; eligible-source recall@10; source diversity by verified publisher category; omission from generated answers; citation fidelity; update lag; technical-diagnostic resolution; appeal turnaround; error-correction persistence. Publish overall and segment-specific measures, confidence intervals and a list of cases with unknown causes. No data have been collected for this proposed audit.

Causal caution: a ranking disparity is an observation, not evidence of discriminatory motive. Stronger causal designs hold page quality, topic, relevance and technical status constant; compare matched publisher pages across queries and languages; and test whether the same effect persists through time. Even then, proprietary internals may prevent attribution to a specific ranking feature.

5. Publisher-facing discoverability diagnostic: product specification

Required inputs

Verified domain; affected canonical URL; target query and region; publication/update date; sitemap entry; current robots and meta directives; structured metadata; captured SERP or AI answer; target source text; and optional Search Console/Bing export. Do not collect reader search histories or unnecessary personal data.

Decision tree and output

1. Delivery: check DNS, TLS, HTTP status, redirects and content accessibility for a normal visitor and, where allowed, a verified crawler.
2. Eligibility: inspect robots, meta robots, canonical references, sitemaps and duplicate content indicators.
3. Index evidence: import first-party webmaster tool result; label unobserved states as 'unknown'.
4. Query relevance: map user intent to source page sections and independent comparison URLs.
5. AI trace: if the URL appears in provided retrieved-source logs, test whether an answer correctly represents and cites it; otherwise mark source availability unverified.
6. Case label: Technical fault / Documented platform notice / Search relevance dispute / Citation error / Insufficient evidence.
7. Action: publish fix instructions, responsible owner, URL of any available platform appeal and a timestamped retest plan.

JSON field sketch: case_id, observed_at, url, query, locale, url_status, robots, index_status, canonical, platform_notice, ai_retrieval_evidence, hypotheses[], known_limitations[], appeal_status, resolution_date. The diagnostic should never output 'censorship proven' based only on absence from a top-ten query.

6. Policy and voluntary implementation choices

Voluntary: publish standardized reason categories, webmaster diagnostic APIs, independent source diversity reporting, and notification/correction paths. Open standard: common schemas for observable exclusion events and publisher appeals, with typed uncertainty and jurisdiction flags. Public policy: evaluate access to independent audit data, carefully scoped transparency obligations and judicially reviewable decisions where state action or regulated platform duties apply. Design decisions should guard trade secrets, security, privacy, due process and legitimate editorial independence.

7. Pilot and success criteria

Pilot with 20 independent publishers across languages and domains. Seek a reduction in the fraction of unexplained technical non-indexing incidents, measured time-to-resolution, and improved factual citation correction. Do not use increased traffic as the sole goal: relevance and public learning matter. Publish cases where diagnostic findings disprove participants' initial assumptions. This pilot is proposed only; no independent ranking tests or authenticated webmaster accounts were accessed for this report.

Sources

1. Google Search Console, URL Inspection Tool. https://support.google.com/webmasters/answer/9012289?hl=en
2. Bing Webmaster Tools, URL Inspection. https://www.bing.com/webmasters/help/url-inspection-55a30305
3. Regulation (EU) 2022/2065, Digital Services Act, especially Articles 17, 20, 34, 37, 40. https://eur-lex.europa.eu/eli/reg/2022/2065
4. European Commission, How the DSA enhances transparency. https://digital-strategy.ec.europa.eu/en/policies/dsa-brings-transparency
5. Commission Delegated Regulation (EU) 2025/2050, researcher access procedure. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:32025R2050
6. NIST, AI RMF and Generative AI Profile (2023/2024). https://www.nist.gov/itl/ai-risk-management-framework

Shared measures of success

A program of cognitive liberty should measure contribution and discovery, not merely the existence of policy statements. Track first-time contributors, original works published, source links to independently hosted material, multilingual contributions, constructive cross-viewpoint exchanges, accessible original-source referrals, correction turnaround, author-reported ability to contribute, and successful resolution of demonstrable indexing or moderation errors. Publish baseline, denominators, sampling method, and uncertainty. Separate actual observed outcomes from targets, hypotheses, and aspirations.

Pilot practice: establish a baseline for four weeks; publish at least one clearly sourced research contribution each week; solicit counterarguments; log corrections; verify accessibility and discovery paths; recruit writers from different linguistic and intellectual backgrounds; and publish monthly results including failures. This is participation-first infrastructure rather than merely rhetorical advocacy.

Evidence, limitations, and responsible claims

• Established principles: The Universal Declaration of Human Rights, Article 19, recognizes freedom of opinion and expression and the freedom to seek, receive, and impart information. Specific enforceable remedies depend on jurisdiction and context; the proposal here does not claim a universal legal right to a search ranking or AI-training placement.
• AI governance: UNESCO's 2021 Recommendation on the Ethics of AI advocates diversity, inclusion, participation, and freedom of expression. NIST AI RMF provides voluntary governance and measurement practices. Neither guarantees any author’s training inclusion.
• Discoverability: Google's own documentation makes clear that compliant public pages are eligible for crawling/indexing but inclusion is not guaranteed. A missing result alone does not prove viewpoint discrimination.
• Accountability: The EU Digital Services Act has distinct obligations for different regulated services and restriction types. It is not a blanket ban on every form of ranking or a universal user appeal right for any search omission.
• Source integrity: The source collection includes secondary and internally inconsistent assertions about FFTAC’s history, legal status, website reachability, ecclesiastical activity, technical systems, and organization size. Treat those as claims to check against records—not established facts. Legal and technical sections are informational and require qualified, jurisdiction-specific review before implementation.

Official and standards references (accessed for this revision): UN UDHR, Article 19; UNESCO Recommendation, 2021; NIST AI Risk Management Framework; Google crawling/indexing FAQ; EU Digital Services Act; FFTAC public participation. The public-site observations are time-bound; any website or legal-status claim should be rechecked at publication.