Analytical Review of LLMWikis Guidance and Wizard Implementation on AIWikis

Full report

On this page

Executive summary

AIWikis is not a random accumulation of AI-generated pages. Its live structure shows a deliberate implementation of LLMWikis patterns: handbook-style orientation pages, governance pages, source memory guides, archive-transfer and intake ledgers, and per-file evidence pages that preserve provenance while separating AIWikis from canonical source authority. AIWikis repeatedly states that LLMWikis remains the handbook source, while AIWikis acts as a dogfood and long-term memory layer; that claim is consistent across the home page, the system-memory archive, the source map, and the LLMWikis source memory guide.

The strongest parts of the implementation are the authority boundary model, the archive-and-provenance mindset, and the exposure of compact, queryable evidence objects such as source memory guides, decisions logs, and intake ledgers. Those parts line up well with the better parts of current LLMWikis guidance: explicit source policy, required trust signals, metadata fields, agent stop conditions, and a browser-only wizard that avoids pretending to be an automatic publisher.

The weakest parts are operational rather than conceptual. Many AIWikis public pages are too thin to stand alone, despite being public routes. Visible trust and freshness metadata are often absent on human-facing pages even though LLMWikis requires them in metadata. Several stale hashed artifact URLs still surface and now return 404. Some pages openly admit missing governance infrastructure, such as the source map’s statement that source manifest records are “not available yet.” The site also carries duplication and audience-blur: large provenance indexes, repeated boilerplate, recovered-summary stubs, and public summaries of internal prompts and local source paths.

The root cause is that current LLMWikis guidance is stronger on high-level governance than on enforced execution. The wizard is explicitly a planning tool, not an importer or repository writer, and it generates copyable packets rather than validated deployments. That makes sense for safety, but it leaves a gap: no mandatory public-page completeness threshold, no required visible trust card, no route-regression testing gate, no freshness TTL for generated explanations, no mandatory moderation/redaction scan for artifact pages, and no mode-specific simplification for teams that do not need the wizard’s full combinatorial surface.

The best path forward is not to discard the LLMWikis model. It is to tighten it. The guidance should keep its source-governed philosophy, but add a narrower, mode-specific wizard; mandatory validation and evaluation steps; rendered, visible metadata; route and redirect QA; stronger private-versus-public rendering rules; and a memory schema that explicitly manages authority, duplication, freshness, visibility, and moderation. Those changes would bring the system much closer to established documentation, governance, memory, and safety practice.

Research scope and method

This review focused on the current public surfaces of LLMWikis.org and AIWikis.org as of May 15, 2026, with emphasis on the LLMWikis handbook pages, the setup wizard, and the AIWikis routes that explicitly implement or preserve those patterns. The most important LLMWikis reference points were the setup wizard, metadata standard, source policy, trust model, agent rules, governance page, build guide, and AI-agent guide.

On AIWikis, the sampled set included direct human-facing pages such as Start Here, What Is an LLM Wiki, How To Build an LLM Wiki, Agent Rules, Governance, Source Map, System Memory Archive, Intake Outcome Ledger, Lessons Learned, Recommendation Adjustments, and several “recovered-source summary” pages and source-file evidence routes. I also examined LLMWikis-specific preservation pages on AIWikis, such as the LLMWikis source memory guide, the LLMWikis UAI system index, the LLMWikis public surface file, and the LLMWikis decisions log.

The most important limitation is visibility. AIWikis often does not surface creation date, reviewer, or author on direct public pages, even though LLMWikis metadata guidance requires owner, last reviewed, review cycle, status, source status, and related fields. Because of that, dates and authors could only be recorded when a page, prompt page, search result, or source-file artifact explicitly exposed them.

A second limitation is that I did not verify private repositories or backend internals. Where AIWikis mentions machine routes or internal folders, this report treats those claims as public-site statements rather than directly verified backend capabilities. That matters because AIWikis explicitly says some important infrastructure is still incomplete, most notably the missing source-manifest records.

Catalog of AIWikis pages and memory artifacts created from LLMWikis guidance

The live AIWikis implementation falls into five clearly recurring artifact classes: direct handbook-like pages, recovered-source summaries, source memory guides, logs and ledgers, and individual file-evidence pages under /uai-system/files/ or /files/.... That layering is consistent with AIWikis’ own description of preserving raw source inputs, compiling reviewed wiki nodes, and keeping public pages, compact handoff packets, and deep source-side memory separate.

The score below uses a simple rubric: 5 means strong and close to publishable best practice, 4 good with minor issues, 3 useful but materially flawed, 2 weak or stub-like, and 1 broken.

Two catalog-level findings stand out. First, AIWikis very clearly did follow LLMWikis concepts: scope, source authority, archive separation, trust framing, agent rules, and long-term memory layers are everywhere. Second, AIWikis implemented those ideas with uneven public rendering quality: the most durable internal ideas often live in the strongest memory artifacts, while the most visible public pages are often the thinnest.

Failure modes and root causes

The most defensible reading of the live site is not “AIWikis is conceptually wrong.” It is “AIWikis expresses the right model through a weak publication pipeline.” The recurring failure patterns below are observed repeatedly across current public routes.

The most important strategic conclusion from those patterns is that AIWikis’ low-quality outcomes are primarily pipeline failures. They do not show that the source-governed model is wrong. They show that the current wizard and guidance leave too much to manual judgment after planning, and too little to validation, moderation, route QA, and freshness management.

Comparison with external best practices

LLMWikis already contains several good instincts that align with modern practice. The problem is not absence of principle. It is that the guidance stops one layer too early.

The comparison is broadly favorable to LLMWikis on philosophy and less favorable on execution discipline. In other words: LLMWikis already has the right bones for a serious governed documentation system, but it needs more of the things that mature production AI systems treat as non-optional—tests, checks, moderation, and clearly rendered trust signals.

Revised guidance and wizard design

The revised design should preserve the source-governed model but shift the wizard from planning-only to planning plus validation plus safe execution handoff. The goal is not auto-publication. The goal is eliminating the gap between a good packet and a weak public result. That approach fits both LLMWikis’ own stop-condition mindset and external guidance that prompt engineering should be paired with evaluation, moderation, and human oversight.

flowchart TD
    A[Choose intent] --> B[Select documentation type]
    B --> C[Declare authority owner and audience]
    C --> D[Register sources and privacy class]
    D --> E[Inventory existing routes and canonical winners]
    E --> F[Generate draft from mode-specific template]
    F --> G[Run automated validation and moderation]
    G -->|Fail| H[Return targeted fixes]
    H --> F
    G -->|Pass| I[Human review and approval]
    I -->|Rejected| H
    I -->|Approved| J[Publish page, redirects, and manifest]
    J --> K[Post-publish link, sitemap, and freshness checks]
    K -->|Regression| L[Rollback to previous version]
    K -->|Clean| M[Update indexes, memory ledger, and TTLs]

Revised prompt templates

The current wizard already emits setup packets and AI-repair prompts. The improvement needed is narrower, versioned, mode-specific prompts with explicit inputs, output schema, and verification contract. That aligns with vendor guidance on reusable prompts and Anthropic’s emphasis on verifiable evaluation outcomes.

Template for generating a public page

SYSTEM
You are producing one public wiki page for a governed LLM Wiki.

You must:
- write for one declared documentation type only: tutorial, how-to, reference, explanation, ledger, or evidence
- use only the supplied evidence and handbook sources
- mark unknowns explicitly
- surface visible trust metadata
- produce citations/verification hooks for important claims
- never expose local file paths, prompt text, secrets, or internal-only notes unless visibility=public and allowlist=true

INPUTS
- page_type:
- canonical_route:
- audience:
- authority_owner:
- source_set:
- prior_route_inventory:
- duplicate_group_id:
- freshness_requirements:
- visibility_policy:
- evidence_extracts:

OUTPUT
- status_card
- answer_first_summary
- body
- source_block
- related_pages
- changelog_note
- machine_metadata_json

FAIL CONDITIONS
- missing owner
- missing last_reviewed
- missing source block for derived claims
- body below minimum length for page_type
- unresolved duplicate without canonical winner

Template for importing an artifact into long-term memory

SYSTEM
You are classifying one artifact for long-term memory.

Goal:
Decide whether the artifact becomes:
- public page
- public evidence page
- maintainer-only note
- private archive only
- rejected / blocked

You must classify:
- authority_class
- sensitivity
- visibility
- freshness_ttl
- source_of_truth
- duplicate_group
- moderation outcome
- redaction needs
- publishability

Never assume public visibility.
If prompt text, local paths, private data, or unresolved claims appear, default to non-public handling.
Return one structured decision packet plus a short rationale.

Template for repairing an existing wiki

SYSTEM
You are repairing a governed wiki under human approval.

You must:
- inventory every route, redirect, sitemap entry, llms entry, and internal link
- group duplicates by concept
- choose one canonical winner per concept
- detect stale hash routes
- preserve useful evidence under the right visibility class
- generate a redirect / noindex / retire plan
- output rollback-safe changes only

REQUIRED OUTPUTS
- route_inventory.csv
- duplicate_groups.json
- redirect_plan.csv
- broken_route_report.md
- pages_needing_visible_status_card.md
- publication_risk_report.md

Template for verification before publication

SYSTEM
You are the publication verifier.

For the proposed page:
- list every material claim
- map each claim to source evidence
- identify unsupported, stale, contradictory, or derivative claims
- run a moderation/redaction pass
- confirm visible metadata
- decide: pass, pass-with-warnings, or fail

Use Chain-of-Verification:
1. draft claim list
2. generate verification questions
3. answer each question from evidence independently
4. return final verdict

Validation checks to add

The present guidance already names lint boundaries. The revision is to make those checks concrete and binding for public publication. This direction is supported by LLMWikis metadata requirements, NIST provenance expectations, OWASP output-handling concerns, and Anthropic/OpenAI evaluation guidance.

Memory schema changes

AIWikis and LLMWikis already think in terms of layers, but the schema should explicitly carry authority, visibility, freshness, duplication, and moderation as first-class memory concepts. That is the missing operational layer between good ideas and reliable publication.

{
  "artifact_id": "uuid",
  "title": "string",
  "route_type": "public-page|source-memory-guide|evidence-page|prompt-summary|log|archive-only",
  "authority_owner": "llmwikis.org|aiwikis.org|uaix.org|other",
  "authority_class": "canonical|handbook|dogfood|derived|evidence|draft",
  "visibility": "public|maintainer|private",
  "sensitivity": "public-safe|internal|restricted",
  "freshness": {
    "source_last_seen_utc": "2026-05-15T00:23:56Z",
    "content_last_reviewed_utc": "2026-05-15T00:23:56Z",
    "review_cycle_days": 30,
    "ttl_days": 90,
    "expiry_action": "warn|hide|re-review|retire"
  },
  "duplication": {
    "duplicate_group_id": "string",
    "canonical_winner": "route-or-null",
    "search_indexable": true
  },
  "evidence": {
    "source_urls": [],
    "source_hashes": [],
    "verification_required": true,
    "verification_status": "pass|warn|fail"
  },
  "moderation": {
    "input_scan": "pass|warn|fail",
    "output_scan": "pass|warn|fail",
    "redactions_applied": [],
    "pii_risk": "low|medium|high"
  },
  "rendering": {
    "show_status_card": true,
    "show_internal_paths": false,
    "show_prompt_text": false
  }
}

Moderation rules for public wiki operation

The live site already has good human-approval language, but public wiki governance needs content moderation rules by artifact class, not just generic stop conditions. This is especially important for prompt pages, evidence pages, and archive-derived summaries.

UX changes that matter most

The UX problem is not mainly visual polish. It is decision support. The wizard should narrow choices early, and published pages should show readers what kind of thing they are looking at. That matches Diátaxis and also reduces AI retrieval ambiguity.

The most important UX changes are:

• A first-step mode chooser that narrows the rest of the wizard to one of five flows: build public handbook, repair existing wiki, import evidence, maintainer memory only, or multisite workspace.
• A visible status card at the top of every public page.
• A compact source block that links back to the canonical authority for derivative claims.
• A rule that public pages may show audit IDs, but not raw local path strings.
• A unified route registry page for humans and machines, replacing the current proliferation of overlapping indexes.
• A dedicated retired routes page and automatic 301/410 management for old hash routes.

Mapping old guidance to proposed replacements

Migration plan and monitoring

A sensible migration is short enough to keep momentum and conservative enough to avoid damaging existing canonical routes. The safest cadence is a twelve-week staged migration with shadow validation first, route repair second, and template consolidation third. That pacing fits NIST’s emphasis on provenance, testing, and documentation, and Anthropic/OpenAI’s emphasis on graded evaluation and human oversight.

gantt
    title AIWikis migration timeline
    dateFormat  YYYY-MM-DD
    axisFormat  %b %d

    section Preparation
    Freeze new public template types           :a1, 2026-05-18, 7d
    Build route inventory and duplicate groups :a2, after a1, 7d

    section Validation
    Add visible trust card renderer            :b1, 2026-06-01, 7d
    Add link, redirect, and canonical tests    :b2, after b1, 7d
    Add moderation and redaction checks        :b3, after b2, 7d

    section Content migration
    Upgrade thin core pages                    :c1, 2026-06-22, 14d
    Repair stale hash routes and deindex rules :c2, after c1, 7d
    Consolidate overlapping indexes            :c3, after c2, 7d

    section Launch
    Shadow-mode publish verification           :d1, 2026-07-20, 7d
    Human review and limited rollout           :d2, after d1, 7d
    Full rollout with monitoring               :d3, after d2, 7d

Milestones, risks, and rollback criteria

Metrics to monitor after deployment

Open questions and limitations

Two questions remain open. First, the public site mentions machine routes and manifests, but this review did not directly verify those endpoints. Second, some author and creation metadata may exist in internal files or markup that was not rendered on the public page surfaces sampled here. Those gaps do not change the major conclusion: AIWikis already implements the LLMWikis model, but needs a tighter execution layer to make the public result consistently strong.

Sample improved outputs

The examples below are not intended as full final pages. They are target-state excerpts that show how public pages should look after the revised guidance is applied. Each “before” description is based on the current live page; each “after” sample uses the proposed trust-card, completeness, and source-block model.

How To Build an LLM Wiki

Before

The current page is a short seven-step list with a single “practical rule.” It is directionally correct, but it behaves more like an index note than a finished how-to guide. It does not visibly show owner, review date, source status, or links to the underlying handbook sections it summarizes.

After

# How To Build an LLM Wiki

Status: Reviewed
Owner: AIWikis maintainers
Source status: Derived from LLMWikis handbook
Last reviewed: 2026-05-15
Canonical handbook source: LLMWikis Build Guide

Build an LLM Wiki in this order: define scope, inventory sources, assign ownership, choose canonical routes, publish core governance pages, and only then connect retrieval.

## Before you start
You need:
- a declared source-of-truth owner
- one canonical route per concept
- a review cadence
- a privacy and moderation policy

## Build sequence
### Define scope
Write what the wiki covers, what it does not cover, and which external sources remain authoritative.

### Register sources
Create a source registry with source owner, source type, review cadence, and visibility.

### Publish governance first
Create governance, trust model, source policy, agent rules, and update rules before relying on the wiki in production.

### Add only source-backed pages
Start with glossary, system overview, decision log, runbooks, and open questions. Do not publish unsupported summaries.

## Verification checklist
- Does every page show owner and last reviewed?
- Does every derived page link back to the canonical source?
- Does every public route pass link and moderation checks?

## Sources
- LLMWikis Build Guide
- LLMWikis Metadata Standard
- LLMWikis Source Policy

This version is still concise, but it is now a proper how-to page rather than a thin paraphrase. It separates action from governance and gives the reader a visible verification path. That is more consistent with Diátaxis and with LLMWikis’ own metadata and source-policy rules.

Source Map

Before

The current Source Map contains good boundary text, but it is minimal and openly says that source manifest records are not available yet. For a site that positions itself as provenance-first, that is too weak for a core governance page.

After

# Source Map

Status: Reviewed
Owner: AIWikis maintainers
Last reviewed: 2026-05-15

This page tells you which site owns which claims and where AIWikis stores approved evidence.

## Authority classes
| Source | Role | AIWikis may do | AIWikis may not do |
|---|---|---|---|
| LLMWikis.org | Handbook authority | summarize, cite, preserve evidence | replace handbook claims |
| UAIX.org | Canonical UAI authority | explain, compare, archive reviewed evidence | redefine UAI-1 |
| AIWikis.org | Dogfood and long-memory site | record lessons, ledgers, source-memory guides | claim upstream ownership |

## Registry
- Source registry version: 2026-05-15.1
- Manifest coverage: 100% of public route families
- Last route audit: 2026-05-15

## Route classes
- handbook-derived pages
- source memory guides
- evidence pages
- logs and ledgers
- private archive only

## Current gaps
None blocking publication.

This improves governance by converting a boundary note into an auditable registry surface. It also follows NIST’s emphasis on provenance, contracts, and documented lineage.

AIWikis Starter README

Before

The current page says the starter pack helps create a source-governed LLM Wiki, but it gives very little practical guidance. It is functionally a stub.

After

# AIWikis Starter README

Status: Draft
Owner: AIWikis maintainers
Source status: Recovered local starter material
Last reviewed: 2026-05-15
Noindex: true

Use this starter only as an AIWikis dogfood reference. For canonical starter structure, use the LLMWikis starter bundle.

## What this starter is for
This starter shows how AIWikis adapts LLMWikis patterns for:
- source memory guides
- intake ledgers
- public-safe evidence pages
- long-term dogfood memory

## Folder layout
- /pages/ public pages
- /sources/ approved source extracts
- /ledgers/ intake and archive outcomes
- /registry/ route and source manifests
- /private/ maintainer-only artifacts

## Publication rules
Do not publish:
- prompt text
- raw local paths
- unresolved duplicate routes
- unsupported claim summaries

This version fixes the core failure: it tells the user what the page is, what it is not, and what to do next. It also avoids confusing AIWikis’ dogfood starter with the upstream LLMWikis bundle.

Autonomous Wiki Architect Prompt

Before

The current public page is a recovered summary of a local curator prompt, with local-source-path provenance. That is useful internally, but public prompt summaries should be much more constrained.

After

# Autonomous Wiki Architect Pattern

Status: Reviewed
Owner: AIWikis maintainers
Source status: Public pattern summary
Last reviewed: 2026-05-15

This page describes the operating pattern used by AIWikis for long-term knowledge maintenance.
The underlying maintainer prompt is internal and is not reproduced here.

## Pattern
The custodian agent:
- preserves approved evidence
- updates reviewed pages only through staging
- keeps open questions visible
- records recommendations separately from canonical source claims

## Not public
- full maintainer prompt text
- local prompt paths
- private workflow notes

## Why this matters
The public should understand the operating model without gaining access to internal prompting details that are unnecessary for trust.

This preserves transparency while reducing unnecessary prompt exposure. That aligns with AIWikis’ own public/private boundary language and with OWASP/OpenAI/Anthropic safety norms.

Intake Outcome Ledger

Before

The current page is valuable, but it mixes the public answer with raw path references and dense archive detail. That makes auditability stronger than readability.

After

# Intake Outcome Ledger

Status: Reviewed
Owner: AIWikis maintainers
Last reviewed: 2026-05-15

This page tells you what changed after source files were processed.

## Latest completed intake
Date: 2026-05-14
Source owner: JustAnIota.com
Files processed: 11
Disposition: Reviewed and preserved
Public outputs:
- Universal Semantics intake report
- Updated source memory guide
- Ledger entry with evidence IDs

## Evidence
- Intake batch ID: JI-2026-05-14-USCR
- Source owner confirmation: yes
- Human approval: yes
- Raw archive retained privately: yes

## Reader rule
Use this page first.
Open evidence pages only if you need deeper provenance.

This keeps the proof-of-use model but moves raw archive mechanics behind an evidence ID. That is closer to a public-safe ledger and closer to the principle that raw archives should not be the primary public surface.

What Is an LLM Wiki

Before

The current page provides a clean definition and good conceptual boundary, but it does not visibly show the derived source relationship or freshness metadata.

After

# What Is an LLM Wiki

Status: Reviewed
Owner: AIWikis maintainers
Source status: Derived from LLMWikis handbook
Last reviewed: 2026-05-15
Canonical handbook source: LLMWikis What Is an LLM Wiki?

An LLM Wiki is a governed knowledge system for humans and AI agents.
It keeps raw evidence, reviewed pages, metadata, and retrieval rules distinct.

## What makes it different
- source-backed rather than chat-memory-backed
- reviewed rather than silently rewritten
- citable rather than implicit
- durable rather than session-bound

## What it is not
- not a certification claim
- not a private chat dump
- not a replacement for canonical source authority

This keeps the current strong definition, but adds what the live version most lacks: visible trust context and a canonical handbook pointer.