Semantic Wiki and Citation Provenance for Tenant-Protected Enterprise Knowledge

Full report

On this page

Research basis and design constraints

A strong design for an enterprise semantic wiki should treat knowledge as both navigable content and verifiable evidence. The research points to six relevant foundations. First, W3C PROV models provenance as relationships among entities, activities, and agents so downstream users can assess quality, reliability, and trustworthiness. Second, Semantic MediaWiki demonstrates that wiki pages can be enriched with machine-readable categories and properties instead of remaining plain text. Third, SKOS provides a standard way to represent concept schemes, hierarchies, labels, and associative relationships, including the ability for one concept to have multiple broader parents. Fourth, modern retrieval quality improves when systems combine text search with vector search and semantic reranking rather than relying on exact string match alone. Fifth, citation intent and mention-level metadata should be modeled separately from the cited work itself. Sixth, authorization must be enforced before any object lookup or retrieval because broken object-level authorization and existence leakage are common API risks.

Those foundations map directly to your requirements. A query such as “prompt budgets” should not depend on literal string overlap; it should traverse concept links like AI infrastructure → tokenization → prompt optimization and still land on the right document. Azure’s hybrid retrieval documentation explicitly describes why vector search helps with conceptual similarity while keyword search preserves precision, and Microsoft’s GraphRAG guidance adds that hierarchical graph structures help answer over private corpora more effectively than flat snippet retrieval alone.

The wiki should therefore be built around canonical knowledge units, taxonomy placements, concept links, and immutable provenance events. A single body of content can appear in many contextual hierarchies without duplication if the system stores one canonical document and many placement edges. SKOS is especially relevant here because it treats vocabularies as concept schemes and explicitly allows a concept to have several broader concepts at the same time. That is the semantic equivalent of “same thing, many paths.”

A separate but equally important conclusion is that URLs must not be modeled as a single mutable row. RFC 3986, RFC 9110, RFC 9530, Memento, WARC, Web Annotation, CiTO, DataCite, and Crossref together imply a better pattern: keep a stable canonical identity for the resource, store each crawl observation and snapshot as its own historical record, and store each citation mention as its own annotation/relationship record. That is the only way to preserve redirects, hashes, text anchors, updates, stale evidence, and citation context without losing earlier states.

Finally, source authority cannot be collapsed into one flat score. An IETF or W3C standard has a different authority scope than a peer-reviewed paper, an arXiv preprint, a vendor product page, or a community anecdote. IETF RFCs distinguish standards-track, best current practice, informational, experimental, and historic statuses; ACM documents describe formal peer review for its peer-reviewed content; arXiv describes itself as a curated research-sharing platform rather than a peer-review venue; and Crossmark exists precisely because research artifacts can later be corrected, updated, or retracted. A production system should therefore score sources by both authority level and authority scope.

Recommended architecture

The most robust architecture is a relational system of record with graph edges and search projections, not a wiki that stores everything as denormalized pages. Keep the canonical truth in normalized tables, project selected fields into a full-text index and a vector index, and enforce authorization before object resolution. This follows the same general logic behind Zanzibar-style authorization—express policies uniformly, evaluate access consistently, and keep the gateway close to the protected resource—and aligns with NIST’s zero-trust guidance that each resource request should be explicitly authorized with least privilege rather than relying on implied trust.

In practice, the architecture has five planes. The content plane stores canonical knowledge units, sections, concepts, and URLs. The provenance plane stores ingestion jobs, evidence records, digests, snapshots, reviewers, and supersession relationships. The authorization plane stores object-to-principal or object-to-group relationships and evaluates policy before any content lookup. The search plane stores keyword, vector, and graph projections of only authorized data. The delivery plane serves the same routes to every tenant, but the caller’s identity and policy context determine what data is visible. Anonymous and cross-tenant requests get identical non-revealing outcomes.

The canonical model should separate four different things that are often incorrectly merged:

1. Knowledge unit: the canonical internal document or atomic wiki entry.
2. Concept: a semantic subject node used for navigation and expansion.
3. Canonical URL: the normalized identity of an external web resource.
4. Citation mention: a particular occurrence of a citation inside a document, section, or report, with its own anchor text, local description, line range, and provenance.

That distinction mirrors established Web models. PROV separates entities, activities, and agents; Web Annotation separates the source resource from the annotation target and selector; CiTO separates citation intent from the cited work; and DataCite/Crossref treat relationships as first-class metadata instead of burying them in prose.

The storage engine can remain mainstream. PostgreSQL is a good fit for normalized records, path tables, and immutable event history; a full-text/vector engine can hold projections for search; and an object store can hold snapshots, extracted text, and WARC-like artifacts. Because WARC is specifically designed to bundle harvested resources together with associated information for retrieval and exchange, it is a useful pattern for external evidence capture even if you do not expose raw WARC files directly.

The diagram below shows the recommended split between authoritative storage, semantic projections, and authorization.

flowchart LR
    A[Report Ingestion] --> B[(Normalized Store)]
    A --> C[(Snapshot Store)]
    B --> D[Search Projection]
    B --> E[Graph Projection]
    F[Policy Engine / AuthZ] --> G[API Gateway]
    D --> G
    E --> G
    B --> G
    G --> H[Human UI]
    G --> I[Agent API]

The reason to keep the same gateway in front of both the human UI and the agent API is parity: the answer surface, provenance surface, and authorization enforcement should be consistent for both, while the response format can differ. That is also the safest way to satisfy the requirement that common routes be reusable across tenants without leaking titles, counts, labels, or existence signals.

Normalized data model and APIs

The core schema should be intentionally normalized. The design below is not the only valid model, but it is the cleanest way to satisfy provenance preservation, multi-path placement, URL reuse history, and tenant isolation.

Core content entities

This content model is aligned with SKOS-style concepts and hierarchies, Semantic MediaWiki’s distinction between categories and properties, and PROV’s emphasis on linking data items back to responsible agents and activities.

URL and citation entities

The key design choice is that canonical_url is stable identity only, while url_observation holds every historical title, description, crawl state, digest, and report attachment. The url_current_state view gives you the “current canonical properties” required for search, but does so without erasing earlier evidence. That directly satisfies the requirement that URL reuse never destroy previous keywords, crawl state, descriptions, or report provenance. RFC 3986 gives the normalization basis; RFC 9110 and Google’s canonical guidance explain redirects and canonical signals; RFC 9530 and NIST hash guidance support digest capture; Memento and WARC support snapshotting; and Crossmark provides a good model for preserving “current status” without mutating history away.

Authority and lifecycle fields

Use two orthogonal dimensions:

• authority_level: anecdotal, informational, reviewed_guidance, approved_guidance, policy_or_standard, normative_external
• authority_scope: organizational, normative_specification, empirical_research, implementation_vendor, secondary_summary, community_discussion

Use lifecycle values such as proposed, draft, in_review, reviewed, approved, adopted, deprecated, superseded, and retired. RFC statuses and Crossmark-style update handling are useful mental models here because they distinguish informational artifacts from best practices, standards, historic items, and post-publication updates.

The source-classification policy should be explicit:

• First-party source: publisher controls the subject matter being described. Good default for company policy, product behavior, or workspace procedure.
• Specification: standards body or formal spec issuer, such as W3C Recommendations or IETF standards/BCPs.
• Research paper: peer-reviewed venue or curated research-sharing platform; store whether it is peer-reviewed, moderated-only, or unknown.
• Vendor claim: vendor-authored product or benchmark claims.
• Secondary summary: analyst blog, documentation summary, or derivative explainer.
• Community anecdote: forum, Q&A, or social discussion.

That classification should be data, not hard-coded logic, because the same host can publish different evidence classes over time. W3C Recommendations, IETF RFC statuses, ACM peer-review policy, arXiv’s research-sharing positioning, and Crossmark’s update model provide a practical basis for those distinctions.

API examples

Shared routes can remain tenant-agnostic because policy selects visible data:

POST /api/ingestion/reports
Authorization: Bearer <token>
Content-Type: application/json

{
  "sourceIdentity": {
    "sourceKind": "internal_report",
    "publisherName": "AI Platform Architecture",
    "publisherRole": "first_party",
    "documentRef": "report://2026/07/prompt-optimization-review",
    "versionLabel": "v1"
  },
  "document": {
    "title": "Prompt Optimization Review",
    "language": "en-US",
    "body": "...",
    "sections": [...]
  }
}

GET /api/search?q=prompt%20budgets&context=architecture&topK=10
Authorization: Bearer <token>

{
  "results": [
    {
      "knowledgeUnitId": "ku_01J...",
      "title": "Prompt Optimization Guidance",
      "lifecycleStatus": "approved",
      "authorityLevel": "approved_guidance",
      "pathHints": [
        "AI infrastructure > tokenization > prompt optimization",
        "LLM operations > cost control > prompt design"
      ],
      "semanticReasons": [
        "concept match: prompt budget ~ token budget",
        "linked concept: tokenization",
        "supported by 4 citation mentions"
      ],
      "citations": [
        { "mentionId": "cm_01J...", "label": "[3]" },
        { "mentionId": "cm_01K...", "label": "[7]" }
      ]
    }
  ]
}

GET /api/citations/cm_01J...
Authorization: Bearer <token>

{
  "mentionId": "cm_01J...",
  "canonicalUrl": {
    "normalizedUrl": "https://example.org/spec/prompt-budgeting",
    "site": "example.org",
    "currentTitle": "Prompt Budgeting Specification"
  },
  "mention": {
    "relationshipType": "supports",
    "anchorText": "prompt budgets",
    "contextualDescription": "Used as the primary external definition of budget partitioning",
    "citationLabel": "[3]",
    "citationOrder": 3,
    "sectionRef": "3.2",
    "lineRange": "118-134"
  },
  "provenance": {
    "createdByActivity": "ingest_report",
    "sourceIdentity": "report://2026/07/prompt-optimization-review",
    "sourceHash": "sha256:..."
  }
}

The search and citation APIs above are intentionally shaped so the agent sees no secret fields that a human cannot also inspect through an authorized UI. That parity rule is important enough to be a product invariant, not just an implementation preference.

Ingestion and citation provenance workflow

The ingestion workflow should be one report at a time, transactionally, because provenance is much easier to reason about when each ingest run has one root activity, one source identity, and one auditable output set. PROV is the right conceptual vocabulary for that: one ingestion activity uses one source entity and generates multiple derived entities, each of which can retain links back to the source, the actor, and the software agent that produced it.

A recommended ingest sequence is:

1. Register source_identity.
2. Store the raw report payload and compute source_content_sha256.
3. Parse title, description, sections, keywords, citations, and structural anchors.
4. Create or update knowledge_unit and section records.
5. Extract concepts, map them to controlled vocabulary nodes, and create knowledge_concept_link edges.
6. Create multiple knowledge_placement rows instead of duplicating body text.
7. For each URL mention, normalize per RFC 3986, resolve redirects carefully, and create or reuse a canonical_url.
8. Store each crawl/seen event as a new url_observation.
9. If crawlable and allowed, store a snapshot and its digest.
10. Create one citation_mention plus one or more citation_selector rows per mention.
11. Rebuild the search projection and graph projection for only the affected objects.
12. Commit an immutable provenance_event graph for the run.

That sequence is strongly supported by the standards corpus. RFC 3986 defines syntax-based normalization such as lowercasing scheme/host, decoding percent-encoded unreserved characters, removing dot segments, and omitting default ports where applicable. RFC 9110 defines permanent redirects via 301 and 308 plus Location, while also warning that Content-Location is an assertion that cannot be trusted unless verified by other means. RFC 9530 defines representation digests; NIST’s Secure Hash Standard provides the cryptographic basis for SHA-family hashes; and Memento plus WARC provide mature models for point-in-time capture.

The citation model should preserve both human-readable and machine-stable anchors. Web Annotation is a strong template here: a TextQuoteSelector can record exact text with prefix and suffix, while a TextPositionSelector can store start and end positions. For PDFs or extracted files, line and page ranges can be stored in parallel. CiTO then provides a way to distinguish relationship types such as supports, extends, usesMethodIn, or disagreesWith. Together, those standards are almost exactly what mention-level provenance requires.

A subtle but important rule: if there are copyright or rights restrictions, avoid copying large quoted segments into provenance records. The Web Annotation model explicitly notes that quote selectors can be risky for copyrighted static texts and suggests position-based selectors where appropriate. For enterprise use, store small anchors plus positional selectors and a content hash, not large copied passages, unless rights policy explicitly allows it.

The workflow below is the recommended ingestion shape.

sequenceDiagram
    participant R as Report
    participant I as Ingestor
    participant N as Normalizer
    participant C as Crawler
    participant S as Store
    participant X as Search Projection

    R->>I: submit one report
    I->>S: create source_identity + provenance_event
    I->>S: store report bytes + SHA-256
    I->>N: extract sections, concepts, citations, URLs
    N->>S: upsert knowledge_unit + placements + concept links
    N->>C: resolve and evaluate URL mentions
    C->>S: append url_observation + snapshot + digests
    N->>S: append citation_mention + selectors
    S->>X: rebuild affected text/vector/graph projections

This ingest model also makes stale evidence and supersession manageable. Do not overwrite old evidence. Instead, create new observations and supersession edges, then let ranking favor the current reviewed item while preserving the historical chain. That is the same broad idea Crossmark uses for corrections and updates in the scholarly record and the IETF uses for obsoletes/updates relationships in RFC metadata.

Retrieval, ranking, and parity

Ranking should be a gated hybrid ranker, not a pure weighted sum. If you let old proposals compete on equal terms with reviewed or approved guidance, they will eventually outrank the right answer because they have more text, more citations, or denser keyword overlap. The solution is to do ranking in stages:

Stage one: apply authorization and lifecycle gates. Stage two: retrieve candidates using keyword, vector, and graph expansion in parallel. Stage three: rerank with authority, freshness, hierarchy context, and citation support. Stage four: apply answer assembly rules and context budgets.

A practical scoring model is:

\[ FinalScore = EligibilityGate \times ( 0.22 \cdot Text

) \]

• 0.20 \cdot Semantic
• 0.14 \cdot Graph
• 0.12 \cdot HierarchyContext
• 0.14 \cdot Authority
• 0.08 \cdot Freshness
• 0.10 \cdot CitationSupport

Where:

• EligibilityGate = 0 for retired or superseded unless the query explicitly requests historical results.
• EligibilityGate = 0.35 for proposed or draft when approved or reviewed guidance exists in the same topic cluster.
• EligibilityGate = 1.0 for reviewed, approved, or adopted.

This gating is a product policy, but it is justified by the external models. Hybrid search documentation shows why full-text and vector retrieval should be fused. GraphRAG supports hierarchical structured retrieval for private corpora. Semantic reranking documentation shows why a second-stage reranker is useful and why limited top-N inputs should be summarized before reranking. RFC and Crossmark status models demonstrate that “currentness” and “supersession” are part of evidence quality, not just metadata decoration.

For conceptual retrieval, the graph score should reward three signals:

• direct concept match
• traversal from nearby broader/narrower/related concepts
• path affinity between the query context and the document placement path

That is how “prompt budgets” can find content whose visible path is “AI infrastructure → tokenization → prompt optimization.” SKOS supports broader, narrower, related, preferred labels, alternative labels, and hidden labels; those are ideal primitives for semantic expansion without duplicating the document body across taxonomies.

Citation support should measure more than raw mention count. The stronger signal is independent, recent, high-authority support. A document supported by one normative spec and one recent first-party approved procedure should generally outrank a verbose proposal citing six internal discussion threads. The citation-support feature should therefore use:

• source diversity
• authority-weighted support
• freshness decay
• contradiction penalty
• unresolved supersession penalty

That keeps the system from rewarding noisy argumentation over actual guidance. Crossmark’s update model and the IETF’s obsoletes/updates model are useful precedents for treating supersession as ranking input.

For context-budgeted crawling and compact agent responses, use a budgeted expansion policy. Azure’s semantic ranker documentation is helpful here because it makes explicit that reranking systems have finite token budgets, prioritize title/keyword/content fields, and trim long inputs before reranking. Your agent pipeline should mirror that philosophy: retrieve broadly, then compress aggressively while preserving citations. A good answer envelope is: brief answer, why it matched, path traversed, and citation mentions.

Human/agent parity should be enforced through four rules. The human UI and agent API must share the same ACL-filtered candidate set, the same canonical records, the same citation-mention objects, and the same lifecycle defaults. Agents may compute semantic expansions, but any cited evidence must resolve to records a human with the same authorization could inspect. That prevents “hidden corporate memory” behavior and keeps the agent explainable by design.

Tenant protection and crawl boundaries

The threat model should assume four adversaries: an anonymous internet caller, an authenticated user from the wrong tenant, an authenticated user inside the right tenant but with insufficient object permissions, and an agent or integration accidentally overfetching data. The major failure modes are enumeration, broken object-level authorization, overbroad search projections, cache leakage, and provenance leakage through metadata. OWASP’s API guidance describes how object identifiers are routinely abused when authorization is not checked per object, and OWASP’s enumeration guidance recommends uniform responses for failed requests to avoid leaking whether an object exists.

The operational rule should be simple: authorize before you resolve. Do not fetch the object and then filter fields afterward. NIST zero trust explicitly frames access as per-resource, per-session, least-privilege authorization through a policy decision point and policy enforcement point, and Zanzibar demonstrates how a uniform authorization model can be used across many services and object types.

For the API surface, use shared routes without tenant identifiers in the path when possible, and return indistinguishable outcomes for “not found” and “not authorized” on object fetches. For search endpoints, avoid leaking counts, facet buckets, taxonomy labels, or snippet previews across tenant boundaries. Even timing differences can become side channels, so sensitive endpoints should normalize response behavior for empty and unauthorized outcomes. That design choice is a direct analogue of OWASP’s generic-error guidance for account enumeration.

Crawling policy should be conservative by default. RFC 9309 says robots rules are not authorization, yet it also says crawlers must follow parseable rules when they successfully fetch robots.txt, may treat 4xx robots responses as “unavailable,” must treat 5xx/network failures for robots as complete disallow, and should not reuse cached robots content for more than 24 hours unless the file is unreachable. It also warns that listing sensitive paths in robots.txt exposes them publicly and that real access control should use actual application-layer security such as HTTP authentication. Google’s documentation adds that X-Robots-Tag can express robot directives at the HTTP-header level and that 429/5xx responses should slow crawling.

For your stated scope—private company/workspace/project knowledge only—the safe policy is:

• internal sources: crawl only via explicitly configured connectors or authenticated site allowlists
• external sources: allow only if the tenant has explicitly enabled them per domain or source collection
• never bypass login walls, CAPTCHAs, paywalls, or rate-limit signals
• honor robots.txt, X-Robots-Tag, and local deny policies
• store fetch failures and blocks as provenance, but never expose them cross-tenant
• treat a blocked or ambiguous legal state as do not crawl

That is product guidance, not legal advice, but it is consistent with the standards: robots control crawling preferences, not authorization; authorization still must be enforced separately; and snapshots should be rights-aware and minimal.

Resource identity policy should also distinguish first-party, specification, research, vendor claim, secondary summary, and community anecdote at ingest time, because the same URL host is not enough. IETF and W3C standards are high-authority normative sources; ACM peer-reviewed content is strong empirical evidence when relevant; arXiv is useful but should default to lower confidence than peer-reviewed work; vendor docs are authoritative about vendor product behavior but not independent proof of vendor performance claims; community posts are useful for troubleshooting and anecdote but low-authority by default.

Migration and test plan

The safest migration strategy is overlay, not rewrite. Start by keeping the existing wiki pages as the canonical bodies while introducing three new structures beside them: concept schemes and taxonomy nodes, immutable provenance events, and normalized URL/citation records. SKOS specifically describes itself as a low-cost migration path for knowledge organization systems, which is exactly the posture you want.

A practical migration path is:

• Phase one: backfill knowledge-unit records from existing wiki pages, preserving page IDs as foreign keys.
• Phase two: extract section structure, keywords, and citations into normalized side tables.
• Phase three: build concept schemes and taxonomy nodes, then attach pages through placement rows.
• Phase four: create URL canonicalization and URL-observation history from existing links.
• Phase five: enable hybrid retrieval and graph expansion behind a feature flag.
• Phase six: move authoring UI to write directly into canonical normalized records.
• Phase seven: retire page-shaped legacy assumptions after parity validation.

This overlay approach works well with zero-trust migration guidance because it lets organizations incrementally adopt stronger authorization and data modeling without big-bang replacement. NIST explicitly describes zero-trust migration as incremental rather than wholesale replacement.

The test strategy should focus on the four failure classes you named.

Deduplication tests should prove that a canonical document keeps one body while appearing in many placements, and that URL variants collapse correctly only when normalization and redirect evidence support equivalence. Include tests for lowercase scheme/host, percent-encoding normalization, removal of dot segments, default port removal, 301/308 permanent redirects, and “same URL text, different fragments” where fragment differences remain distinct when semantically meaningful.

Provenance-preservation tests should prove that re-ingesting a URL with a new title, new crawl status, or new digest creates a new url_observation without overwriting older evidence, and that each citation mention retains its original local selector, anchor text, label, order, and source report provenance. Add regression tests for Crossmark-like status updates, RFC-like obsoletes/updates chains, and snapshot re-capture with unchanged digest.

Conceptual-retrieval tests should prove that queries match by concept path and synonymy, not just string overlap. Example: query prompt budgets should retrieve a knowledge unit whose title omits that exact phrase but is linked to concepts such as tokenization, context windows, or prompt optimization, and placed under an architecture path. Use both positive tests and adversarial tests where seemingly similar words belong to different concept branches. Hybrid retrieval and graph-based retrieval should outperform exact keyword matching on those cases.

Isolation tests should prove that anonymous and cross-tenant requests reveal nothing useful: no titles, no hit counts, no facet buckets, no taxonomy labels, no redirect targets, no crawl-state leakage, and no timing side channels that correlate with object existence. Include red-team tests for manipulated object IDs, unauthorized search filters, cached response confusion, and citation mention lookups across tenants. OWASP’s broken-object-authorization and enumeration guidance makes these tests non-optional.

The final target state is a system where a human can browse by taxonomy, an agent can navigate by concept graph, both can land on the same canonical knowledge unit, and every answer can point back to immutable citation mentions and provenance events. That is the main architectural conclusion from the standards and systems surveyed here: combine PROV-style event provenance, SKOS-style concept organization, Web Annotation-style anchoring, CiTO-style citation intent, RFC-based URL/crawl handling, and zero-trust authorization as the first gate on every operation.