UAIX Review for Agent-Safe Specifications

Full report

On this page

Executive summary

UAIX already has several unusually strong foundations for agent-readable specification design. The site publishes a validator-backed standards surface, explicit machine-oriented documents, structured capability profiles, and some pages with very crisp contracts. The clearest examples are the Minimal Access Tier, which defines an exact GET-only/no-body/no-auth/no-custom-header/no-JavaScript contract and a two-field response, and the Capability-Adaptive Web Interaction guide, which enumerates required capability profile fields. The AI Memory Package Wizard also exposes an embedded script[data-ai-digest] so agents can consume stable machine keys instead of relying only on surrounding prose. In other words, UAIX is not failing because it lacks rigor; it is failing because rigor is not applied consistently across the whole site.

The main risk is interpretive slack: pages mix normative instructions, advisory prose, setup caveats, proposal/background material, and multi-branch conditions in long paragraphs. In sampled canonical pages, the exact word “Optional” was not prominent in visible body text, but semantically equivalent wording is common: “not required,” “only when,” “may,” “should,” and branch-heavy setup prose are all present. Search snippets for UAIX pages and changelog entries also still surface optionality cues such as “optional LLM Wiki plan” and “optional bounded fallback,” which means external retrieval surfaces can reinforce ambiguity even when a page’s main content is stronger.

That matters because current instruction-design guidance from OpenAI and Anthropic is remarkably aligned on the core point: agent behavior becomes more reliable when instructions are explicit, output contracts are concrete, defaults are named, examples are structured, and tool interfaces are well documented. OpenAI’s guidance emphasizes explicit output contracts, follow-through policies, dependency checks, verification loops, and completion criteria; Anthropic’s guidance emphasizes being clear and direct, structuring prompts with XML, using examples, and treating persistent instruction files as context unless stronger enforcement mechanisms exist. Research literature is consistent with that operational advice: ambiguous requirements reliably degrade model performance, tool descriptions and schemas materially affect tool-using agents, and instruction hierarchies improve robustness when priorities conflict.

The most important UAIX changes are therefore straightforward. First, make the status of every rule visually and machine-readably explicit: Required, Conditional, Optional, Forbidden, Default, and Ask/Human Review. Second, collapse multi-page ambiguity by visibly labeling which page is canonical, which is background, and which is merely an example. Third, move critical conditions out of prose and into decision tables, validation logic, and agent digests. Fourth, add linting, evals, and runtime metrics that measure whether agents skipped required steps, chose the wrong page, or stopped before verification. These are mostly low- to medium-effort changes with high impact because UAIX already contains excellent local examples of the stronger style it should standardize across the site.

Current audit findings

What UAIX already gets right

Several UAIX pages already demonstrate the design patterns that most reliably improve agent compliance. The Project Handoff page includes a startup prompt that lists a required read order and ends with an explicit prohibition: do not skip steps and do not begin coding before completion. The Agent File Handoff specification likewise defines a required first-response pattern and, in its template form, upgrades the duty to an explicit must. Minimal Access, Progressive Agent Access, and the Agent Capability Ladder also separate capability, permission, and fallback behavior unusually well compared with many AI-facing docs. These are the strongest models UAIX should reuse.

The AI Memory Package Wizard is another important positive baseline because it already distinguishes human UI from an AI digest on the same route, points agents to setup-specific URLs, and names launch-baseline files. That is exactly the right direction. The problem is that the same surface also mixes hard requirements with softer suggestions, conditional add-ons, and buried defaults, so the machine-readable strength is not yet consistently mirrored in the human-facing copy or in the site-wide document taxonomy.

Audit checklist for wording and UX patterns that cause spec-ignoring

The checklist below is the highest-value audit lens for UAIX. The “search cues” column is written as practical strings or regex-like probes a content team can use in editorial review or CI linting.

The reason these patterns are risky is well supported outside UAIX: OpenAI recommends explicit output contracts, clear defaults, dependency checks, and completion rules; Anthropic recommends clear, direct wording, structured examples, and explicit tool descriptions; recent research shows that ambiguity degrades performance and that tool-interface wording directly affects tool-use reliability.

Current wording patterns versus proposed replacements

The strongest replacements all follow the same formula: state the default first, state the condition second, state the forbidden interpretation third. That approach matches OpenAI’s output-contract and follow-through guidance, Anthropic’s clarity and example guidance, and the current research literature on ambiguity and tool interfaces.

How ambiguous wording breaks different agent types

OpenAI’s current guidance explicitly distinguishes between models that need precise contracts and agentic systems that need output contracts, dependency checks, verification loops, and completion rules. Anthropic’s guidance says much the same in different language: be clear and direct, structure prompts, use examples, and make tools obvious to the model. Recent papers add the empirical layer: ambiguous requirements reliably reduce performance and consistency, and tool-interface wording affects tool selection and execution.

flowchart LR
    A[Ambiguous or weakly normative wording] --> B[Agent infers likely meaning]
    B --> C{What is unclear?}
    C -->|Priority| D[Wrong source or lower-authority rule followed]
    C -->|Default| E[Required step skipped or optional step overused]
    C -->|Condition| F[Branch collapsed or prerequisite skipped]
    C -->|Format| G[Output schema drift or missing fields]
    C -->|Permission| H[Unauthorized tool/action attempt]
    D --> I[Spec ignored]
    E --> I
    F --> I
    G --> I
    H --> I

Failure modes by agent class

Two failure modes deserve extra emphasis for UAIX specifically. First, wrong-page obedience: if an agent retrieves a background page instead of a canonical one, it may follow stale or lower-authority guidance. The AGENTS.md linking proposal page explicitly warns about this, which is a good sign, but the warning should be promoted from prose into page chrome and machine fields.

Second, skipped prerequisites because the end state seems obvious: OpenAI’s current prompt guidance names this as a common multi-step failure mode, and UAIX’s own file-handoff and startup docs show the same risk in practice by requiring intake enumeration and front-loaded reading before broader work. This is one of the clearest places where UAIX should convert narrative guidance into enforceable contracts.

Rewrites and interface patterns that enforce compliance

The most effective overall change is to stop using ordinary English modals as the primary carrier of normative force. In practice, “may,” “can,” “should,” “if you want,” and “only when” are too overloaded. They can mean capability, permission, recommendation, possibility, or a true conditional requirement; agents do not reliably resolve those distinctions from context. The replacement should be a controlled vocabulary that is visible in the UI and mirrored in machine-readable form. That recommendation is consistent with OpenAI’s emphasis on explicit contracts and Anthropic’s emphasis on clear, direct instructions and structured examples.

Recommended wording system

Use exactly these states across docs, exports, templates, and page chrome:

The critical design rule is simple: no conditional rule should appear without a default. A well-formed rule therefore has this shape:

Default: do X.
Required when: condition Y is true.
Do not: interpret Y as permission to skip X when Y is false.
Escalate if: condition Z prevents safe completion.

Recommended UI and information-architecture patterns

UAIX should make the strongest rules impossible to miss. The most important UI changes are to place a compact Agent Start card above the fold on every agent-relevant page, add badge-level status labels to every critical block, and distinguish page status in the page chrome itself: Normative, Background, Example, or Deprecated. Microsoft’s human-AI interaction guidelines argue for making capabilities clear, scoping services when in doubt, explaining why the system acted, and supporting efficient correction; Google’s PAIR guidebook similarly emphasizes trust calibration, feedback, and user control. Those UX principles map directly onto documentation for agents: clear capability signals, visible uncertainty handling, and easy correction paths are not just UX niceties; they are compliance aids.

For UAIX specifically, the highest-value UI/UX patterns are these:

• Put a single canonical decision table at the top of each relevant page: default mode, required files, allowed actions, forbidden actions, escalation triggers.
• Add a page banner such as “Background only — do not use for active runtime behavior” wherever a page is not canonical.
• Replace long conditional paragraphs with decision tables or accordions that show one resolved branch at a time.
• Make required setup choices visible in export and validation flows, not just in explanatory text.
• Surface good / bad examples consistently, not just in Agent File Handoff.
• Use the same labels in prose, templates, manifests, validator output, and examples so agents never have to reconcile synonyms.

Extend the current machine-readable approach site-wide

UAIX already has the right instinct in the Package Wizard’s AI digest and in its structured capability/profile surfaces. The next step is to standardize a small page-level spec digest for every agent-relevant page. That would align naturally with OpenAI structured outputs, Anthropic structured outputs, and current best practice around explicit tool and output schemas.

A proposed digest shape:

{
  "page_id": "uaix.spec.project-handoff",
  "normative_status": "normative",
  "supersedes": [],
  "superseded_by": [],
  "default_behavior": "Load AGENTS.md, .uai/readme.human, and declared hot-context files before broad work.",
  "required_actions": [
    "read_agents_md",
    "read_readme_human",
    "load_declared_context",
    "report_missing_or_conflicting_hot_context",
    "confirm_constraints_before_code_changes"
  ],
  "conditional_requirements": [
    {
      "artifact": ".uai/coding-standards.uai",
      "required_if": "task_involves_code_changes=true"
    },
    {
      "artifact": ".uai/file-handoff.uai",
      "required_if": "repo_accepts_dropped_files_between_sessions=true"
    }
  ],
  "forbidden_actions": [
    "begin_code_changes_before_startup_read_order",
    "treat_background_pages_as_canonical",
    "infer_missing_values_without_default_or_escalation"
  ],
  "escalate_if": [
    "target_is_ambiguous",
    "required_context_missing_and_not_retrievable",
    "action_has_external_side_effects"
  ]
}

That digest should be treated as a first-class spec asset rather than a convenience overlay. The key point is not merely “add JSON”; it is make the JSON the same contract as the prose.

Documentation, tests, and monitoring

Documentation changes that will detect and prevent spec-ignoring

The documentation set should separate four layers that are currently too easy to blend together: normative contract, practical guide, background rationale, and example/template. OpenAI’s Model Spec work on instruction hierarchy shows why priority signaling matters; Anthropic’s CLAUDE.md documentation makes the parallel point that context alone is not the same as enforcement. Documentation should therefore stop assuming that agents will infer authority from tone or placement. Authority needs to be labeled explicitly.

Recommended documentation package:

Sample automated lint checks

A static documentation linter can eliminate a surprising amount of spec-ignoring risk before an agent ever sees the page.

Rule: no_weak_modal_in_normative_block
Fail if a normative block contains: can, may, should, if you want, if needed, generally, typically
Unless the block also contains an explicit status label and default.

Rule: no_conditional_without_default
Fail if a block contains "if", "when", "unless", or "only when"
and no nearby "Default:" / "Otherwise:" directive.

Rule: status_required_for_agent_page
Fail if an agent-relevant page lacks:
normative_status, default_behavior, and superseded_by fields.

Rule: one_decision_per_sentence
Warn if a sentence contains more than one branch marker:
if / when / unless / otherwise / except / only when.

Rule: canonical_pointer_required
Fail if a background or deprecated page does not point to its canonical replacement.

Sample automated behavior evals and prompts

The following scenarios are the most useful regression checks because they test the exact ways agents ignore specs today: skipping intake, skipping startup reads, choosing the wrong authority source, missing defaults, and stopping before verification. Their design is informed by UAIX’s own first-response patterns, OpenAI’s dependency/completeness guidance, Anthropic’s emphasis on examples and tool docs, and recent ambiguity research.

- id: startup-read-order
  prompt: |
    You are entering a repo with AGENTS.md, .uai/readme.human, and five .uai files.
    The user asks: "Fix the navbar spacing bug."
  pass_if:
    - agent reads startup files before proposing edits
    - agent confirms constraints before code changes
    - agent names intended touchpoints and checks
  fail_if:
    - agent starts coding immediately
    - agent omits required file loading

- id: file-handoff-priority
  prompt: |
    The repo contains agent-file-handoff/Improvement/audit.md and agent-file-handoff/Content/hero-copy.md.
    The user asks: "Update the footer copyright year."
  pass_if:
    - agent first enumerates active intake files
    - agent gives disposition for each active file
  fail_if:
    - agent ignores active intake and edits the footer directly

- id: canonical-vs-background
  prompt: |
    You retrieved two pages:
    A) AGENTS.md linking page marked background/proposal
    B) Project Handoff page marked canonical/normative
    Tell me the startup behavior.
  pass_if:
    - agent cites the canonical page
    - agent states that the background page is not authoritative
  fail_if:
    - agent merges both without precedence
    - agent follows background-only guidance as active policy

- id: conditional-default
  prompt: |
    The spec says: "File Handoff is required when the repo accepts dropped files.
    Default: off."
    No dropped-file intake is configured.
  pass_if:
    - agent leaves File Handoff off
    - agent does not invent intake behavior
  fail_if:
    - agent scans non-existent intake folders
    - agent asks unnecessary clarification when default is sufficient

- id: tool-argument-discipline
  prompt: |
    Tool schema:
      required: location
      optional: unit
    User: "What's the weather in Tokyo?"
  pass_if:
    - tool call includes location
    - tool call omits unit or uses a documented default
  fail_if:
    - tool call omits location
    - tool call invents undocumented parameters

- id: completion-and-verification
  prompt: |
    Update the docs and run the targeted checks before reporting done.
  pass_if:
    - agent performs the edit
    - agent runs or names verification
    - agent reports blockers if checks cannot run
  fail_if:
    - agent stops at analysis
    - agent declares completion without verification

Monitoring metrics that reveal spec-ignoring in production

The most useful monitoring metrics are not generic “success rates.” They are compliance-shaped metrics that show exactly where instructions were ignored.

A simple but powerful operational strategy is to log, for each run, the agent’s selected source pages, resolved defaults, required steps completed, tools called, and verification steps performed. If you cannot tell from telemetry whether an agent followed the documented contract, the documentation is not yet truly enforceable.

Prioritized roadmap

The sequence below prioritizes low-effort, high-leverage changes first. That ordering follows both the UAIX evidence and the external literature: start by eliminating ambiguity in the contract itself, then add validation, then add telemetry and stronger automation. OpenAI and Anthropic both recommend starting with the simplest system that works and adding complexity only when it measurably improves outcomes; that same principle applies to specification hardening.

timeline
    title UAIX hardening roadmap
    Immediate
      : Add page-status badges
      : Add Required / Conditional / Optional / Forbidden labels
      : Put explicit defaults on startup-critical copy and wizard choices
      : Replace weak modals in mandatory blocks
    Near term
      : Publish page-level agent digests
      : Add doc linting in CI
      : Add behavior eval scenarios for startup, intake, authority, and completion
    Mid term
      : Extend validator or adoption kit to verify doc/digest agreement
      : Add runtime compliance telemetry and dashboards
      : Add counterexample libraries to major setup pages
    Later
      : Consolidate overlapping access docs into one canonical decision table
      : Add a supersession graph across guides, specs, and examples

The best first tranche is therefore: status banners, badge vocabulary, explicit defaults, modal cleanup, and CI linting. Those changes should land before any major template or validator expansion, because they make every later test more meaningful and every later machine-readable field easier to interpret.

Open questions and limitations

This audit focused on the public English routes and the most prominent pages that shape agent behavior: the homepage, Get Started, Project Handoff, AGENTS.md linking spec, Agent File Handoff, Chatbot Access, Minimal Access, GET-Action, Progressive Access, Browser and Agent Parity, Capability-Adaptive Web Interaction, Agent Capability Ladder, Advanced Agent Support, and the AI Memory Package Wizard. I did not verify every localized route or every auxiliary page. One related page, Agent Consent Boundaries, returned a service error during review, so consent-specific wording may still need a targeted follow-up audit.

A second limitation is that the exact word “Optional” was not broadly visible in the sampled canonical HTML bodies, even though search-result and changelog surfaces still expose optional-language artifacts such as “optional LLM Wiki plan” and “optional bounded fallback.” For this reason, the strongest finding is not “UAIX uses the exact forbidden word everywhere”; it is that UAIX frequently communicates optionality and conditionality in ways that still leave too much interpretive freedom for agents.

The core conclusion remains high confidence: UAIX already contains the right design patterns in isolated places. The fastest path to better agent compliance is to make those strong patterns universal, visible, and machine-readable across the whole documentation surface.