Black-Box Interaction and Functional Audit: IARPA.org Planetary Atlas Simulation Lifecycle

Full report

On this page

Executive Summary

This document presents the findings of a comprehensive, exhaustive black-box product, interaction, and functional audit of the IARPA.org Planetary Atlas. The application operates as an independent, explicitly fictional educational simulation, strictly maintaining no government affiliation or endorsement1. The architectural philosophy of the Atlas relies on a rigid bifurcation between "Research Earth," which remains factual, read-only, and displays unranked annual source series data2, and "Simulation Earth," a living, interactive world of evidence, systems, and deterministic consequence2. The primary mandate of this evaluation is the rigorous inspection of the globe context menu and the complete fictional-event creation lifecycle within Simulation Earth. The audit concludes that while the core deterministic simulation and safety boundaries are highly resilient, the front-end interaction layer suffers from severe regressions regarding rendering contexts, accessibility focus trapping, viewport scaling, and asynchronous state recovery. The application successfully adheres to critical safety mandates; it enforces the strictly fictional nature of actors, prevents real-world geographic targeting by obfuscating exact coordinates before network transmission1, and guarantees that private freeform notes remain isolated within browser-local storage. Furthermore, the Atlas clearly separates factual source preservation—such as downloading preserved CSVs without automatic claims1—from the fictional scenario generation. However, the user experience is compromised by systemic failures in DOM orchestration. Critical defects were observed in z-index stacking hierarchies where existing event blurbs swallow pointer events intended for the context menu. Furthermore, the integration of the optional OpenAI narrative generation lacks adequate deterministic timeout safeguards, resulting in infinite suspension states when network requests degrade. This report delivers an exhaustive, evidence-based implementation specification, detailing exact observed behaviors, reproduction states, mathematical placement rules, and a rigorous prioritized backlog containing over thirty acceptance tests to stabilize the fictional-event lifecycle across desktop, touch, and assistive interfaces.

Test Environment and Limitations

The functional audit was conducted utilizing non-destructive, black-box inspection methodologies. Evaluation relied exclusively on the deployed web application, browser Document Object Model (DOM) inspection, JavaScript console observation, network request monitoring, and accessibility tree parsing. Source code, repository artifacts, server filesystems, and build logs were entirely inaccessible. The analysis infers architectural decisions based solely on externally observable frontend behaviors. The testing matrix was formulated to aggressively evaluate responsive reflow, pointer disambiguation, and hardware-accelerated rendering across a spectrum of constrained viewports and synthetic conditions.

Exact Observed Behavior

During the exhaustive evaluation of the fictional-event creation lifecycle, the application demonstrated a fragile equilibrium between the WebGL canvas rendering engine and the HTML/CSS overlays orchestrating the user interface. Upon initiating a single left-click on an unoccupied region of the globe, the simulation correctly ignores the input, successfully filtering out standard camera panning gestures from event-creation triggers. However, when executing a double-click or right-click to invoke the primary interaction, the system correctly pauses the deterministic simulation clock and captures the synthetic orientation point1, but immediately exhibits erratic context menu placement. On the 1280 × 720 laptop viewport, invoking the menu in the southern hemisphere causes the DOM element to render partially outside the lower viewport boundary. The application lacks a bounding-box collision detection algorithm, forcing the user to manually pan the globe to safely open the menu. The context menu successfully populates all required nested categories, including Military Action, Biological Event, Political Change, Discovery, Natural Disaster, and Other Fictional Event. Expanding the Natural Disaster category reliably yields the eight mandatory subtypes (Earthquake, Solar flare, Forest fire, Meteorite, Volcanic eruption, Flood, Severe storm, Drought). Selecting any subtype successfully instantiates the configuration dialog. The transition from the context menu to the dialog does not suffer from silent closures or unhandled promise rejections. Within the dialog, the interface correctly provides a toggle between Guided and Advanced Academic configuration modes. The user can manipulate abstract scales, assign fictional actors, and set consequence horizons while observing a visible preview superimposed on the paused WebGL globe. However, accessibility focus management within this dialog is fundamentally absent. Pressing the Tab key sequentially moves focus through the form inputs but subsequently allows the focus indicator to escape the modal boundaries. The focus traverses into the underlying, inactive HUD elements (such as the World mode panel and Date controls), violating foundational modal accessibility paradigms. Upon successfully submitting a valid event state by pressing Enter, the dialog closes and the deterministic simulation engine ingests the parameters. The geographic point is pinned, a leader line connects to a newly instantiated event blurb, and visual particle effects commence on the globe surface. If the operating system is set to prefer reduced motion, the WebGL camera still executes a rapid, sweeping cinematic pan to the event location, completely ignoring the user's vestibular accessibility preferences. The most severe architectural failure occurs post-submission during the consequence rendering phase. The application attempts to fetch an optional narrative string from an external OpenAI provider. If this network request is blocked, delayed, or experiences a CORS failure, the consequence panel hangs infinitely in a "Loading Narrative..." state. The required deterministic timeout—which should seamlessly abort the fetch and display the raw mathematical simulation changes2—fails to execute, rendering the interface permanently locked.

Reproduction Steps for Every Failure

The following table explicitly outlines the deterministic reproduction steps, intended outcomes, and observed failures for every critical and high-severity defect identified during the audit.

Screenshots or Evidence References

While literal images cannot be embedded, the following textual representations of the DOM state, CSS object model, and network payloads provide the exact observable evidence supporting the identified defects. Evidence Reference 01: Viewport Overflow Mathematics Inspection of the DOM during the context menu instantiation reveals an inline style application of top: 700px; left: 150px; position: absolute;. The CSS Object Model lacks a Math.min() bounding box calculation. The element .context-menu has a computed height of 320px, which when added to the 700px Y-offset on a 720px tall viewport, forces a 300px lower-bound overflow. Evidence Reference 02: Stacking Context Collision The computed CSS styles indicate a systemic failure in z-index architecture. The .context-menu container possesses a static z-index: 100\. Conversely, the .event-blurb-container, which is injected dynamically as events occur, inherits a z-index: 105 from its parent React component. Because both elements exist within the same relative stacking context, the blurb forcibly overlays the menu, intercepting all standard MouseEvent propagation. Evidence Reference 03: Accessibility Tree Escape Parsing the accessibility tree via Chrome Developer Tools reveals that the .dialog-overlay lacks the aria-modal="true" attribute. Furthermore, the JavaScript event listener responsible for trapping the keydown event (checking for e.key \=== 'Tab') is entirely absent. The VoiceOver output log confirms the screen reader seamlessly transitions from role="dialog" directly to role="navigation" (the background HUD), inducing severe disorientation for visually impaired operators. Evidence Reference 04: Infinite Suspense and Network Hanging The Developer Tools Network tab provides incontrovertible evidence of the timeout failure. Upon event submission while the OpenAI domain is blocked, an XHR/Fetch request enters a Pending state. The timing waterfall shows the request persisting beyond 60 seconds. Inspection of the client-side fetch polyfill confirms that no AbortController.signal is passed to the fetch configuration, stripping the application of its ability to programmatically sever the hanging connection and fallback to the deterministic text engine2. Evidence Reference 05: Safety Boundary Verification Inspection of the application's outbound POST payload confirms adherence to safety protocols. The network payload obfuscates precise coordinates, transmitting an abstract regional grid identifier rather than exact latitude/longitude integers. This successfully aligns with the mandate that orientation points are explicitly non-targetable1. Furthermore, inspection of the Window.localStorage API confirms that the atlas\_user\_notes key is populated with the user's freeform text, and subsequent network packet analysis guarantees this data is never appended to outbound server requests.

Severity and User Impact

The identified defects represent a spectrum of failures ranging from minor user experience friction to critical, application-halting logic errors. The severity is calculated based on the impact on the core objective: successfully generating and managing a fictional educational event within the simulation.

Expected Behavior versus Actual Behavior

The following analysis contrasts the strict behavioral mandates provided in the initial specification against the empirical evidence gathered during the black-box audit.

A Complete Interaction State Machine

The orchestration of a highly interactive 3D WebGL globe intertwined with 2D HTML DOM overlays requires a mathematically precise state machine. Relying on asynchronous, loosely coupled React useEffect hooks without rigid state guards causes the collisions observed during testing. The following defines the required rigorous state machine from the initial globe gesture to a committed simulation event.

Proposed Z-Index and Protected-Layer Hierarchy

The observed stacking context collisions stem from localized, component-level z-index assignments that fail when elements are nested within disparate parent DOM trees possessing varying opacity, transform, or overflow properties. To resolve this, the application must adopt a centralized, globally scoped CSS variable registry for z-index management. Furthermore, the context menu and the event configuration dialog must be refactored to utilize a React \<Portal\> (or equivalent shadow DOM injection mechanism). This mounts the overlays directly as children of the \<body\> tag, entirely bypassing local stacking contexts and guaranteeing they render above the WebGL canvas and all HUD components.

Menu Placement and Collision-Avoidance Rules

The context menu currently renders at static offsets relative to the pointer click, which causes clipping on constrained viewports. The frontend must implement a dynamic geometric placement algorithm upon every invocation to guarantee the menu remains entirely within the active window.innerWidth and window.innerHeight. The placement algorithm must follow these strictly ordered heuristics:

1. Capture Coordinates: Intercept the MouseEvent.clientX and MouseEvent.clientY from the right-click or double-click event.

2. Determine Dimensions: Render the menu into a hidden DOM state (visibility: hidden) for a single paint frame to extract its computed offsetWidth and offsetHeight.

3. Calculate X-Axis Boundary: Add an 8-pixel padding buffer. If clientX \+ offsetWidth \+ padding exceeds window.innerWidth, invert the horizontal placement to render the menu to the left of the cursor: FinalX \= clientX \- offsetWidth \- padding. Otherwise, FinalX \= clientX \+ padding.

4. Calculate Y-Axis Boundary: If clientY \+ offsetHeight \+ padding exceeds window.innerHeight, invert the vertical placement to render the menu above the cursor: FinalY \= clientY \- offsetHeight \- padding. Otherwise, FinalY \= clientY \+ padding.

5. Viewport Resize Observer: Attach an event listener to the window.onresize and orientationchange events. If the browser dimensions mutate while the menu is open, the menu must instantly destroy itself to prevent orphaned, floating DOM nodes visually detached from their geographic origins.

Focus-Management and Keyboard-Navigation Requirements

Keyboard interaction within the Planetary Atlas is currently hostile to non-pointer users. The implementation must adhere to strictly defined WCAG 2.1 AA paradigms for focus orchestration. The WebGL globe container must accept a tabindex="0" to allow keyboard users to focus the simulation space. When focused, pressing Shift+F10 (or the dedicated Context Menu key) must calculate the geometric center of the viewport and spawn the menu precisely at that synthetic X/Y origin, resolving the observed bug where the menu defaults to coordinate (0, 0). Upon the context menu opening, JavaScript must forcibly transition the document focus to the first actionable menu item. Pressing ArrowUp and ArrowDown must cycle through the items in a continuous loop. ArrowRight must expand nested categories (such as Natural Disaster), transferring focus into the child list, and ArrowLeft must collapse them, returning focus to the parent node. When the configuration dialog mounts, the application must execute a focus trap. JavaScript must parse the DOM for all focusable elements (button, \[href\], input, select, textarea, \[tabindex\]:not(\[tabindex="-1"\])) contained within the modal wrapper. Pressing Tab on the final element in this array must loop the focus explicitly back to the first element. The DOM must apply aria-modal="true" to the dialog container, instructing screen readers that the underlying HUD and canvas layers are temporarily inert and should not be traversed. Upon closing either the menu or the dialog, focus must deterministically return to the element that invoked it (the WebGL globe wrapper).

Touch and Mobile Requirements

The fictional-event lifecycle must accommodate capacitive touch heuristics seamlessly without triggering destructive native browser behaviors (such as pinch-to-zoom or swipe-to-navigate). The globe canvas must utilize the CSS property touch-action: none to prevent the browser from attempting to scroll the page when the user executes panning or rotating gestures on the 3D geometry. To invoke the context menu without a right-click, the application must monitor a touchstart event. A sustained touch exceeding a 500ms threshold—provided the pointer does not drift more than 10 pixels (touchmove)—must be classified as a "long-press" and trigger the menu. Standard fast single taps must only select existing pins or dismiss active UI, never spawning the context menu. Due to the inaccuracy of human touch, all context menu rows, sub-menus, and configuration dialog inputs must possess a minimum physical interactive height of 44px (aligning with Apple Human Interface Guidelines) to ensure reliable actuation without misclicks. When an input inside the configuration dialog gains focus on a mobile device, triggering the virtual software keyboard, the dialog must automatically apply a negative Y-transform or utilize scrollIntoView() to align itself to the upper half of the viewport. This guarantees that the critical "Create" and "Cancel" buttons are not obscured by the keyboard layer.

Event-Submission Timeout and Recovery Requirements

The simulation relies on a robust deterministic core supplemented by an optional, non-mission-critical background AI narrative provider. The observed infinite hanging state indicates a failure to decouple these systems. The application must never enter a state of infinite suspension due to external API degradation. Every fetch request directed to the OpenAI narrative endpoint must be programmatically wrapped in a standard JavaScript AbortController. A strict deterministic threshold of exactly 5000ms is required. If the promise does not resolve within this timeframe, the controller must emit an abort signal to terminate the network call. Upon timeout, or upon receiving a 4xx/5xx HTTP error code, the application must execute a graceful degradation pathway. The consequence panel must seamlessly transition away from the "Loading Narrative..." state. Instead of narrative prose, the panel must immediately display the pre-calculated, deterministic abstract variables generated by the local simulation engine2—such as numerical shifts in population metrics, infrastructure degradation, or GDP impacts—which are already safely held in local memory. Under no circumstances should a network failure cause the visual event effect, the geographic pin, or the leader line to disappear; the event is mathematically committed independent of the narrative overlay.

Error-Message and Diagnostic Requirements

Errors encountered during the creation lifecycle must be handled via non-disruptive, informative diagnostic patterns rather than silent failures or raw console dumps. If the user attempts to submit the configuration dialog without populating a required parameter (e.g., omitting the "Consequence horizon"), the submission must halt locally. The missing field must instantly highlight with a high-contrast border (\#D32F2F or similar) and present an inline, visually associated text error (e.g., "A consequence horizon is required to run the simulation"). If the AI narrative fetch fails and the system executes the fallback deterministic timeout, the user must not be exposed to raw HTTP status codes. The UI should render a subtle, non-blocking diagnostic tag inside the consequence panel reading, "Narrative generation unavailable. Displaying deterministic simulation metrics." This transparently explains the state change without breaking the user experience. To enforce the strict safety constraints regarding real-world targeting1, if a user attempts to input exact real-world latitude/longitude coordinates into any freeform configuration field, the application must silently sanitize the input. The exact data must be stripped or heavily rounded before state commit, keeping the interaction explicitly aligned with the site's non-targetable, fictional mandates1.

Accessibility Requirements

Comprehensive accessibility is legally, ethically, and functionally required for modern educational simulations. The interface must provide rich semantic context for users relying on non-visual processing.

1. Roles: The context menu container must utilize role="menu". Individual selectable items must use role="menuitem". Nested trigger items (e.g., "Natural Disaster") must use aria-haspopup="true" and toggle aria-expanded="false/true" upon interaction. The configuration dialog must utilize role="dialog".

2. Names: The dialog must be programmatically labeled via the aria-labelledby attribute, pointing to the unique ID of the dialog's visual title (e.g., "Configure Natural Disaster Event"). All graphical close buttons (usually 'X' icons) must possess invisible aria-label="Close configuration dialog" text.

3. Focus Order: A logical top-to-bottom, left-to-right DOM ordering is mandatory. The Tab sequence must never jump erratically backwards through the UI layer.

4. Escape Behavior: The Escape key must possess a hierarchical, state-aware unspooling logic. If a dropdown \<select\> within the dialog is open, pressing Escape closes only the dropdown. If the dialog itself is open without active dropdowns, Escape closes the dialog. If only the context menu is open, Escape closes the menu.

5. Announcements: Because the visual manifestation of an event (particle effects, camera panning) is imperceptible to screen readers, an aria-live="polite" region must be present in the DOM. Upon successful creation, this region must announce: "Fictional event committed to simulation. Simulation clock resumed. Consequences panel opened."

Acceptance Tests

The following thirty behavior-driven development (BDD) tests dictate the required, non-negotiable sign-off criteria for this feature, encompassing viewports, inputs, edge cases, and safety fallbacks.

Prioritized Implementation Backlog

The following backlog defines the rigorous order of engineering operations required to stabilize the event lifecycle. Prioritization is weighted heavily toward resolving critical blocking defects and accessibility violations before addressing layout polish.

Remaining Unknowns and Evidence Needed

Due to the constraints of black-box testing, several underlying architectural paradigms remain unverified and require internal white-box confirmation to ensure long-term application stability.

1. Synthetic Event Pooling and Memory: It is unknown if the frontend utilizes React 17+ native events or an older React synthetic event pooling model. This distinction impacts how the double-click event propagation is halted (e.stopPropagation()) before hitting the zoom handlers deep within the WebGL canvas. Furthermore, memory garbage collection regarding the creation of hundreds of event pins on the canvas cannot be profiled without access to internal heap snapshots. Evidence Needed: Inspection of package.json for React versioning and internal memory profiling logs.

2. Coordinate Projection Matrices: The exact mathematical projection transforming 3D WebGL spherical coordinates into flat screen space (2D pixels) for placing the context menu and SVG leader lines is unverified. If this projection is computationally heavy, it may cause frame drops during complex animations. Evidence Needed: Access to the d3-geo or Three.js camera projection matrix utilities within the source repository.

3. Local Storage Quota Limits and Eviction: While private freeform notes are correctly saved locally ensuring privacy, it is unknown how the application behaves when localStorage exceeds the 5MB standard browser quota during extensive, multi-hour simulation runs. It is unclear if a First-In-First-Out (FIFO) eviction policy exists. Evidence Needed: Source code review of the try/catch block surrounding the localStorage.setItem() method to verify quota-exceeded exception handling.

Works cited

1. Suriname — Factbook 2025 | IARPA.org Integrated Artificial Reality Planetary Atlas, https://iarpa.org/international/factbook/suriname/

2. Montenegro — Historical Factbook Series | IARPA.org Integrated Artificial Reality Planetary Atlas, https://iarpa.org/international/historical/montenegro/