Uai.LlmRuntime Improvement Research Report

Full report

On this page

Executive summary

The repository is a well-structured early scaffold, not yet a production-ready LLM runtime. Its strongest qualities are the clean separation of contracts, abstractions, runtime orchestration, deterministic test backend, and a deliberately narrow initial scope. That matches the design direction in the accompanying architecture papers: start with a provider-neutral managed core, keep backend seams explicit, and only then expand into real adapters, tokenizer parity, memory-mapped model loading, vectorized kernels, telemetry, and deployment hardening.

The most important conclusion is that the next best investment is not “jump straight to full GGUF/CUDA/native-runtime parity.” The next best investment is to harden the scaffold into a credible library package and integration core: centralize dependency/version management, add reproducible restore and audit gates, make CI executable and observable, expand the contracts for real-world inference features, harden the .uai memory store, add telemetry, and implement the first real adapter. That staged path is also the one most consistent with the attached research materials, which repeatedly favor a managed-first core, thin backend boundaries, and incremental specialization rather than an all-at-once port of every low-level backend concern.

A second conclusion is that the project’s operational posture needs more work than its code shape. The repo currently targets net8.0, but the SDK pin is 8.0.100, while Microsoft recommends staying current on supported SDK servicing, and currently supports .NET 10 through November 2028, .NET 9 through November 2026, and .NET 8 through November 2026. Microsoft also explicitly recommends upgrading to the latest SDK version even when targeting older runtimes.

The most urgent improvements are therefore straightforward and high leverage: conditionalize ContinuousIntegrationBuild, add Central Package Management, enable lock files, enable NuGet audit, add package validation, ship Source Link and symbols, add GitHub Actions + CodeQL, and instrument the core with ILogger/Meter/ActivitySource-friendly telemetry so a future host can adopt OpenTelemetry cleanly. Microsoft’s own guidance supports each of those moves.

One important caveat applies to all execution-related findings: this analysis environment did not have the dotnet CLI installed, so the repository could be reviewed statically, but dotnet restore, dotnet test, packing, and vulnerability checks could not be executed here. The recommendations below therefore separate static-review findings from runtime verification work that should be completed in CI immediately.

Current state assessment

Static review shows a small, coherent single-solution .NET library with one library project, one xUnit test project, a deterministic local adapter, a .uai file-backed memory store, a whitespace tokenizer, vector-math helpers, and a top-k selector. Architecturally, that is a sound seed for a provider-neutral runtime core, and it aligns with the attached architecture materials that recommend canonical request/response contracts, tokenizer abstraction, memory-store abstraction, and backend isolation as the stable center of the system.

What is present is useful. What is absent is more important:

The largest performance hotspot today is not VectorMath; it is the .uai memory subsystem. The current file store serializes all append and read operations through a single SemaphoreSlim, reads the entire file into memory on every query, reparses the whole document each time, and has no index or compaction strategy. That is acceptable for deterministic tests and demos, but it becomes the first scalability bottleneck as soon as conversations grow or multiple runtime instances share a storage path. The attached runtime references strongly favor explicit memory design, bounded hot paths, and scalable state management; the current implementation is still at the toy-to-prototype end of that spectrum.

The largest correctness gap is token accounting and prompt preparation. The current tokenizer is intentionally simple, but that means token counts are not provider-accurate and message budgeting does not reflect role overhead, chat-template overhead, or provider-specific serialization rules. The architecture notes repeatedly emphasize tokenizer correctness and metadata-driven model behavior as foundational rather than optional; that should move closer to the top of the roadmap.

The largest operational gap is repository hygiene. The repo already sets Deterministic and AnalysisLevel=latest, which is good, but ContinuousIntegrationBuild is currently enabled unconditionally. Microsoft documents that ContinuousIntegrationBuild=true is intended for official CI builds and normalizes stored file paths; on a local machine this can weaken the debugging experience because normalized paths may no longer map cleanly to local source files. That property should be conditioned on CI environment variables instead of being always on.

The target architecture should look like this:

flowchart LR
    Client[Library Consumer or Host]
    Client --> Runtime[Uai.LlmRuntime Core]

    Runtime --> Contracts[Canonical Contracts]
    Runtime --> Prompt[Prompt Preparation]
    Runtime --> Tokenizer[Tokenizer Service]
    Runtime --> Memory[Memory Store]
    Runtime --> Telemetry[Metrics and Traces]
    Runtime --> Scheduler[Request Scheduling]
    Runtime --> Adapter[IModelAdapter]

    Adapter --> Provider[OpenAI Compatible Adapter]
    Adapter --> Azure[Azure OpenAI Adapter]
    Adapter --> Local[Process Isolated Local Worker]

    Local --> GGUF[GGUF or ONNX Worker]
    Telemetry --> OTel[OpenTelemetry Export Path]

That direction is directly aligned with the attached enterprise/runtime architecture notes: a stable application-facing core, thin adapters, and optional local backend isolation rather than immediate monolithic in-proc engine complexity.

Prioritized improvements

The table below orders the work by value delivered per engineering hour, not by conceptual ambition.

Packaging, dependency, and CI hardening

This should happen first because it reduces risk everywhere else. Microsoft recommends Central Package Management for solutions with multiple projects, supports packages.lock.json for restoring the full dependency closure, and documents --locked-mode for CI so dependency graphs do not drift unexpectedly. Microsoft also documents NuGet audit configuration at repository level, including NuGetAudit, NuGetAuditMode, and NuGetAuditLevel.

This improvement should include these changes:

• Add Directory.Packages.props.
• Move test package versions there.
• Add RestorePackagesWithLockFile=true.
• Commit lock files.
• Set ContinuousIntegrationBuild only on CI.
• Enable CheckSdkVulnerabilities=true.
• Enable EnablePackageValidation=true.
• Add Source Link, symbols, and repository metadata.
• Add GitHub Actions with restore, build, test, pack, coverage, and security jobs.
• Add CodeQL for C# because GitHub documents that CodeQL supports C# and identifies vulnerabilities and errors surfaced as code scanning alerts.

Contract expansion

The current request/response model is good for smoke tests but too narrow for realistic LLM use. The attached enterprise architecture note argues for a canonical contract boundary that can normalize provider differences rather than leaking provider DTOs everywhere, and that is exactly the right next step here.

Additive contract changes should include:

• ToolDefinition and ToolChoice
• ResponseFormat with JSON schema mode
• stop sequences
• seed
• presence/frequency penalties where relevant
• finish reason enum
• adapter-side error contract
• richer streaming events
• optional multimodal content envelopes
• model capabilities metadata

This is medium effort and high leverage because every future adapter or local backend will rely on it.

First real adapter

The highest-ROI functional improvement is to add one real adapter. The enterprise architecture material strongly supports a provider-neutral runtime with pluggable adapters for OpenAI-compatible and other hosted providers, while isolating local inference as a more specialized backend path.

The first adapter should be OpenAI-compatible HTTP rather than native GGUF. Reasons:

• It makes the library genuinely useful immediately.
• It exercises canonical contracts, serialization, retries, streaming, and telemetry.
• It keeps the local-backend decision open.
• It reduces risk compared with implementing a local kernel/runtime path too early.

Estimated effort is 24–40 hours including DTO mapping, SSE streaming, retry discipline, and tests.

Tokenization and token budgeting

Improving tokenization is mandatory before serious provider or local integration. The current whitespace tokenizer is appropriate as a deterministic fallback, but it is not an accurate billing, truncation, or context-budget mechanism. The accompanying runtime papers repeatedly treat tokenizer and metadata correctness as core runtime concerns, not convenience helpers.

Recommended shape:

• Keep ITokenizer but add encode/decode token IDs.
• Add provider-aware token accounting.
• Add truncation helpers that operate at message-sequence level.
• Add deterministic fallback tokenizer only as a last-resort implementation.
• Add golden tests for sample prompts and edge punctuation.

.uai memory store hardening

The current append-only store should evolve from “simple file persistence” into “durable, verifiable local journal.” At minimum, it needs:

• escaping or encoding that cannot collide with user content delimiters
• on-read SHA-256 verification
• sidecar index for fast conversation reads
• file-size rotation or compaction
• multiprocess-safe file append strategy
• corruption-tolerant parser behavior
• bounded memory-injection policy based on tokens, not just characters

This recommendation is consistent with the broader architecture notes that emphasize explicit memory mechanics and bounded runtime behavior.

Telemetry and monitoring

Telemetry should land inside the core package, but exporters should remain optional in a host package. Microsoft’s OpenTelemetry guidance for .NET explains that the .NET ecosystem already instruments around ILogger<T>, System.Diagnostics.Metrics.Meter, and System.Diagnostics.ActivitySource, with OpenTelemetry used to collect and export that data. That maps perfectly to this repository’s zero-runtime-dependency preference: keep instrumentation primitives in core, and let hosts choose exporters later.

Add these minimum signals:

• request count
• request duration histogram
• stream duration histogram
• adapter error count
• token counts in/out
• memory store append latency
• memory store read latency
• memory entries injected
• memory file size gauge in host
• cancellation count

Local backend strategy

If local inference is still a goal, the evidence strongly favors a process-isolated worker rather than immediately moving native/local runtime code into this package. The architecture materials discuss managed-first orchestration, aggressive backend isolation, and optional local/native expansion; the enterprise runtime note also favors keeping core contracts stable while plugging in backends externally.

For this repo, the near-term path should therefore be:

• Uai.LlmRuntime remains the canonical contracts/orchestration package.
• Uai.LlmRuntime.OpenAI becomes the first external adapter package.
• Uai.LlmRuntime.LocalWorker becomes a later process-isolated bridge package.
• Native AOT compatibility analyzers can be added to library projects that are intended to support AOT. Microsoft documents that IsAotCompatible=true turns on trim, single-file, and AOT analyzers, and that verification of referenced assemblies can also be enabled.

Master improvement prompt

The prompt below is designed to be pasted into an engineering agent or coding assistant to drive the next serious iteration of the repository.

You are improving the repository `Uai.LlmRuntime`, a .NET 8 library that currently provides:
- canonical inference contracts
- abstractions for runtime, adapter, tokenizer, and `.uai` memory store
- a deterministic local test adapter
- a whitespace tokenizer fallback
- vector math fallback helpers
- a top-k selector
- xUnit tests
- build scripts for restore/test/pack

Your mission is to evolve it from a scaffold into a production-grade provider-neutral runtime core without breaking its main architectural intent.

Primary goals

- Preserve the clean, provider-neutral public API shape.
- Keep the core package small, focused, and library-first.
- Make all changes additive or carefully compatibility-preserving unless absolutely necessary.
- Prefer primary .NET platform capabilities and official guidance.
- Keep runtime dependencies in the core package minimal.
- Push optional hosting, exporters, and provider-specific implementations into separate packages when appropriate.

Required deliverables

- A hardened `Uai.LlmRuntime` core package.
- A first real hosted-model adapter package, preferably `Uai.LlmRuntime.OpenAICompatible`.
- Reproducible CI with restore/build/test/pack/security gates.
- Better contract coverage for tools, structured outputs, streaming, and adapter errors.
- A safer and more scalable `.uai` memory-store implementation.
- Telemetry hooks that are OpenTelemetry-friendly but do not force exporters into the core package.
- Expanded automated tests, including negative and concurrency cases.
- Updated documentation, changelog, and package metadata.

Repository-wide engineering rules

- Target `net8.0` initially, but build with a current supported SDK.
- Keep XML documentation on all public APIs.
- Keep nullable enabled.
- Treat warnings as errors.
- Use UTC consistently for persisted timestamps.
- Do not introduce reflection-heavy patterns into the core package.
- Do not hard-wire provider DTOs into canonical contracts.
- Avoid breaking public APIs unless justified and documented.
- Maintain deterministic behavior in tests.

Step-by-step work plan

1. Package and build-system hardening
- Add `Directory.Packages.props` and move all package versions there.
- Add `RestorePackagesWithLockFile=true` and commit lock files.
- Add `CheckSdkVulnerabilities=true`.
- Configure NuGet audit centrally at repository level.
- Keep `ContinuousIntegrationBuild` conditional on CI environment variables only.
- Add package validation (`EnablePackageValidation=true`).
- Add Source Link support and symbol publishing metadata.
- Add repository metadata so the package is debuggable and traceable.

2. CI/CD
- Add a GitHub Actions workflow for:
  - restore
  - build
  - test
  - coverage collection
  - pack
  - artifact upload
- Add a separate CodeQL workflow for C#.
- Fail CI on restore drift, test failure, security-scan failure, and pack validation issues.
- Use locked restore in CI.

3. Runtime contracts
- Extend `InferenceRequest` with:
  - stop sequences
  - seed
  - tool definitions
  - tool choice
  - response format / JSON schema mode
  - optional metadata for multimodal content placeholders
- Extend `InferenceResponse` with:
  - strongly typed finish reason
  - adapter/provider status info
  - normalized warnings/errors metadata
- Extend streaming contracts so events can represent:
  - started
  - delta
  - tool-call delta
  - usage
  - completed
  - error
- Add a canonical adapter error model that can capture:
  - HTTP/network failures
  - rate limits
  - validation errors
  - provider-specific status mapping

4. Tokenization
- Preserve the whitespace tokenizer as a deterministic fallback only.
- Add a richer tokenizer abstraction that supports:
  - encode/decode token IDs
  - count for single text and full message sequences
  - truncation to token budget
- Add provider-aware token accounting helpers.
- Add golden tests for punctuation, whitespace normalization, role overhead, and truncation behavior.

5. Real adapter
- Create a new adapter package for OpenAI-compatible HTTP APIs.
- Keep canonical contracts in the core package.
- Implement:
  - non-streaming generation
  - streaming generation
  - model metadata lookup if feasible
  - cancellation
  - timeout handling
  - retry recommendations for transient failures
- Map provider responses into canonical runtime responses.
- Do not leak provider-specific DTOs across the boundary.

6. Telemetry and diagnostics
- Instrument the core runtime with:
  - `ActivitySource`
  - `Meter`
  - counters and histograms for request count, duration, token counts, and failures
- Add structured logging integration points without forcing a specific logging implementation into the core.
- Make telemetry names stable and documented.

7. `.uai` memory store hardening
- Replace delimiter-fragile raw content storage with a safer encoding strategy.
- Verify stored content hashes on read.
- Add corruption-tolerant parsing.
- Add optional sidecar indexing or a lightweight read-optimization strategy.
- Add file-size rotation/compaction policy.
- Document single-process vs multi-process guarantees explicitly.
- Ensure memory injection is bounded by token budget, not only character count.

8. Tests
- Expand unit tests to cover:
  - malformed memory files
  - content containing delimiter-like text
  - multi-entry ordering and filtering
  - cancellation during streaming
  - deterministic tie handling in top-k
  - token truncation and memory-budgeting logic
- Add integration-style tests for the hosted adapter using fake HTTP handlers.
- Add package-validation and `dotnet pack` verification in CI.

9. Documentation
- Update README to explain:
  - current scope
  - package structure
  - extension strategy
  - telemetry hooks
  - limitations
  - roadmap
- Add a SECURITY.md and CONTRIBUTING.md.
- Update package/changelog docs to reflect all architectural changes.

Hard constraints

- Do not claim GGUF/CUDA/DirectML parity.
- Do not add heavyweight runtime dependencies to the core package.
- Do not move the project away from a provider-neutral public contract model.
- Do not replace existing deterministic tests; extend them.
- Do not break `.uai` memory compatibility silently. If a format change is required, add migration or versioning.

Acceptance criteria

- `dotnet restore` succeeds with lock files.
- `dotnet test` passes on CI.
- `dotnet pack` succeeds with package validation enabled.
- CodeQL workflow is active.
- The package emits useful telemetry primitives.
- The first real adapter is usable in tests and documented.
- The memory store is more robust, verifiable, and scalable than the current implementation.
- Public API changes are documented in the changelog.

That prompt is consistent with the repository’s own direction and with the attached architecture notes favoring a provider-neutral core, additive adapters, and later specialization of local/native backends.

Concrete implementation guidance

Dependency and repository hardening patch

The repository should adopt Central Package Management and lock files, because Microsoft explicitly documents CPM for multi-project solutions and packages.lock.json for reproducible restore, including --locked-mode for CI.

+++ Directory.Packages.props
+<Project>
+  <PropertyGroup>
+    <ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>
+  </PropertyGroup>
+  <ItemGroup>
+    <PackageVersion Include="Microsoft.NET.Test.Sdk" Version="18.6.0" />
+    <PackageVersion Include="xunit" Version="2.9.3" />
+    <PackageVersion Include="xunit.runner.visualstudio" Version="3.1.5" />
+    <PackageVersion Include="coverlet.collector" Version="10.0.1" />
+  </ItemGroup>
+</Project>

--- Directory.Build.props
+++ Directory.Build.props
 <Project>
   <PropertyGroup>
     <Nullable>enable</Nullable>
     <ImplicitUsings>enable</ImplicitUsings>
     <LangVersion>12.0</LangVersion>
     <Deterministic>true</Deterministic>
-    <ContinuousIntegrationBuild>true</ContinuousIntegrationBuild>
     <AnalysisLevel>latest</AnalysisLevel>
+    <RestorePackagesWithLockFile>true</RestorePackagesWithLockFile>
+    <CheckSdkVulnerabilities>true</CheckSdkVulnerabilities>
+    <NuGetAudit>true</NuGetAudit>
+    <NuGetAuditMode>all</NuGetAuditMode>
+    <NuGetAuditLevel>moderate</NuGetAuditLevel>
   </PropertyGroup>
+
+  <PropertyGroup Condition="'$(GITHUB_ACTIONS)' == 'true' Or '$(TF_BUILD)' == 'true'">
+    <!-- Only enable CI-specific path normalization and official-build behavior in CI. -->
+    <ContinuousIntegrationBuild>true</ContinuousIntegrationBuild>
+  </PropertyGroup>
 </Project>

Two important notes go with that patch. First, ContinuousIntegrationBuild should be conditional rather than unconditional because Microsoft says it is intended for official CI builds and normalizes file paths in ways that can hurt local debugging. Second, if you take coverlet.collector to 10.0.1, its own package page says you should use SDK 8.0.414 or newer and at least Microsoft.NET.Test.Sdk 18.4.0, so the current global.json pin should be updated at the same time.

Package quality patch

Microsoft recommends Source Link, symbol publishing, deterministic builds, and package validation for library authors. Package validation can be enabled directly with EnablePackageValidation=true.

--- src/Uai.LlmRuntime/Uai.LlmRuntime.csproj
+++ src/Uai.LlmRuntime/Uai.LlmRuntime.csproj
 <Project Sdk="Microsoft.NET.Sdk">
   <PropertyGroup>
     <TargetFramework>net8.0</TargetFramework>
     <GenerateDocumentationFile>true</GenerateDocumentationFile>
     <NoWarn>$(NoWarn);CS1591</NoWarn>
     <TreatWarningsAsErrors>true</TreatWarningsAsErrors>
+    <EnablePackageValidation>true</EnablePackageValidation>
+    <PublishRepositoryUrl>true</PublishRepositoryUrl>
+    <RepositoryUrl>https://github.com/your-org/UaiLlmRuntime</RepositoryUrl>
+    <EmbedUntrackedSources>true</EmbedUntrackedSources>
+    <IncludeSymbols>true</IncludeSymbols>
+    <SymbolPackageFormat>snupkg</SymbolPackageFormat>
     <PackageId>Uai.LlmRuntime</PackageId>
     <Version>0.1.0</Version>
     <Authors>Mike</Authors>
     <Company>Customer Engineering</Company>
     <Description>C# LLM runtime core with .uai file memory, canonical inference contracts, adapter-based execution, tokenization, vector math, and automated test coverage.</Description>
@@
   </PropertyGroup>
   <ItemGroup>
     <None Include="../../README.md" Pack="true" PackagePath="" />
   </ItemGroup>
+  <ItemGroup>
+    <!-- Source Link package selection depends on repository host. -->
+    <PackageReference Include="Microsoft.SourceLink.GitHub" PrivateAssets="All" />
+  </ItemGroup>
 </Project>

Core telemetry patch

This repository can add observability without pulling exporters into the core package. Microsoft’s guidance for .NET observability is to instrument via ILogger, Meter, and ActivitySource, with OpenTelemetry collecting and exporting that data as needed.

--- src/Uai.LlmRuntime/Runtime/LlmRuntime.cs
+++ src/Uai.LlmRuntime/Runtime/LlmRuntime.cs
 using System.Runtime.CompilerServices;
+using System.Diagnostics;
+using System.Diagnostics.Metrics;
 using System.Text;
 using Uai.LlmRuntime.Abstractions;
 using Uai.LlmRuntime.Contracts;
 using Uai.LlmRuntime.Memory;

 namespace Uai.LlmRuntime.Runtime;

 public sealed class LlmRuntime : IInferenceRuntime
 {
+    private static readonly ActivitySource ActivitySource = new("Uai.LlmRuntime");
+    private static readonly Meter Meter = new("Uai.LlmRuntime");
+    private static readonly Counter<long> RequestCounter =
+        Meter.CreateCounter<long>("uai.llm.requests");
+    private static readonly Counter<long> FailureCounter =
+        Meter.CreateCounter<long>("uai.llm.failures");
+    private static readonly Histogram<double> RequestDurationMs =
+        Meter.CreateHistogram<double>("uai.llm.request.duration.ms");
+
     private readonly IModelAdapter _adapter;
     private readonly ITokenizer _tokenizer;
     private readonly IUaiMemoryStore? _memoryStore;
     private readonly RuntimeOptions _options;

@@
     public async Task<InferenceResponse> GenerateAsync(InferenceRequest request, CancellationToken cancellationToken = default)
     {
         ArgumentNullException.ThrowIfNull(request);
         ValidateRequest(request);
+        RequestCounter.Add(1);
+        var startedAt = Stopwatch.GetTimestamp();

-        InferenceRequest prepared = await PrepareRequestAsync(request, cancellationToken).ConfigureAwait(false);
-        InferenceResponse response = await _adapter.GenerateAsync(prepared, cancellationToken).ConfigureAwait(false);
-        await PersistRequestAndResponseAsync(request, response, cancellationToken).ConfigureAwait(false);
-        return response;
+        using Activity? activity = ActivitySource.StartActivity("llm.generate", ActivityKind.Internal);
+        activity?.SetTag("uai.model.requested", request.Model);
+        activity?.SetTag("uai.conversation.id", request.ConversationId);
+        activity?.SetTag("uai.use_memory", request.UseMemory);
+
+        try
+        {
+            InferenceRequest prepared = await PrepareRequestAsync(request, cancellationToken).ConfigureAwait(false);
+            InferenceResponse response = await _adapter.GenerateAsync(prepared, cancellationToken).ConfigureAwait(false);
+            await PersistRequestAndResponseAsync(request, response, cancellationToken).ConfigureAwait(false);
+
+            RequestDurationMs.Record(Stopwatch.GetElapsedTime(startedAt).TotalMilliseconds);
+            activity?.SetTag("uai.model.effective", response.Model);
+            activity?.SetTag("uai.provider", response.Provider);
+            activity?.SetTag("uai.input_tokens", response.Usage.InputTokens);
+            activity?.SetTag("uai.output_tokens", response.Usage.OutputTokens);
+            return response;
+        }
+        catch
+        {
+            FailureCounter.Add(1);
+            activity?.SetStatus(ActivityStatusCode.Error);
+            throw;
+        }
     }
 }

.uai memory safety patch

The current memory format is human-readable, but it is fragile because it depends on sentinel delimiters in raw content. A safer shape is to store content in an unambiguous encoding and verify content hash on read.

--- src/Uai.LlmRuntime/Memory/UaiFileMemoryStore.cs
+++ src/Uai.LlmRuntime/Memory/UaiFileMemoryStore.cs
@@
     private static string Serialize(UaiMemoryEntry entry)
     {
+        string normalizedContent = NormalizeNewLines(entry.Content);
+        string encodedContent = Convert.ToBase64String(Encoding.UTF8.GetBytes(normalizedContent));
+
         var builder = new StringBuilder();
         builder.AppendLine(EntryStart);
@@
         builder.AppendLine($"created_utc: {entry.CreatedUtc.ToUniversalTime():O}");
         builder.AppendLine($"content_sha256: {entry.ContentSha256}");
-        builder.AppendLine("content:");
-        builder.AppendLine(ContentStart);
-        builder.AppendLine(NormalizeNewLines(entry.Content));
-        builder.AppendLine(ContentEnd);
+        builder.AppendLine("content_encoding: base64");
+        builder.AppendLine($"content_length: {encodedContent.Length}");
+        builder.AppendLine("content:");
+        builder.AppendLine(ContentStart);
+        builder.AppendLine(encodedContent);
+        builder.AppendLine(ContentEnd);
         builder.AppendLine(EntryEnd);
         builder.AppendLine();
         return builder.ToString();
     }
@@
-            string parsedContent = NormalizeNewLines(content.ToString()).TrimEnd('\n');
+            string rawContent = NormalizeNewLines(content.ToString()).TrimEnd('\n');
+            string parsedContent = string.Equals(Get(fields, "content_encoding"), "base64", StringComparison.OrdinalIgnoreCase)
+                ? Encoding.UTF8.GetString(Convert.FromBase64String(rawContent))
+                : rawContent;
+
+            string expectedHash = NullIfEmpty(Get(fields, "content_sha256")) ?? ComputeContentSha256(parsedContent);
+            string actualHash = ComputeContentSha256(parsedContent);
+            if (!CryptographicOperations.FixedTimeEquals(
+                    Convert.FromHexString(expectedHash.ToUpperInvariant()),
+                    Convert.FromHexString(actualHash.ToUpperInvariant())))
+            {
+                throw new InvalidDataException("A .uai memory entry failed SHA-256 validation.");
+            }
+
             yield return new UaiMemoryEntry
             {
@@
-                ContentSha256 = NullIfEmpty(Get(fields, "content_sha256")) ?? ComputeContentSha256(parsedContent)
+                ContentSha256 = expectedHash
             };
         }
     }

That is still not the final memory-store design, but it closes the easiest corruption and delimiter-collision problems immediately.

Suggested commands

# Restore with lock files and fail if dependency graph drifts
dotnet restore UaiLlmRuntime.sln --locked-mode

# Build as CI would
dotnet build UaiLlmRuntime.sln -c Release

# Run tests with coverage
dotnet test UaiLlmRuntime.sln -c Release --collect:"XPlat Code Coverage"

# Pack and validate package quality
dotnet pack src/Uai.LlmRuntime/Uai.LlmRuntime.csproj -c Release -o artifacts/packages

# Audit vulnerable dependencies
dotnet list UaiLlmRuntime.sln package --vulnerable --include-transitive

The locked restore, lock files, and package-audit recommendations are directly supported by Microsoft’s NuGet documentation.

Version and dependency posture

The repository uses only a handful of test dependencies, which is good. The main problem is not dependency count; it is version drift and build posture.

A subtle but important dependency note: there is no evidence from the current repository that it needs more runtime package dependencies inside the core package yet. If DI helpers, provider adapters, OpenTelemetry exporters, or HTTP stacks are added, they should land in separate packages unless their inclusion is essential to the public runtime core. That keeps faith with the current package’s direction and with the attached architecture guidance.

Roadmap and milestones

The timeline below assumes one experienced .NET engineer with light review support, working part-time to full-time on this repository. Because team size is unspecified, these are planning estimates rather than commitments.

gantt
    title Uai.LlmRuntime proposed roadmap
    dateFormat  YYYY-MM-DD
    axisFormat  %Y-%m-%d

    section Hardening
    Package and dependency hardening      :a1, 2026-06-22, 3d
    CI workflow and CodeQL               :a2, after a1, 3d
    Source Link and package validation   :a3, after a2, 2d

    section Runtime core
    Contract expansion                   :b1, 2026-06-29, 4d
    Tokenizer redesign and budgeting     :b2, after b1, 4d
    Telemetry instrumentation            :b3, after b1, 3d

    section Persistence
    Memory-store safety fixes            :c1, 2026-07-06, 4d
    Memory indexing and compaction       :c2, after c1, 4d

    section Functional adapters
    OpenAI-compatible adapter            :d1, 2026-07-14, 6d
    Hosted-adapter integration tests     :d2, after d1, 3d

    section Advanced backend
    Local worker design spike            :e1, 2026-07-24, 3d
    AOT and isolated-backend prototype   :e2, after e1, 5d

Milestones

Monitoring and alerting additions

The first monitoring pass should focus on signals that explain runtime behavior rather than infrastructure trivia:

Microsoft’s OpenTelemetry guidance supports keeping the instrumentation in standard .NET logging/metrics/tracing primitives, with exporters added by hosts later. GitHub’s CodeQL guidance supports automated scanning in CI for C# repositories.

Open questions and assumptions

Assumptions

Open questions

Final recommendation

If only three improvements are funded immediately, they should be:

1. Package/CI/dependency hardening
2. Contract expansion plus telemetry
3. First real hosted-provider adapter

That sequence turns Uai.LlmRuntime from a clean scaffold into a credible runtime core with a stable growth path toward both enterprise-hosted and future local/native backends, which is exactly the path the accompanying architecture research argues for.