RemoteEndpoints.com Design and Implementation Report

Full report

On this page

Executive Summary

RemoteEndpoints.com should be designed as a split-plane system: a cloud control plane for identity, policy, approval, signaling, and audit, plus an end-to-end media plane between the browser and the LocalEndpoint Connect App using WebRTC whenever possible. That architecture best matches the requirements for remote control and remote desktop viewing because WebRTC already standardizes secure real-time media and data transport, NAT traversal through ICE/STUN/TURN, and browser interoperability, while OAuth 2.0 / OpenID Connect provide a mature basis for user identity and delegated authorization. TLS 1.3 protects control-plane traffic, and DTLS-SRTP protects media traffic; signaling must still be protected independently because WebRTC does not make signaling secure by itself.

The strongest v1 design is an attended-access model in which every remote session requires live local approval on the Connect App before the session can proceed. Practically, that means separating device pairing from remote-session approval. Pairing binds a device to an account and registers a device-held keypair. Each remote session then requires a just-in-time approval ceremony on the local device, with a one-time session intent that expires quickly and cannot be replayed. This directly reduces the risk of unauthorized access, token replay, and stale delegated access. For web users, use OIDC login with PKCE and step-up MFA such as WebAuthn for sensitive actions; for device-to-cloud communication, use mutual TLS and certificate-bound or otherwise sender-constrained tokens rather than bearer-only trust.

For the Python stack, the best fit is typically FastAPI/Starlette for the public API, admin API, signaling endpoints, and WebSocket interfaces; gRPC Python for long-lived outbound control streams between the Connect App and the cloud edge; PostgreSQL for durable state; Redis Streams for durable event fan-out and reconnectable session orchestration; Authlib for OAuth/OIDC integration; SQLAlchemy + Alembic for persistence; pyca/cryptography and optionally PyNaCl for application-level key handling and signatures; and OpenTelemetry for correlated traces, metrics, and logs. Django plus Channels remains a viable alternative if the application is admin-heavy and benefits from Django’s ecosystem, but for a real-time, async, signaling-heavy system, FastAPI/Starlette is the cleaner default.

Assumptions and Design Goals

Several important parameters were left open-ended, so this report makes explicit design assumptions. The recommendations below assume a multi-tenant SaaS with browsers as the main remote operator clients, a cross-platform LocalEndpoint Connect App installed on a controlled desktop or mobile device, and attended sessions only in v1 because the requirement says local approval is mandatory before remote connection. The design also assumes that user and device counts may grow into the tens of thousands or higher, that concurrency may eventually reach the low thousands of simultaneous sessions, and that the product should remain viable under mixed network conditions, including highly restrictive enterprise NATs and firewalls.

The primary design goals should therefore be:

1. No inbound listening port required on the LocalEndpoint device in normal operation; connectivity should begin with outbound connections from the device toward the service edge.
2. Mandatory local approval per remote session, with explicit approval state captured and auditable.
3. End-to-end encrypted media and control data between browser and Connect App whenever feasible, even when relayed through TURN.
4. Immediate revocation semantics for approvals and sessions.
5. Small, composable trust boundaries, so that the signaling service, TURN relays, and web tier do not need access to decrypted desktop frames.
6. Operationally scalable regional architecture, because relay traffic and signaling latency are geographically sensitive. These goals are consistent with the capabilities and threat model of WebRTC, ICE/STUN/TURN, OAuth/OIDC, and zero-trust identity-centered architectures.

A further assumption is that the Python web application is the control plane, not necessarily the optimal implementation language for every byte of the media path. Python is well suited to API orchestration, signaling, policy, and administration, and it can prototype or even operate light WebRTC server logic through aiortc, but a production desktop/mobile Connect App will often benefit from platform-native capture and hardware encoding pipelines. The report therefore treats Python as the anchor of the cloud platform and orchestration layer, while allowing the endpoint agent to use native platform facilities when needed. aiortc remains valuable for proofs of concept, integration tests, and specialized media gateways.

Recommended Architecture

The recommended architecture is a three-zone system: a browser-facing web tier, a cloud control plane, and a separate connectivity/media edge. The browser interacts with the web tier over HTTPS and WebSocket. The Connect App maintains an outbound authenticated stream to the control plane, ideally over gRPC on HTTP/2 with mTLS. When a remote session is requested, the control plane validates identity and policy, obtains local approval from the device, and then brokers WebRTC signaling so the browser and Connect App can establish a peer session directly or through TURN. This keeps the heavy media path off the Python application servers while still allowing the Python system to remain authoritative for identity, approval, and audit. gRPC’s long-lived multiplexed streams and authentication hooks are a good fit for device control channels, while WebSocket remains the pragmatic browser-facing signaling mechanism.

flowchart LR
    subgraph BrowserSide["Remote operator"]
        B["Browser UI"]
    end

    subgraph Cloud["RemoteEndpoints.com control plane"]
        FE["FastAPI / Starlette web tier"]
        AUTH["OIDC / session service"]
        SESS["Session broker & policy engine"]
        SIG["Signaling service"]
        DEV["Device registry"]
        AUD["Audit / telemetry pipeline"]
        PG[("PostgreSQL")]
        REDIS[("Redis Streams")]
    end

    subgraph Edge["Connectivity edge"]
        TURN["Regional STUN/TURN cluster"]
    end

    subgraph Endpoint["LocalEndpoint Connect App"]
        AGENT["Connect App"]
        CAP["Screen capture / encoder"]
        CTRL["Input injector / local approval UI"]
    end

    B -->|"HTTPS + secure session"| FE
    FE --> AUTH
    FE --> SESS
    FE --> SIG
    SESS --> DEV
    SESS --> PG
    SESS --> REDIS
    SIG --> REDIS
    FE --> AUD
    SESS --> AUD
    DEV --> PG

    AGENT -->|"Outbound gRPC stream over mTLS"| SESS
    AGENT -->|"Signaling via broker"| SIG
    AGENT --> TURN
    B --> TURN

    B -. "WebRTC media + data channel\nP2P when possible, TURN fallback" .-> AGENT
    AGENT --> CAP
    AGENT --> CTRL

This architecture should be implemented with a durable core and an ephemeral edge. Durable entities include users, devices, pairings, policy assignments, approvals, and audit records in PostgreSQL. Ephemeral entities include session intents, signaling state, candidate exchange, approval waits, and reconnect queues in Redis Streams rather than plain Redis Pub/Sub. That distinction matters because Redis Pub/Sub is explicitly at-most-once and messages can be lost during subscriber disconnects, while Redis Streams provide an append-only event log and consumer-group semantics that are better suited to reconnecting sessions and delivery recovery.

For the Python application framework choice, FastAPI/Starlette is the default recommendation because the control plane is highly asynchronous and connection-oriented. FastAPI is designed for modern typed async APIs, and Starlette directly supports WebSockets. Django + Channels is credible if the product’s operator experience, admin tooling, and back-office workflows dominate, but it introduces more framework surface area than is necessary for a signaling- and session-broker-heavy service. A pragmatic compromise is to use FastAPI for the runtime surface and keep administrative tools separate if needed.

An illustrative REST/WebSocket surface could look like this:

POST   /v1/devices/link/start
POST   /v1/devices/link/complete
POST   /v1/sessions
GET    /v1/sessions/{session_id}
POST   /v1/sessions/{session_id}/cancel
POST   /v1/sessions/{session_id}/approve-callback
POST   /v1/sessions/{session_id}/turn-credentials
WS     /v1/signaling/{session_id}
GET    /v1/audit/events
GET    /v1/devices
POST   /v1/devices/{device_id}/revoke

And the device-control stream can be modeled as a gRPC service:

service DeviceControl {
  rpc Connect(stream DeviceEnvelope) returns (stream ControlEnvelope);
}

message DeviceEnvelope {
  string device_id = 1;
  oneof body {
    DeviceHello hello = 2;
    ApprovalResponse approval = 3;
    IceCandidate candidate = 4;
    SessionState state = 5;
    TelemetryEvent telemetry = 6;
  }
}

message ControlEnvelope {
  string session_id = 1;
  oneof body {
    SessionIntent intent = 2;
    ApprovalRequest approval = 3;
    SdpOffer offer = 4;
    IceCandidate candidate = 5;
    SessionCommand command = 6;
  }
}

Identity, Approval, and Session Security

RemoteEndpoints.com needs three distinct identity domains rather than one overloaded “login” concept: human users, registered devices, and running workloads/services. Human users should authenticate through OpenID Connect on top of OAuth 2.0, using the authorization code flow with PKCE. That remains the most interoperable and security-reviewed model for web identity. Sensitive actions such as starting a remote-control session, granting elevated permissions, or enrolling a new device should require step-up authentication, ideally with WebAuthn-backed phishing-resistant MFA. For native apps, OAuth guidance recommends using an external user-agent, primarily the browser, rather than embedding credentials directly in the app.

The device enrollment flow should be a paired ceremony:

1. The Connect App generates a device keypair locally.
2. It requests a short-lived link challenge from the control plane.
3. The user authenticates in the browser and binds the device to an account.
4. The Connect App requires local, device-side confirmation before finalizing registration.
5. The control plane stores the device public key and issues a device credential, preferably a certificate or equivalent sender-bound identity.

That design uses the same spirit as the OAuth device authorization grant for cross-device approval, while preserving the product requirement that local approval be required before remote access is possible.

The most important authorization decision is to treat session approval as distinct from account authorization. An authenticated user with the right account role should still not be able to connect until the local device approves the specific session request. In practice, this means a remote session request should yield a one-time session intent, not an immediately active session. That intent should contain at least: session_id, actor_id, device_id, requested capabilities, issuance time, expiration time, a unique jti, and a nonce/challenge to prevent replay. The token can be JWT-based if offline validation materially helps scale, but for the most sensitive approval artifacts, an opaque reference token with introspection is often operationally safer because it enables immediate server-side revocation. JWT best-practice guidance warns against weak validation and inappropriate trust expansion, and OAuth introspection is specifically designed for protected resources to determine whether a token is active.

For device-to-cloud trust, bearer-only tokens are not strong enough. Bearer tokens can be replayed by any party that obtains them, which is why the OAuth bearer-token specification stresses protecting them from disclosure. Remote endpoint software is higher risk than a normal web client because compromise of the device channel creates a path to desktop access. The control plane should therefore require mutual TLS from the Connect App and, ideally, certificate-bound access tokens as described in RFC 8705. A lighter alternative is DPoP-style sender-constrained tokens, but mTLS is the stronger default for managed device software because it is enforced at the transport boundary rather than only at the application layer.

A recommended token lifecycle is:

• Browser session: secure, httpOnly, same-site session cookie via a backend-for-frontend pattern; short idle lifetime; step-up re-authentication before sensitive actions.
• Device bootstrap credential: one-time link artifact, expiry in minutes, unusable after completion.
• Device certificate: short-lived and automatically rotated; device private key never leaves the device.
• Device control-plane access token: very short-lived, sender-constrained, scoped to device control.
• Session intent: one-time-use, expires within about 60–120 seconds, bound to user, device, and requested capabilities.
• TURN credentials: ephemeral and per session, not tenant-global. Coturn supports a TURN REST style of authorization using a shared auth secret, which is well-suited to minting short-lived relay credentials.

The approval flow should look like this:

sequenceDiagram
    autonumber
    participant U as Remote user in browser
    participant W as RemoteEndpoints web app
    participant A as Auth service
    participant S as Session broker
    participant D as LocalEndpoint Connect App

    U->>W: Request remote session
    W->>A: Validate web session / step-up auth
    A-->>W: User identity + assurance level
    W->>S: Create session intent
    S-->>W: Pending session_id + one-time intent
    S->>D: ApprovalRequest(session_id, actor, capabilities, nonce)
    D->>D: Show local consent UI
    D-->>S: Approve or deny + signed response
    alt approved
        S-->>W: Session approved, issue signaling tickets
        W-->>U: Proceed to signaling
    else denied
        S-->>W: Session denied
        W-->>U: Access denied
    end

An illustrative Python-style approval handler:

from datetime import datetime, timedelta, timezone
import secrets

def build_session_intent(actor_id: str, device_id: str, capabilities: list[str]) -> dict:
    now = datetime.now(timezone.utc)
    return {
        "session_id": secrets.token_urlsafe(18),
        "jti": secrets.token_urlsafe(18),
        "actor_id": actor_id,
        "device_id": device_id,
        "capabilities": capabilities,
        "issued_at": now.isoformat(),
        "expires_at": (now + timedelta(seconds=90)).isoformat(),
        "nonce": secrets.token_urlsafe(24),
        "state": "pending_local_approval",
    }

async def start_remote_session(actor, device, capabilities, db, broker):
    if not actor.can("session.request", device_id=device.id):
        raise PermissionError("Not authorized for this device")

    # Strong recommendation: require recent MFA/WebAuthn for control sessions.
    if not actor.has_recent_step_up_auth:
        raise PermissionError("Step-up authentication required")

    intent = build_session_intent(actor.id, device.id, capabilities)
    await db.save_session_intent(intent)              # durable UTC timestamps
    await broker.publish_device_approval(device.id, intent)
    return {"session_id": intent["session_id"], "status": "awaiting_local_approval"}

Connectivity and Streaming

For browser-based remote viewing and control, the default transport should be WebRTC, not a custom reverse-tunnel protocol and not a browserified VNC gateway. The technical reason is straightforward: WebRTC already standardizes peer connection establishment, NAT traversal through ICE, secure media transport with DTLS-SRTP, and secure bidirectional data channels using SCTP over DTLS. That combination cleanly maps to remote desktop requirements: video frames as media, keyboard/mouse/control messages as data-channel traffic, and standards-based fallback to relays when direct connectivity fails. RFB/VNC remains historically important and is still useful as a reference protocol, but it is framebuffer-centric rather than browser-native, and it does not natively deliver the same modern browser transport and NAT traversal stack.

A practical session-establishment sequence is:

sequenceDiagram
    participant B as Browser
    participant SIG as Signaling service
    participant D as Connect App
    participant TURN as STUN/TURN

    B->>SIG: WebSocket connect with signaling ticket
    D->>SIG: gRPC/WebSocket signaling attach with signaling ticket
    B->>SIG: SDP offer
    SIG->>D: Forward SDP offer
    D->>SIG: SDP answer
    SIG->>B: Forward SDP answer
    B->>TURN: Gather ICE candidates
    D->>TURN: Gather ICE candidates
    B->>SIG: Trickle ICE candidates
    D->>SIG: Trickle ICE candidates
    Note over B,D: ICE connectivity checks choose P2P path if possible
    Note over B,D: Else TURN relay path selected
    Note over B,D: DTLS handshake establishes SRTP/DataChannel keys
    Note over B,D: Video stream + control data channel become active

That sequence depends on Trickle ICE, which allows candidates to be exchanged incrementally instead of waiting for complete candidate gathering, reducing setup latency. Once a candidate pair succeeds, the peers complete the DTLS handshake and derive media/data-channel protection keys. TURN is a relay fallback, not a security terminator; it forwards packets but does not need access to the decrypted desktop stream.

For remote-control input, use two WebRTC data channels rather than one monolithic control stream. A reliable, ordered channel is appropriate for keystrokes, clipboard commands, and modal actions such as “request control” or “release control.” A lower-latency unordered/partially reliable channel can be used for high-rate mouse movement and pointer hover deltas so that stale motion updates do not create head-of-line blocking. WebRTC data channels are defined specifically to carry generic peer-to-peer data over SCTP, and SCTP interleaving and related extensions were added to reduce blocking behavior important to WebRTC workloads.

For NAT traversal, the platform should prefer this order:

1. Direct ICE-selected path using host and server-reflexive candidates discovered with STUN.
2. TURN relay when direct connectivity fails or policy forces relay.
3. Reverse control tunnel only as an application fallback for signaling and emergency control-plane reachability, not as the default media path.

The reason to keep reverse tunnels as a tertiary fallback is operational and security-related: once you run interactive video and control through a custom application tunnel, you centralize bandwidth, increase Python/backend involvement in the hot path, and create more proprietary protocol surface to secure. TURN already exists to solve “no direct path” conditions in a standardized way, and coturn gives a mature open-source implementation.

The streaming side of the Connect App can be prototyped in Python with aiortc, even if production endpoints later move to more native implementations:

from aiortc import RTCPeerConnection, RTCSessionDescription, RTCIceServer, RTCConfiguration
from aiortc.contrib.media import MediaRelay

TURN_URLS = ["turn:turn.remoteendpoints.example?transport=udp",
             "turns:turn.remoteendpoints.example?transport=tcp"]

async def prepare_peer(turn_username: str, turn_password: str):
    config = RTCConfiguration(iceServers=[
        RTCIceServer(urls=["stun:stun.remoteendpoints.example"]),
        RTCIceServer(urls=TURN_URLS, username=turn_username, credential=turn_password),
    ])
    pc = RTCPeerConnection(configuration=config)

    # Ordered / reliable channel for keyboard and commands.
    reliable = pc.createDataChannel("control", ordered=True)

    # Unordered / limited retransmits for mouse movement.
    fast_mouse = pc.createDataChannel("pointer", ordered=False, maxRetransmits=0)

    # screen_track would wrap a native capture/encoder pipeline.
    screen_track = get_screen_video_track()
    pc.addTrack(screen_track)

    return pc, reliable, fast_mouse

async def accept_offer(pc, offer_sdp: str) -> str:
    await pc.setRemoteDescription(RTCSessionDescription(sdp=offer_sdp, type="offer"))
    answer = await pc.createAnswer()
    await pc.setLocalDescription(answer)
    return pc.localDescription.sdp

aiortc supports the relevant ICE server configuration and browser-like WebRTC programming model, which makes it suitable for prototypes, integration tests, and low-volume specialty use cases. For production desktop capture, however, the decisive performance factor is usually the local capture/encode path rather than the signaling logic itself.

Design Option Comparison

The table below compares the most consequential architectural choices for RemoteEndpoints.com. The recommendations favor standards-based interoperability, sender-constrained identities, and keeping decrypted desktop media out of the cloud control plane. The trade-offs are synthesized from the cited standards and official documentation, not copied from a single source.

Threat Model and Operational Controls

The core threat model should assume that any one of the following could be hostile or compromised: the browser session, the network path, the signaling service, a TURN relay, a stolen access token, a malicious insider attempting unauthorized control, or an infected endpoint. WebRTC’s own security architecture explicitly treats the system as a composition of signaling, transport, browser/device code, and intermediaries; it does not assume the signaling layer is trusted. The correct response for RemoteEndpoints.com is therefore to verify each transition independently: user identity, device identity, session approval, signaling authenticity, media-channel establishment, and action logging. That is also the essence of zero trust: do not trust network location; evaluate users, devices, and workloads continuously and near the resource.

A robust logging model should treat audit records as first-class security objects. Every device registration, certificate rotation, remote session request, local approval prompt, approval decision, signaling attachment, ICE completion, TURN allocation issuance, session start, session stop, and forced revocation should be logged with UTC timestamps, actor/device identifiers, cryptographic key IDs or certificate thumbprints where relevant, and correlation IDs that also appear in traces and metrics. OpenTelemetry is useful here because it standardizes cross-signal correlation between logs, traces, and metrics. OWASP guidance is equally important: high-value logs often contain sensitive context and must be protected against leakage and misuse.

From a cryptographic lifecycle perspective, the service should use envelope encryption or a dedicated key service for stored secrets and audit payloads, not ad hoc application-managed raw keys. NIST key-management guidance and commercial/off-the-shelf KMS and Vault systems all converge on the same principle: long-lived root or wrapping keys should be centrally controlled, while short-lived working keys are used for actual data protection. For a self-hosted path, Vault Transit is a strong option; in cloud-native deployments, a managed KMS is simpler operationally.

Deployment, Scalability, and CI/CD

The control plane should be deployed as stateless horizontally scalable services behind regional load balancers, with state externalized to PostgreSQL, Redis, and a telemetry backend. Kubernetes is a natural fit because Horizontal Pod Autoscaling is built to scale workloads such as Deployments and StatefulSets based on resource or custom metrics. The edge-sensitive components—especially TURN—should be deployed regionally close to users because relay latency directly affects interactive control quality. The web application, session broker, and signaling service can remain in fewer regions than TURN in early stages, but TURN should be geographically distributed earlier than most control-plane services.

A scalable production topology would normally include:

• a browser SPA or server-rendered web tier;
• an API/BFF tier for authenticated sessions;
• a signaling/session broker tier;
• PostgreSQL HA for durable state;
• Redis Streams for live orchestration;
• regional coturn clusters;
• centralized telemetry with OpenTelemetry collectors;
• object storage or a security lake for audit export.

Where multi-region growth matters, PostgreSQL can remain the source of truth and Redis remain regional/ephemeral. Cross-region active-active for everything is not necessary on day one and can easily complicate revocation semantics. Scale the stateless parts first; shard or partition by tenant or region only once there is evidence of that need. PostgreSQL’s own logical replication capabilities can support selective scale-out and reporting patterns later.

For CI/CD, the recommended controls are straightforward and modern:

• GitHub Actions OIDC to obtain cloud credentials without long-lived CI secrets.
• Trivy for image, dependency, filesystem, and IaC scanning.
• SLSA provenance so build outputs carry verifiable metadata about how they were produced.
• Cosign to sign container images and other artifacts before deployment.

Those choices materially improve the provenance and trust story for a remote-access product, where a compromised build pipeline would be especially dangerous. GitHub’s OIDC support exists specifically to replace stored cloud secrets with federated identity. SLSA focuses on provenance authenticity and completeness. Cosign provides artifact signing and verification, and Trivy is widely used for security scanning across container and IaC workflows.

A practical deployment pipeline is:

commit
  -> unit tests + mypy + lint
  -> dependency / SBOM generation
  -> Trivy scan of workspace and image
  -> build container image
  -> generate SLSA provenance
  -> sign image with Cosign
  -> deploy via GitHub Actions OIDC to cloud / cluster
  -> progressive rollout + synthetic WebRTC session test
  -> post-deploy verification and audit event

Finally, the platform should continuously test the whole session path, not just APIs: browser auth, session request, local approval, signaling, ICE establishment, TURN fallback, video path, data channel control path, and teardown. A remote-control system fails in practice at the boundaries between these layers, not within the neat confines of one microservice. That is why the architecture should favor standards where possible and keep custom protocol logic narrow, observable, and revocable.