Relay

#The wireKeeps 7

A change is a version 2 beside it, with its own attach context and its own suite, never an edit to this one. The suite is e2e/relay/client; a relay that passes it is a relay (./run.sh conformance <url>).

client → GET /<account-id>   Upgrade: websocket; the id is 1–128 of [A-Za-z0-9_-]
relay  → text    "challenge <nonce: 64 hex>"
client → text    "hello <key: 64 hex> <sig: 128 hex> [anything]"
                 sig = Ed25519(key, "your.os relay attach v1\0" ‖ account-id ‖ 0x00 ‖ nonce)
relay  → text    "ok"
either → binary  to(32) ‖ ciphertext   forwarded unchanged to the socket attached as `to`

The wire fixes a frame at 33 B to 64 KiB (a shorter one is dropped) and the hello at 512 B; 64 KiB holds the handshake (1,362 B) and any message the core makes. How many sockets an account may hold and how fast each may send are the relay's own; the reference relay's are in relay/README.md. Text frames after the hello are ignored, except a meet (below). The context string names the version, so an attach signature means nothing anywhere else.

#MeetingKeeps 1

The one addition to version 1 (1.1). It changes nothing a client already did: a text frame after the hello was ignored, and a relay without it still ignores one, so a pairing through it waits until its code expires.

client → text    "meet <mailbox: 32 hex>"   wait for one other socket of this account at this mailbox
relay  → text    "met <key: 64 hex>"        to both, once a second socket meets the same mailbox

Pairing (spec/devices.md) meets at the mailbox its code derives: the device that made the code waits there, the new one meets it, and each learns the other's key; then the pairing's two frames go by to like any others. A wait lives with its socket and nothing is written down, so both devices must be online; it ends when its socket closes, when the socket meets again (one wait per socket) or after ten minutes, a code's life. This is version 2's pairing mailbox cut to what needs no storage: a device that is away can't be paired yet.

#PingKeeps 1

The second addition (1.2). Any socket, attached or not, may ask whether the relay is still there:

client → text    "ping"
relay  → text    "pong"

The reference relay answers with the Durable Object's WebSocket auto-response, so a ping never wakes a hibernated object, isn't captured and doesn't count against the socket's rate. A relay without it ignores the text, and a client that hears nothing redials: the same as a dead socket, so pinging an older relay costs a redial every ping, never a wrong answer.

#Release hintsKeeps 1

The third addition (1.3), relay to client only:

relay  → text    "release <node> <page>"    the latest node release, and page build (0: none)

It comes after ok when the relay knows the release, and again whenever it learns of a later one. A client that doesn't know it ignores the text, as clients already ignore text they don't expect. A node told a node release later than its own checks the manifest at once, by the same rules as its own check: the signature, never older, and only once idle to restart. A page told a page build later than the one it runs shows a quiet "reload for the new version" and leaves the reload to the person: a page can hold an unsaved word or a half-typed code, and a reload a person didn't ask for is how those go.

The reference relay knows the release from its own origin's /dl/manifest, read through the page's binding when it attaches a socket or wakes for a message, and kept for a minute; with no page bound it sends nothing. Ours does the same and is also told: publishing a release posts the signed manifest to the relay, which checks the release key's Ed25519 half (enough to stop anyone else making it shout; nodes check both halves), and, for a later number than it last spread, wakes each account's object once with the hint. That is one Durable Object request per account per release. To know the accounts, ours keeps a list of account ids, added the first time each attaches (What ours sees, below).

A relay can withhold the hint, keeping a node on an old release. So the hint is only ever early: a node also fetches the manifest itself every fifteen minutes, from the static files, which no relay serves.

#A deploy is a blipKeeps 1

The page, the editor, the installer and the releases at /dl/ are their own Worker, on the origin: a file is served from the cache before any script runs, so a page load costs nothing, and only a path with no file behind it (an account's socket, a publish) runs its script, which hands it to the relay's Worker over a service binding. So publishing a release or changing the page deploys that Worker, and every socket stays where it was: deploying a Worker leaves the connections it carries open, and only a change to a Durable Object's code closes the sockets attached to it. The relay's own code changes rarely: what differs between plans is data it reads, not code (the relay split in plan.md).

When the relay does change, every object restarts and every socket closes. A client redials after a random wait of up to two seconds, so a thousand nodes don't arrive at once; the session it had outlives the pipe (D-7), so calls in flight are resent once it's back, rooms rejoin, and a session on a direct pipe doesn't use the relay at all. Cloudflare's gradual deployments can spread the restart over minutes, at the cost of two versions answering at once; the wire is the same in both, so that is safe whenever the wire hasn't changed.

#Version 2Keeps 4

Not frozen yet: version 1 stays as it is beside it (D-18).

Mailboxes. Version 1 drops a frame for a key that isn't attached. A mailbox holds it instead, bounded and expiring: messages for a device that is away, log entries for a device that missed them (log/sync-before-serve), and pairing, whose new node has no key the relay knows (D-11). A device can also leave a push endpoint, so a frame for it wakes a closed browser (D-19).

The relay's key. A relay is a node (D-3), so it gets a key the account's log records, and signs its challenge with it. A new account records the relay it was made at; a pairing code carries the relay's key with its address. TLS stays, for middleboxes and for hiding the name, but nothing trusts it: a proxy with its own certificate authority can open the TLS and still can't pass for the relay.

#Who runs itKeeps 6

The wire is open, a relay you can run is open, and leaving ours is a setting. What we keep closed is running ours: limits by plan, metering, abuse handling, TURN and names on our domain. A direct pipe's server side is small: offers and answers travel as sealed frames through the relay, and STUN only tells a node its own address. Names on our domain need it on the Public Suffix List; one account on yours needs nothing of the kind.

What ours sees, per socket: the account id, the attached key, the address it came from, when it opened and closed, and how many frames of what size it sent; the list of account ids it has seen, for release hints; from version 2, the ciphertext waiting for a device that is away, bounded and expiring, and the push endpoint of a device that asked to be woken. Never a frame past its first 32 bytes.

Where it breaks: a relay you run sees the same, but it is you. A relay you run is as available as where it runs. TURN is a relay too, one that sees ciphertext (D-9).