QxCloudSync
Syncing your devices through a relay that stores and reads nothing.
- 8 min read
- 7 sections
Device-to-device sync (rooms, messages, parameters, friend-room keys) between
two sessions of the same user_id, in deepMerge mode. The server is a
blind, amnesic relay: it stores nothing, reads nothing, logs nothing.
9.1Transport
- Client → server:
{ "op": 60, "d": { "toClientId": "<48 chars|empty=broadcast>", "encrypted": { ...opaque... }, "requestId?": "<≤128 chars>" } } - Server → siblings:
{ "op": 61, "d": { "fromClientId": "...", "toClientId": "...", "encrypted": { ...opaque, forwarded as-is... } } }(wire format unchanged, older clients keep decrypting). - Server → sender (only if
requestIdpresent):{ "op": 60, "d": { "ok": true, "delivered": <n>, "dropped": <n>, "peers": ["<clientId>", ...], "peerCount": <siblings>, "requestId": "..." } }.deliveredcounts the sibling queues that accepted the frame,droppedthose whose queue was full (512) or gone, never silent anymore.peerslists theclientIds that received it,peerCountthe total sibling sessions for thisuser_id. Older clients ignore the extra fields. - Presence directory, client → server:
{ "op": 62, "d": { "requestId?" } }→{ "op": 62, "d": { "ok": true, "self": "<own clientId>", "peers": [{ "clientId": "...", "platform": "web|mobile|desktop" }], "requestId?": "..." } }. Same-user_idsiblings only, sorted byclientId; sessions without aclientIdare omitted (not routable). - Presence events, server → siblings:
{ "op": 63, "d": { "event": "join"|"update"|"leave", "clientId": "...", "platform": "..." } }.joinon identify with a newclientId(re-identifies with an unchanged id stay silent) and onclientIdrotation via op 8;updateon platform-only change;leaveon disconnect. Best-effort fan-out, ignored by older clients.
Handler: relay_cloud_sync_op (websocket/protocol.rs), modelled on
relay_room_signal (op 55→56):
- Rate limit
cloud_sync:session:{sid}120/10s (was 30/10s: a full mesh needs N(N−1)/2 handshakes plus chunked snapshot floods on join, which choked meshes of 3-4+ devices mid-handshake so they never converged). Directorysync_peers:session:{sid}15/10s. Global WS guard 1200/60s still caps sustained abuse. d.encryptedmust be a JSON object, serialized length ≤ 64 KiB (MAX_CLOUD_SYNC_BYTES), else"Payload too large".- Sender must be identified (
user_id+usernamenon-empty;revalidate_sessionalready enforced by dispatch) and carry a non-emptyclientId, else"Missing clientId". Without a return address siblings could only answer by broadcast, turning every handshake into a mesh-wide storm at N ≥ 3. - Fan-out:
playerswith sameuser_id,session_id != self,(toClientId == "" || client_id == toClientId). Collected under the read lock,try_sendafter the lock is dropped; full/closed queues incrementdroppedinstead of vanishing silently (same 512-message queue as op 55/111). - No DB write, no
room_messagespush, no dead-drop, no history. Invisible devices are included (unlike room broadcasts). - Stale routing is now visible: a
toClientIdthat no longer maps to a connected session yieldsdelivered: 0withpeerCountshowing the live mesh, so the sender falls back to broadcast and refreshes via op 62 immediately instead of stalling until the 3-minute client-side fallback (§9.3). A broadcast withpeerCount: 0is the normal single-device case and needs no retry.
9.1.1Client healing contract (mesh ≥ 3)
- Unicast frame with
delivered == 0→ stale route: resend once with emptytoClientId, thenop: 62to relearn the mesh. Do not spin:dropped > 0means congested queues, retry that leg with backoff. - On
op: 63joinfrom an unknownclientId→ open a handshake to it; onleave→ mark that leg'speerWsstale at once (no more pushing into the void); onupdate→ refresh the Settings → Sync icon. - On boot (or after reconnect), fetch
op: 62: if the mesh is non-empty and no session exists for a listed peer, handshake it directly, do not wait for its hello. This is what keeps 4-8-device meshes fully interconnected without the manual Pair button.
9.2Trust root and key schedule (client-only)
masterSecretfrom the 12 recovery words (PBKDF2-SHA256 100k, saltqxphantom:master→ HKDFqxp-master), never transmitted.syncRoot = HKDF(master, "", "qxcloudsync:root:v1"),syncAuth = HKDF(syncRoot, "", "qxcloudsync:auth:v1")(HMAC key for hellos).- Handshake (all inside
d.encrypted, signed):hello(eph P-256 + ephemeral ML-KEM-768 pk) →accept(+ ML-KEM ct to initiator) →confirm(+ ML-KEM ct to responder). Each hello carriesauth = HMAC(syncAuth, canonical)plus a hybrid signature: device ECDSA P-256 and SLH-DSA-SHA2-128f (FIPS 205, 17088-byte signatures,crypto/slhdsa.ts). Both must verify (fail closed); a wrong HMAC or a missing/invalid PQ signature is silently dropped. Each device holds a long-term SLH-DSA identity keypair (qxcloudsync-device-v1). - Post-quantum posture: KEX = ECDH + 2× ML-KEM-768 (FIPS 203), safe via the ML-KEM component; identity = ECDSA + SLH-DSA (FIPS 205); session data = AES-256-GCM (PQ-safe symmetric, Grover halves 256-bit to a comfortable 128-bit). PQ signatures are deliberately handshake-only: a 17 KiB signature on every data part would eat the 64 KiB relay budget and CPU per push, while AES-GCM already gives PQ authenticity per part.
syncMaster = HKDF(ecdh || ss1 || ss2 || syncRoot, transcript, "qxcloudsync:master:v1"), RAM-only. Ephemeral private keys wiped after use. Sessions are additionally persisted client-side, AES-GCM encrypted underHKDF(syncRoot, "", "qxcloudsync:persist:v1"), so a browser restart does not force a re-pair (the server still stores nothing).- Topology is a full mesh of pairwise sessions with no transitive trust and no forwarding: A paired with B and C does not make B and C know each other, and A never relays between them. Every pair needs its own direct handshake (hello → accept → confirm → dedicated master/epoch). N devices = N(N−1)/2 handshakes. Version vectors still gossip-merge, so data converges once the pairwise legs exist.
- Pairing: with sync enabled and the 12 words present, a client with zero
sessions broadcasts a signed hello on boot (jittered 2–6 s); any holder of
the same words verifies the HMAC and answers. A client that already holds
sessions also handshakes proactively: any peer learned via the op 62
directory or an op 63
joinevent without a live session gets a direct hello (see §9.1.1), the manual Pair button and re-handshakes (§9.3) remain as fallback. Hello broadcast, accept/confirm/data unicast viatoClientId. Duplicate hellos are ignored while the existing session shows recent verified inbound traffic, but answered when that session is stale (reinstall / divergence recovery), the completed handshake then replaces the old session and its secrets are wiped. - Hellos carry a signed normalized
platform(mobile|web|desktop) shown in Settings → Sync with one icon per type. - Epoch keys:
syncEpochKey_k = HKDF(syncMaster, BE64(k), "qxcloudsync:epoch:v1"), TTL 7 days, auto-rotation at TTL − 10% via a signedrekeyhello (no new ECDH). Old epoch key dropped. - Data envelopes: AES-256-GCM under the epoch key (AAD
syncId:epoch:n:from:to) + device ECDSA signature. Anti-replay on(syncId, epoch, n). roomKeynever travels raw:AES-GCM(wrapKey, roomKey)wherewrapKey = HKDF(epochKey, "", "qxcloudsync:roomkey-wrap:v1").
9.3Mesh self-healing (client-only)
peerWs (the peer's WS client_id used for unicast) goes stale on every
reconnect, tab reload, or phone wake. The relay now reports this instantly
(delivered: 0 on the op 60 ack, plus op 63 leave/join events and the
op 62 directory, see §9.1.1); the mechanisms below remain as the safety net
for older servers and for frames sent without requestId. A rekey lost in
a stale route still splits epochs (both sides then drop everything: the
sender already advanced, the receiver's own rekey is rejected as
epoch != sess.epoch + 1), hence the epoch healing below.
lastInboundAtper session: last verified inbound traffic only. (lastSeenis also bumped on send, so it can never detect a one-way black hole: a sender pushing into the void would look "fresh".)- Stale route → broadcast fallback: past 3 min without inbound, pushes,
rekeys and revokes go out with empty
toClientId(server fans out to all same-user_idsessions). Reception refreshespeerWsviafromClientIdand the ack heals the way back, one round trip restores the leg. - Epoch healing on receive, only for envelopes addressed to us (
outer.tomust equal our device id; third-party broadcast fallbacks are ignored): missed rekey → bounded local fast-forward (≤ 10 epochs; the subsequentopenDataauthenticates, so only the master holder can trigger adoption); peer behind → resend of the current signed rekey; unknownsyncId→ fresh handshake. Re-handshakes and rekey resends are rate-limited to 1/min/peer. - Epoch is never advanced without a successful rekey send
(
sendRekeyMessage, shared by the rotation scheduler and the resend path).
9.4deepMerge rules
Every object carries { updatedAt, by }; rooms/messages carry ids.
Version vectors { deviceId: counter } select deltas; merge = union + LWW
(tie-break: lexicographically greater by wins). Ratchets merge by max.
Trusted sender keys merge by union; a divergent JWK keeps the local one and
raises an error (no silent overwrite). Room-key conflict policy: refuse +
manual choice, the local key is kept, the conflict is surfaced in
Settings → Sync, the user picks Keep local / Request remote re-push.
Large snapshots are split into valid sub-snapshots (rooms/params chunk +
message batches of ~100/room) so each relay frame stays ≤ 64 KiB. Full history
lives in client IndexedDB (qxcloudsync-v1), beyond the 500/room
localStorage cap. Sync pauses under client-lock, RAM-only OPSEC, or decoy.
Room deletion (op 57/58) travels as a deleted tombstone collection (30-day
TTL): the deleted room is dropped locally (lists, messages, keys, pins,
IndexedDB) and never re-imported from stale snapshots while the tombstone
lives. Pins (pinnedRooms, ≤5) sync LWW; room members are attached for
newly imported rooms only, so live rosters are never clobbered.
9.5Event propagation
Snapshots are full-state and idempotent, but they are not only periodic:
persist() itself notifies subscribers (internal mutation calls included),
coalesced into one push 2.5 s after the last change. Out-of-band stores
(custom theme, locale) are observed with synchronous watchers. Applying a
remote snapshot never re-notifies (internal guard), so there is no echo loop.
The 90 s timer remains as a safety net.
9.6Security properties (audited)
- Quantum-proof (hybrid): KEX = ECDH P-256 + 2× ML-KEM-768 (FIPS 203), safe via the ML-KEM component; identity = ECDSA P-256 + SLH-DSA-SHA2-128f (FIPS 205), both required; session data = AES-256-GCM (≈128-bit PQ margin).
- Transcript integrity: the KDF transcript is hashed over canonical JSON
(sorted keys), never
JSON.stringify, the server re-serializes in sorted order, so a naive hash would diverge per side. - Anti-replay: per-peer
(syncId, epoch)window with monotonic high-water (tolerance 5000 for reordering) plus a bounded dedup set (6000 entries, pruned). Survives restarts via the persistedsendN; LWW merge makes residual replays harmless. - At rest: without client lock, session blobs are AES-GCM under a syncRoot-derived key (device boundary, same as the stored recovery words). With client lock active, sessions and the SLH-DSA identity rest only under AES-GCM envelopes keyed by the lock key; locking wipes all RAM secrets (masters, epoch keys, syncRoot, SLH cache) and never downgrades envelopes.
- Residual (accepted): hello replay is a bounded nuisance (server rate 30/10 s, 5 recent pendings max, 120 s sweep); the server observes timing, frame counts and approximate sizes (no padding to fixed buckets, unlike PHANTOM); PBKDF2-100k for the master follows the pre-existing PHANTOM parameters.
