DocsSource

Architecture

This is the deep dive into why tbase is built the way it is — the FUSE choice, the concurrency model, the media plane. For what each path does, see filesystem-reference.md and the other topic pages linked from docs/README.md; this page won't repeat their detail.

1. Why a synthetic filesystem

Every FUSE request carries the caller's uid/gid from the kernel, so the server knows who is reading or writing without trusting paths or pre-chowned per-user directories — that kernel-attested identity is the load-bearing security property of tbase. An earlier version of tbase used real FIFOs on disk and per-user directories; a synthetic filesystem dissolves the problems that came with that:

  • Nodes materialize on lookup() — no pre-created FIFOs, no init step, no restart to add a user or channel.
  • No goroutine parked per open FIFO; only request handlers.
  • Each open() of a stream gets its own bounded subscription buffer, so one slow reader drops only its own messages instead of stalling everyone behind a shared mux.
  • History is persisted, so messages aren't lost when nobody's listening.
  • The server knows open handles, so presence is real rather than a ping/pong convention.
  • Errors map to an errno per request; a bad request fails that request, not the server.

The core is structured as a VFS-agnostic layer (tree + pubsub + store) specifically so the transport is replaceable — a 9P frontend for non-SSH remote access is a natural fit later without touching this layer. The FUSE binding itself uses github.com/hanwen/go-fuse/v2/fs, chosen because it exposes per-request fuse.Caller.

2. Filesystem layout and core semantics

A channel is a directory: the public everyone/ channel, or a private room under private/<names>/. Every channel has the same shape — send, three live stream views, history, members, ctl/, and the file/, voice/, radio/, tv/, cam/ subtrees. The full per-path table is filesystem-reference.md; the semantics that don't fit a table:

  • send: a message's boundary is one open()…close() (not the newline), so a multi-line block written in one open posts as one message with its internal newlines intact — echo/printf/the shell client already open-close per line, so nothing changes for them. Capped at maxMessageBytes; a write that would exceed it is refused rather than truncated. ANSI escapes are stripped at ingest server-side, closing off terminal-escape injection.
  • stream.bot/stream.human/stream.llm: open() registers a subscription, read() blocks until a message arrives. Each is a bounded, drop-oldest ring buffer per handle, with a {"type":"dropped","n":k} marker so a lagging client knows it missed messages — never a shared buffer, so one slow reader can't affect another's feed.
  • history: append-only JSONL per channel, the only thing that touches disk, with a retention cap adjustable per-channel via ctl/retention (details: messaging.md).
  • members/presence: derived from live open handles on the stream files, not a stored roster.
  • Rooms are derived, not created. A room's name is its membership — a sorted, comma-joined set of known users — so it materializes on lookup() for any caller who's a member, with no create step and no registry. private/ lists the rooms that have history on disk, so a room reappears after a restart for free.
  • File transfer is fixed by audience size, not an opt-in mode. A file can't ride the chat pubsub path — pubsub drops the oldest message for a slow reader, but a dropped chunk corrupts a file. So the sender copies bytes live with blocking backpressure instead: a two-person pair is 1:1; an audience of three or more is broadcast to everyone listening now, paced by the slowest, with a stalled receiver dropped past a timeout. Details and event types: file-transfer.md.

3. Three streaming models

One idea — bounded per-consumer buffers, no global lock, never block the producer — shows up in three variations, one per data-loss tolerance:

Chat pubsub File transfer Voice / media plane
Loss model Drop-oldest, never blocks the publisher Lossless, blocking backpressure Drop-oldest, never blocks
Audience Whoever has a stream open Snapshotted at send time Whoever's connected, live
Ordering N/A (independent messages) In-order copy Fixed 20ms clock, N-to-M mixed

Voice needs the combination neither of the other two has alone — file transfer's live fan-out to present participants, plus chat's loss tolerance, plus mixing on a fixed clock — so it runs its own per-channel mixer coordinator (§6) rather than reusing either path directly. Radio, tv, and cam are a fourth variation on the same media plane: single-writer instead of N-to-M, so there's no mixing at all, and because nothing on that path is interactive, they trade voice's low-latency buffer for a much deeper one tuned for robustness instead (profile details: broadcast.md).

4. Access control

Access control is still expressed as Unix owner/group/mode on the synthetic nodes, but enforced by the server per request against fuse.Caller — there's no chown of real files and no service-user gymnastics.

Chat participation is gated by Unix group membership: a caller's uid resolves to a chat name only if that user is a member (primary or supplementary gid) of a configured group (tbase by default), which must exist at mount time. This deliberate enrollment step means a system or service account is never mistaken for a chat participant. Room membership falls entirely out of the channel name — no ACL storage — and lookup() returns ENOENT for a non-member, so a room is invisible, not merely unreadable, to someone outside it. Full mechanism, including the self alias and presence: access-control.md.

5. Concurrency architecture

One goroutine per channel: an actor owns that channel's topic, subscriber set, and history appender, so there's no shared lock on the hot path and a slow consumer can never block another. Publish is a non-blocking send to each subscriber's bounded buffer.

File transfer and voice/broadcast each run on their own per-channel coordinator, deliberately kept outside the chat actor: a long-running file copy, or a fixed-clock mixer tick, must never sit on the goroutine that owns chat. The file-transfer coordinator holds a FIFO of receivers waiting on recv; the voice/broadcast coordinators tick their mix (or, for single-writer broadcast, just forward) every 20ms for voice and radio, every 10ms for tv and cam, and idle-park with zero ticking when nobody's listening.

6. The UDP media plane

Piping raw audio bytes over SSH means TCP, and TCP's in-order delivery means one lost packet stalls every packet behind it — a latency cliff-edge under loss that no application-level drop-oldest can fix, because the loss is hidden below it in the kernel's byte stream. Real-time media wants UDP, so voice, radio, tv, and cam all ride a small UDP media plane alongside the ordinary filesystem, which stays the control plane.

Direct UDP works here with no NAT-traversal problem to solve, because tbase's media is always server-mediated — every participant's only media peer is the server, and the server is already the SSH host every participant logs into. A client's first outbound datagram opens its own NAT pinhole; the server replies to the observed source address, so a client that roams networks mid-call just keeps working. That leaves only authenticating and encrypting the flow, and tbase already has an authenticated, encrypted channel to every participant — the mount itself. This is mosh's architecture (SSH for identity and key exchange, then symmetric-AEAD UDP), and mosh is a decade of precedent that it holds up.

Concretely: opening a dial node runs the ordinary ACL path, then mints a one-shot session ticket ({port, session, key}) and EOFs — the ticket is the entire key exchange, since it only ever travels inside a connection the system already authenticated. The session is the live UDP flow, not the open handle — an earlier design tied session lifetime to the handle staying open, but ssh host cat .../dial allocates no pty, so when the client dies, sshd never signals the orphaned remote cat and the handle never closes. Presence instead tracks the flow: a session starts pending, its first authenticated packet activates it and posts a join event, and a reaper collects any session gone silent past a few seconds (or a client sends one authenticated empty-payload "bye" datagram for a clean exit).

The wire format, ticket handshake, and mixing rules are in voice.md; the single-writer radio/tv/cam profiles built on the same plane are in broadcast.md.

7. Implementation notes and ops

  • Node types. Three FUSE node shapes cover the whole tree: a plain dirNode, a stubNode for static placeholders, and snapshotNode — a finite, cat-able read node that materializes content(caller) once per open and serves it with DIRECT_IO so each open re-renders. history, members, file/members, voice/stats, and every <kind>/<owner>/stats broadcast view (including the caller-resolved self/stats) all fold onto snapshotNode, differing only in mode and renderer. voice/members is the one exception: it's a real directory, since per-listener gain needs per-entry read and write, which a single rendered file can't express.
  • Package layout. The core filesystem logic is the root tbase package. mediawire is the UDP wire-protocol package — one package for voice, radio and video, which differ only in frame size and tick — shared by the server and by the tbase-voice/tbase-cast clients so those clients carry no FUSE dependency and can run wherever a participant's mic, speakers, or camera are — including Windows. mediaclient is the shared client session/loop package used by tbase-voice/tbase-cast: dialing a ticket over SSH, the encrypted UDP session, and the paced uplink/downlink loops, leaving each binary as just its argument grammar and profile table. cmd/ holds the three binaries: tbase-server, tbase-voice, tbase-cast.
  • Kernel caching. Entry and attr TTLs on the mount are kept short (1s) so dynamic nodes — presence, members, room listings — aren't served stale by the kernel; the self alias node goes further, resolving fresh on every request (an effectively zero-length TTL), since a cached dentry for one caller must never be served to another.
  • Ops. A single unprivileged binary: tbase-server mount <dir> (needs user_allow_other in /etc/fuse.conf for -allow-other; see getting-started.md). A crash just leaves a stale mount for a supervisor to remount — history survives on disk, so there's no state to rebuild. Logging is structured (slog); errors map to errno per request rather than panicking, so one bad request fails only that request.

8. Known limitations

  • Voice sessions are per-open, not per-user. Two dial opens from one user are two talkers the mixer sums, under one name in members — multi-device falls out of that for free, but whether a second open should instead evict the first is unresolved.
  • A broadcast owner who loses the single-writer race gets no signal. The media plane has no error datagram, so a client that loses the activate-time race after passing the optimistic Open() check keeps sending into a dead session until the user notices.
  • No codec. Voice and radio are raw PCM; tv/cam are unencoded MPEG-TS from the broadcaster's own ffmpeg. Opus would cost a cgo libopus binding — the one dependency class this design otherwise avoids — so it stays a possible future flag, not a default.
  • Buffer-depth constants for radio/tv/cam are untuned. Voice's own buffer depths were validated against real jitter and loss; the broadcast profiles' deeper buffers are reasoned starting points that haven't had the same treatment.
  • Stereo audio is deferred, not designed away — it roughly doubles bytes/datagram at a fixed sample rate, which forces a real tradeoff (lower sample rate, or a shorter frame) against the no-fragmentation datagram budget every profile currently shares.
  • No downstream decode visibility. stats on any media path reports ingest health (gaps, late frames, underruns) — the one vantage point that can see a specific talker's uplink — but nothing reports whether listeners can actually decode what they're receiving.