# Architecture ``` ┌──────────────────────────────────────────────────────────┐ │ Your application │ │ backend ──(API key)──► POST /v1/rooms, POST /v1/tokens │ │ client ◄─(peer token)─┘ │ └───────────────┬──────────────────────────┬───────────────┘ │ HTTPS / WSS │ STUN/TURN (UDP/TCP/TLS) ▼ ▼ ┌──────────┐ ┌────────────────────────────┐ ┌─────────────────────────┐ │ nginx │──►│ signaling (Go) │ │ TURN relay │ │ TLS edge │ │ REST API · WS hub · tokens │ │ pion/turn (Go) or │ └──────────┘ │ ICE config · rate limits │ │ coturn (C) │ └──────────┬─────────────────┘ └─────────────────────────┘ │ events (HTTP, signed) ▲ shared TURN_SECRET ▼ │ (time-limited HMAC creds) ┌────────────────────────────┐ │ │ analytics (Python/FastAPI) │────────────┘ │ CDRs · quality · usage │ └────────────────────────────┘ ``` ## Services | Service | Language | Role | |---------|----------|------| | `cmd/signaling` | Go | Multi-tenant REST API, JWT peer tokens, WebSocket signaling hub, TURN credential issuer, static site | | `cmd/turnserver` | Go (pion/turn) | STUN + TURN relay over UDP, TCP, TLS; validates time-limited credentials | | `services/analytics` | Python (FastAPI) | Event ingestion, call detail records, quality samples, usage reports | | `deploy/` | — | Docker Compose, nginx TLS edge, coturn alternative, systemd units, Ubuntu bootstrap script | | `sdk/js` | JavaScript | Browser SDK wrapping `RTCPeerConnection` + the signaling protocol | ## Call flow 1. Backend creates a room and mints a peer token per participant. 2. Client calls `GET /v1/ice` with the token and receives STUN/TURN URLs plus a credential valid for `TURN_CREDENTIAL_TTL`. The credential is `HMAC-SHA1(TURN_SECRET, ":")`, so the relay needs no database. 3. Client opens `wss://…/v1/ws?token=…`. The hub validates the JWT, checks the room exists and has capacity, and sends `welcome` with the current peers. 4. Peers exchange `offer`/`answer`/`ice` through the hub; media flows directly or via TURN. 5. The hub emits `call.started`, `peer.joined`, `peer.left`, `call.ended` to the analytics service and the app's webhook. Clients may push `stats` samples every few seconds. ## Consultation layer `internal/consult.Manager` sits on top of rooms and owns one state machine per consultation plus consultant presence. It hooks into the hub (`OnPeerJoined` / `OnPeerLeft`) so the meter starts only when *both* parties are connected and the session ends the moment either side drops. Timers per consultation: `request` (ring timeout → expired), `connect` (after accept, both must join → ended/connect_timeout), `warn` (low balance → `meter` warning + webhook), `stop` (hard cap → `balance_exhausted`, room closed) and `tick` (meter resync every 15 s). `extend` re-arms `warn`/`stop`; a timer that was replaced while waiting for the lock is ignored. Billing truth lives in the `consultation.ended` event: `billed_seconds` (whole seconds between both-connected and end, capped), `duration_ms`, `reason`, `ended_by`. The platform converts balance ↔ seconds and charges; the server never sees money. Presence is held in memory per node. For multiple API nodes, move presence and the `byRoom` index to Redis and route a consultant's socket and their rooms to the same node, or make the hub Redis-backed. ## Scaling - **Signaling** is stateless except for the in-process room map. To run several nodes, put a Redis pub/sub behind `signaling.Hub` (one channel per room) and move `store.Store` to Postgres. Sticky sessions are *not* required once the hub is Redis-backed. - **TURN** scales horizontally: run relays in several regions and return the closest in `GET /v1/ice` (GeoIP on the client IP). Each relay only needs the shared secret. - **Analytics** is write-light (a few events per call). SQLite is fine to tens of thousands of calls per day; move to Postgres beyond that. - **Group calls above ~6 peers** should move to an SFU (LiveKit / ion-sfu). The token, room and TURN layers stay the same. ## Performance notes - Go signaling handles tens of thousands of concurrent WebSockets per node; the per-peer send queue (256 frames) drops slow consumers instead of blocking the hub. - Relay throughput is bound by kernel UDP performance. For maximum throughput use coturn (`--profile coturn`), tune `net.core.rmem_max`/`wmem_max`, and consider an XDP fast path when a node exceeds roughly 1 Gbps of media. ## Security - API keys: 40 random chars, stored as SHA-256, constant-time compare, rotation endpoint. - Peer tokens: HS256 JWT bound to app + room + peer, issuer-checked, max TTL per app. - TURN credentials expire; the relay refuses private-network peers (coturn `denied-peer-ip`). - Rate limiting per key/IP, CORS allow-list, WebSocket origin check, 64 KiB frame limit, request body limit 1 MiB. - All media is DTLS-SRTP encrypted end to end; the relay cannot decrypt it.