# Webhook delivery Every application with a `webhook_url` receives lifecycle events as JSON POSTs (`call.started`, `peer.joined`, `peer.left`, `call.ended`, `stats.reported`, `usage.quota_reached`, `consultant.status`, `consultation.*`, plus `webhook.test` from the portal). Deliveries are **signed per app**, **retried with backoff**, **logged** (last 1000 per app) and can be **replayed** from the portal or the API. Payload (unchanged from the base API): ```json { "type": "peer.joined", "app_id": "app_…", "mode": "live", "room_id": "room_…", "call_id": "call_…", "peer_id": "user-42", "timestamp": "2026-10-27T10:00:01Z", "data": { "name": "Alice", "scope": "peer" } } ``` ## Headers | Header | Value | |--------|-------| | `X-Webhook-Id` | Delivery id (`whd_…`). **Stable across retries and replays** of the same event. Use it for idempotency. | | `X-Webhook-Timestamp` | Unix seconds when this attempt was sent. Covered by the signature. | | `X-Webhook-Signature` | `v1=` | | `X-Webhook-Event` | Event type, e.g. `call.ended`. | | `X-Webhook-Attempt` | 1-based attempt number for this delivery id (retries and replays increment it). | | `X-Webhook-Replay` | `true` only when an operator replayed the delivery. | | `Content-Type` | `application/json` | | `User-Agent` | `callingserver/1.0` | Each attempt is signed afresh (new timestamp, new signature) over the **same body bytes**. Each attempt has a 5 s timeout. ### Signing secret Each app has one secret (32 random bytes, hex). It is generated automatically on the first delivery, or explicitly with **Rotate secret**, which returns it **once**. The portal and `GET …/webhook` only ever show the first 8 characters. Rotation takes effect on the next attempt; update your backend before rotating. The secret is never included in app objects returned by the API. Analytics forwarding (`X-Analytics-Key` / `X-Signature` to the analytics service) is a separate path and unchanged. ## Verifying a delivery Always compute the HMAC over the **raw request body** (before any JSON parsing or re-serialisation), compare in constant time, and reject timestamps older than a few minutes to limit replay of captured requests. **Node (Express)** ```js import crypto from 'node:crypto'; import express from 'express'; const SECRET = process.env.CALLGRID_WEBHOOK_SECRET; const app = express(); app.post('/webhooks/callgrid', express.raw({ type: '*/*' }), (req, res) => { const ts = req.get('X-Webhook-Timestamp'); const sig = req.get('X-Webhook-Signature') || ''; if (Math.abs(Date.now() / 1000 - Number(ts)) > 300) return res.status(400).end('stale'); const expected = 'v1=' + crypto.createHmac('sha256', SECRET).update(ts + '.').update(req.body).digest('hex'); const a = Buffer.from(expected), b = Buffer.from(sig); if (a.length !== b.length || !crypto.timingSafeEqual(a, b)) return res.status(401).end('bad signature'); const id = req.get('X-Webhook-Id'); if (alreadyProcessed(id)) return res.status(200).end(); handle(JSON.parse(req.body)); markProcessed(id); res.status(200).end(); }); ``` **Python (FastAPI)** ```python import hmac, hashlib, time, os from fastapi import FastAPI, Request, HTTPException SECRET = os.environ["CALLGRID_WEBHOOK_SECRET"].encode() app = FastAPI() @app.post("/webhooks/callgrid") async def callgrid_webhook(req: Request): body = await req.body() ts = req.headers.get("x-webhook-timestamp", "") sig = req.headers.get("x-webhook-signature", "") if abs(time.time() - int(ts or 0)) > 300: raise HTTPException(400, "stale timestamp") expected = "v1=" + hmac.new(SECRET, ts.encode() + b"." + body, hashlib.sha256).hexdigest() if not hmac.compare_digest(expected, sig): raise HTTPException(401, "bad signature") delivery_id = req.headers["x-webhook-id"] if already_processed(delivery_id): return {"ok": True} handle(await req.json()) mark_processed(delivery_id) return {"ok": True} ``` **Go** ```go import ( "crypto/hmac" "crypto/sha256" "encoding/hex" "io" "net/http" "strconv" "time" ) func verify(secret string, r *http.Request) ([]byte, bool) { body, _ := io.ReadAll(r.Body) ts, err := strconv.ParseInt(r.Header.Get("X-Webhook-Timestamp"), 10, 64) if err != nil || time.Since(time.Unix(ts, 0)).Abs() > 5*time.Minute { return nil, false } mac := hmac.New(sha256.New, []byte(secret)) mac.Write([]byte(strconv.FormatInt(ts, 10) + ".")) mac.Write(body) want := "v1=" + hex.EncodeToString(mac.Sum(nil)) return body, hmac.Equal([]byte(want), []byte(r.Header.Get("X-Webhook-Signature"))) } ``` If you run the Go server code, `events.VerifySignature(secret, timestamp, signature, body, tolerance)` in `internal/events` implements the same check. ## Responses, retries and the schedule | Your response | Result | |---------------|--------| | 2xx (3xx is also accepted) | `delivered` | | 5xx, 429, timeout (5 s), connection error | retried per the schedule, then `failed` | | any other 4xx | `failed` immediately, no retry (the endpoint rejected the event) | Default backoff after a failed attempt: **2 s, 10 s, 1 m, 5 m, 30 m** (6 attempts in total, about 36 minutes). Override with the environment variable: ``` WEBHOOK_RETRY_SCHEDULE="2s,10s,1m,5m,30m" ``` An empty schedule means one attempt only. Deliveries run on a worker pool (4 workers, queue of 4096); when the queue is saturated an attempt is delayed, never dropped. Respond quickly (write to a queue, then return 200). Work that takes longer than 5 s times out and is retried, which is where idempotency matters. ## Idempotency Retries and replays send the **same body with the same `X-Webhook-Id`**; only `X-Webhook-Timestamp`, `X-Webhook-Attempt`, `X-Webhook-Signature` (and `X-Webhook-Replay`) change. Store processed ids (a unique index on `webhook_id` is enough) and return 200 for duplicates. Events may arrive out of order under retry: order by `timestamp` in the payload, not by arrival. ## Delivery log and replay The server keeps the last 1000 deliveries per app in `DATA_DIR/webhooks/.json`: ```json { "id": "whd_…", "app_id": "app_…", "event_type": "call.ended", "url": "https://…", "state": "pending", "next_attempt_at": "2026-10-27T10:00:11Z", "attempts": [ { "at": "2026-10-27T10:00:01Z", "status": 503, "duration_ms": 41 }, { "at": "2026-10-27T10:00:03Z", "status": 0, "duration_ms": 5002, "error": "context deadline exceeded" } ], "payload": { "type": "call.ended", "…": "…" }, "created_at": "…", "updated_at": "…" } ``` `state` is `pending` (more attempts scheduled), `delivered` or `failed`. Pending retries survive a restart: on start the log is loaded and each pending delivery is rescheduled for its `next_attempt_at` (deliveries that were in flight are retried immediately). **Replay** re-sends the stored payload to the app's *current* webhook URL as a new attempt on the same delivery id with `X-Webhook-Replay: true`; if the replay fails with a retryable status it follows the retry schedule again from the start. ## Owner API (portal session, `Authorization: Bearer `) | Method and path | Description | |-----------------|-------------| | `GET /v1/account/apps/{id}/webhook` | `{url, has_secret, secret_prefix, retry_schedule}` | | `POST /v1/account/apps/{id}/webhook/rotate-secret` | `{secret, secret_prefix}`; the secret is shown once | | `POST /v1/account/apps/{id}/webhook/test` | Sends `webhook.test` and returns `{ok, attempt, delivery, error?}` after the first attempt (retries continue in the background). 400 when the app has no URL. | | `GET /v1/account/apps/{id}/deliveries?state=&limit=` | Newest first; `state` is `pending`, `delivered` or `failed`; `limit` defaults to 50 (max 1000) | | `GET /v1/account/apps/{id}/deliveries/{did}` | One delivery with its attempts and payload | | `POST /v1/account/apps/{id}/deliveries/{did}/replay` | Replays; returns `{ok, attempt, delivery, error?}` | The app must belong to the session's account (otherwise 404). The webhook URL itself is still set with `PATCH /v1/account/apps/{id}` (`{"webhook_url": "…"}`). **Admin mirror** (`X-Admin-Key`): `GET /v1/admin/apps/{id}/deliveries`, `GET /v1/admin/apps/{id}/deliveries/{did}`, `POST /v1/admin/apps/{id}/deliveries/{did}/replay`. ## Portal page (open it directly after logging in at `/portal/`; it reuses the portal session from `localStorage`). Select an app, save the URL, rotate the secret, send a test event, browse deliveries with their attempts and payloads, replay, and copy the Node or Python verification snippet.