Built end to end — architecture to deployment
Wagend
A multi-tenant WhatsApp Business CRM. Agencies and their teams run sales, support and marketing from a shared WhatsApp inbox over the Meta WhatsApp Cloud API — contacts, broadcasts, keyword automations and AI-assisted replies, with every tenant isolated by agency id.
01The problem
Under the inbox it's an event-plumbing problem, not a CRUD app: ingest and fan out live WhatsApp traffic across horizontally-scaled, stateless Node pods, keep every tenant's data isolated, and meter every billable action exactly once — even though Meta's webhooks and Stripe's webhooks can each be delivered more than once.
02The constraint
A single agent is usually signed in from several devices at once — phone and desktop — each socket landing on a different pod, so 'deliver to this agent' means finding every pod that holds one of their sockets, and the event that should reach them normally originates on a pod that holds none of them. Meta's rate limits are per-WhatsApp-number (~250 msg/sec), not per-tenant, so broadcasts can't just blast. Multi-tenant leakage is a critical failure, so every query carries the agency id. And every send and every billing write has to survive a retried webhook.
03What I built
Inbound: a message hits the webhook, has its SHA-256 signature validated, resolves to a Contact, is logged to the conversation, atomically bumps the inbox thread's unread state in SQL, and is pushed to the client through the notifications gateway — then a trigger engine fires any keyword auto-replies.
Real-time: a Redis presence registry holds a userId→serverIds map — which pods currently hold each agent's device sockets — under TTL keys, and every pod subscribes to its own channel. To deliver, a service looks the recipient up and publishes a targeted payload to only the pods holding their sockets (all their devices at once) instead of broadcasting to every pod; the Socket.io Redis adapter still carries room fan-out where a genuine broadcast is wanted. Typing indicators go to the workspace room but skip the sender's own room, so your desktop doesn't echo what you're typing on your phone.
Broadcasts never run in the request: the audience is chunked onto BullMQ queues, the API returns immediately, and workers merge personalization tags and send through a gateway-provider abstraction (Meta or Twilio), writing a log per message. A p-limit concurrency manager holds sends under Meta's per-number ceiling.
Billing is a credit economy enforced at two gates — an entitlements service (plan cached in Redis, 5-minute TTL) checks quota and a credit ledger debits the wallet, with a dual-enforcement service that refunds on failure. Stripe checkout and webhooks are processed idempotently on a BullMQ queue.
The AI layer routes through OpenRouter (multi-model, bring-your-own-key) for draft replies and summaries, and an agent pipeline parses, chunks and embeds each tenant's knowledge base into Qdrant, then retrieves against it to ground replies.
04The decision
Options considered
The Socket.io Redis adapter solves cross-pod delivery, but its default is to publish every emit to a channel every pod subscribes to — so each pod receives every message and discards the ones whose socket it doesn't hold. Run N pods and the real-time bus does N× the work it needs to, exactly when you've scaled out because you're already busy.
- ASticky sessions — pin each agent to one pod. Avoids the fan-out, but pins their devices to one pod and breaks the moment they connect a second device elsewhere.
- BThe adapter's default broadcast — every emit goes to every pod, each filters for the sockets it owns. Simple, but every pod pays for every message as you scale.
- CA global userId→serverIds map in Redis, each pod subscribing to its own channel — look the recipient up before sending and publish only to the pod(s) that actually hold their sockets.
Chose The userId→serverIds map with targeted, per-pod delivery.
The trade-off I accepted
The default adapter is the easy answer and it's correct — it just makes every pod process every message, so at scale the bus does N× the work. Instead I keep a userId→serverIds map in global Redis: each pod subscribes to its own channel, and before an emit I look up which pods hold the recipient's sockets and publish only to those — one message reaches the one or two pods that need it, not all N. Multi-device falls out for free, since an agent simply maps to several server ids. The cost is that I now own that map's consistency — write the server id on connect, drop it on disconnect, expire stale entries, and handle the reconnect and failover races — but it bought real-time fan-out that scales with the sockets that actually want a message, not with pod count.
05The result
- Stateless socket pods scale horizontally behind the Redis adapter — a pod can drop without taking a tenant's live inbox with it.
- A credit economy ($1 = 12 credits) meters every message and AI token, gated at two enforcement points and idempotent under Stripe webhook retries.
06Stack
- NestJS
- Socket.io
- Redis adapter
- BullMQ
- Prisma
- MySQL
- Meta WhatsApp API
- Stripe
- OpenRouter
- Qdrant
- React
- TypeScript