Built end to end — MU-plugin + ingestion service
WP Activity Logs
A WordPress must-use plugin on every managed site that captures admin, security and content events and pushes them to a central NestJS ingestion service, which durably queues and stores them and exposes the whole fleet's activity to a read-only AI agent over MCP.
01The problem
Fanning telemetry IN from many independent, untrusted WordPress installs. The hard parts are all inbound-concentration: correlated spikes (a fleet-wide plugin update or a brute-force wave hits many sites in the same minute, so events arrive bunched), untrusted heterogeneous clients on a shared key, and the WordPress plugin lifecycle — the plugin runs inside a visitor's PHP request, so it can't block on the network or keep a durable local queue, and the request can be killed mid-flight.
02The constraint
The client side must add ~0 ms to page response and keep zero local state, yet the server must never drop an event — even during a burst or a Redis outage. That rules out both 'make the site wait for an ack' and 'trust the queue as the only buffer'. The plugin is deliberately fire-and-forget with no retry, so durability can only live on the server.
03What I built
The MU-plugin captures events across 14 domains — auth, security, posts / pages / custom types, media, comments, plugins, themes, core, menus, users and 21 whitelisted options — plus a file-integrity scan (core-checksum and uploads diffs, daily), a connection monitor for Instagram Feed / Business Reviews (15-minute cron), and a lorem-ipsum scanner. Elementor, Beaver Builder and Gutenberg edits are normalised into one added / removed / modified diff.
Each event is one immediate non-blocking POST (2-second timeout, X-Api-Key) with no retry — so it adds ~0 ms to the page and keeps no local state. If the POST fails it's dropped by design; durability is the server's job.
Ingestion (NestJS) authenticates with a constant-time key check, validates the payload against a DTO, writes it to a Postgres outbox table synchronously, enqueues a BullMQ job carrying only the outbox row id, and returns 202 immediately.
A BullMQ worker (concurrency 5, 3 attempts, exponential backoff) atomically claims each outbox row and persists it via Prisma; a 15-minute cron re-enqueues rows stuck over 30 minutes (covering worker or Redis crashes), and an hourly cron prunes the outbox in 5,000-row batches.
The stored logs feed a read-only AI tool layer — the default tool answers natural-language 'who did what, when' across the fleet, surfaces sites with placeholder content or failing integrations, and pages results with keyset cursors (500 rows/page).
04The decision
Options considered
Events arrive bunched from many sites, the WordPress client won't retry, and Redis or Postgres latency must never reach the client. The question is where durability lives.
- AWrite straight to Postgres inside the request — durable, but the client waits on the DB and a spike hits the database directly.
- BEnqueue to BullMQ/Redis and ack — fast, but Redis is the only buffer, so a Redis outage drops events the client never retries.
- CA transactional outbox — a synchronous durable insert, then an async enqueue of just the row id.
Chose The transactional outbox.
The trade-off I accepted
The client can't retry, so if I acked from Redis alone a queue outage would silently lose events. Writing straight to Postgres in the request couples the client to database latency and points a correlated spike right at the DB. The outbox splits the difference: a fast synchronous insert makes the row the source of truth and returns 202 before any processing, then only the row id goes on the queue. The cost is two DB writes per event and a cleanup cron to prune the outbox, plus eventual consistency — a log is queryable after the worker runs, not at request time. In exchange, a fleet-wide spike is absorbed by the outbox and queue instead of the client or the database, and if Redis is down the row simply stays pending until a recovery cron re-enqueues it. Nothing is dropped.
05The result
- 14 event domains captured per site and pushed as one non-blocking 2-second POST — ~0 ms added to any page load.
- A transactional outbox plus a recovery cron means no event is lost across a correlated spike or a Redis outage; the plugin never retries because it never has to.
- The whole fleet's activity is queryable by a read-only AI agent over MCP, paged with keyset cursors.
06Stack
- PHP
- WordPress
- NestJS
- Prisma
- PostgreSQL
- BullMQ
- Redis
- MCP