Pulse: monitoring and alerts
Health score, one-click fixes, alerts, the weekly email, and alerts by email, Slack, Discord, Microsoft Teams, PagerDuty, Opsgenie, Telegram, ntfy or webhook.
Pulse watches your databases with nothing to set up. It gives each a health score, findings, graphs, alerts, slow queries and a Monday email. Most problems come with Apply fix.
It starts when you add a database, on every plan. This page shows PostgreSQL; see By database for the others.
What the agent collects
Every 60 seconds the agent runs a few cheap catalog queries (session rowsafe-agent-monitor, Unix socket, 2-second timeout). A struggling server shows a gap, never a pile-up. Monitoring never blocks a backup, Proof or Mark.
Privacy
pg_stat_statementstext is normalized: values become$1,$2. So are query plans.- Activity query text is not. It can hold personal data.
ROWSAFE_COLLECT_QUERY_TEXT=falsein/etc/rowsafe/agent.envleaves it out. - Statements that look like they carry a password are redacted on your server.
ROWSAFE_MONITORING=falseturns monitoring off (heartbeats and task results only).- The database's own log is separate: Logs.
Retention
| Resolution | Kept for |
|---|---|
| 1 minute | 48 hours |
| 5 minutes | 14 days |
| 1 hour | 90 days |
Resolved alerts: 90 days. Sent notifications: 30 days.
By database
Score, alerts, uptime, disk forecast, slow queries and the weekly email work for every engine. Values are removed from statements on your server.
| What Pulse watches | One-click fixes | |
|---|---|---|
| MySQL, MariaDB | connections, queries per second, slow queries (performance_schema), row lock waits, the buffer pool, long transactions, replication delay | stop a query or end a blocking session, rebuild a table with a lot of free space (OPTIMIZE TABLE), turn on the slow query log |
| MongoDB | connections, operations, queues, the WiredTiger cache, write conflicts, the oplog window, secondary lag, slow operations (the profiler) | stop an operation (killOp), turn on query statistics |
| ClickHouse | queries, inserts, merges, memory, data parts per partition, mutations, replication, broken parts, the queries running now | stop a query (KILL QUERY), cancel a stuck mutation (KILL MUTATION) |
| OpenSearch | cluster health, shards, disk against OpenSearch's watermarks, read-only indices, heap and garbage collection, refused requests, the last snapshot | set replicas to 0 on one node, allow writes again once the disk has room (details) |
| Qdrant | memory, collections and their status, the issues Qdrant reports (like unindexed fields), snapshots left on disk, keys | create the index Qdrant suggests, move a collection's vectors to disk, remove admin keys (details) |
| Meilisearch | size on disk and what the data uses, indexes, the task queue (waiting, failed), the indexing memory limit | compact an index, clear old finished tasks (details) |
| Redis, Valkey | memory against maxmemory, evictions, hit rate, commands per second, clients (blocked, rejected), fragmentation, persistence, replication and lag, the slow log (command names only), latency events; a server without a password is flagged | set an eviction policy, give memory back (MEMORY PURGE) or turn on active defragmentation, disconnect a stuck client, raise the replication backlog |
All also get: back up now, Restart, updates, security updates and reboots, Tune for this server, and Recommendations' index changes. Redis and Valkey don't have updates, tuning or Recommendations yet (details). More: MySQL and MariaDB, MongoDB, ClickHouse.
Health score
Each database gets a score from 0 to 100 and its findings, worst first: what's wrong, why it matters, what to do, and a button when Rowsafe can do it.
| Score | Meaning |
|---|---|
| 90-100 | Healthy |
| 70-89 | Needs attention |
| 50-69 | At risk |
| Under 50 | Critical |
Points come off for backups and Proof, availability, disk, connections, vacuum and wraparound, locks and slow queries, replication, and security (up to 25).
- One critical problem caps the score at 59, a warning at 89. A good score never hides a serious problem.
- Missing data means fewer findings, never a false alarm.
- Recorded every 15 minutes, so you see the trend.
rowsafe pulse # every database, worst first
rowsafe pulse app # one database: score, findings, and what's finerowsafe pulse exits with code 3 when a database scores below 70.
Apply fix
Fix buttons appear under Pulse → Health, and next to the same problem on Insights, Monitoring, Queries and Alerts. Click, confirm if asked, and watch each step and the result ("Cleaned up 3 tables; about 1.2 million dead rows removed.").
| Problem | Button | What Rowsafe does |
|---|---|---|
| No backup yet, or the last one is too old | Back up now | Starts a full backup. |
| The latest Proof failed, or none passed this week | Run Proof now | Runs a restore test. |
| Backups can't reach your storage | Check again | Tests that changes reach your bucket, once you've fixed the reported problem. |
| Backups aren't on yet | Turn on backups | Applies the setup plan (after you confirm). |
| PostgreSQL is waiting for a restart | Restart PostgreSQL now | Restarts it, where restarts are allowed. |
| Backups can be up to 5 minutes behind | Apply fix | Sets archive_timeout to a minute. A reload, no restart. |
| A Rewind copy is about to expire | Keep it 24 hours longer | Keeps the copy 24 more hours. |
| Data kept aside by a rewind uses disk | Delete the old data now | Deletes the data from before the rewind. Can't be undone. |
| Dead rows piling up, vacuum behind, transaction or multixact ID wraparound | Clean up | VACUUM (ANALYZE) on those tables, with FREEZE for wraparound. |
| Cleanup is blocked: something holds old rows that VACUUM can't remove | One button per blocker | Ends the old session (or cancels its query), rolls back a forgotten prepared transaction, removes an inactive replication slot or moves a logical one forward. You confirm each. A standby's queries: under Do it yourself. |
| Every autovacuum worker is busy for 30 minutes | Give cleanup more room | Raises autovacuum_vacuum_cost_limit at once and autovacuum_max_workers by 2. Before PostgreSQL 18, the extra workers need a restart, which you choose when. |
| A query got slower | Refresh statistics | ANALYZE on the tables it reads, when their statistics may be stale, then checks again. Otherwise (or for inserts) it shows the likely causes. |
| An index with a lot of wasted space | Rebuild index | REINDEX INDEX CONCURRENTLY (PostgreSQL 12 and later). |
A table with a lot of wasted space, where pg_repack is installed | Rebuild (pg_repack) | Rebuilds the table while your app keeps working (a short lock at the start and end), after a Mark. Needs a primary key and free disk for the copy. |
| An unused or duplicate index | Remove | DROP INDEX CONCURRENTLY, after saving a Mark. |
| An index would make slow queries faster (tested on a copy) | Create index | CREATE INDEX CONCURRENTLY; a failed build leaves nothing behind. |
The hypopg package is installed but off | Turn on hypopg | CREATE EXTENSION hypopg, so index ideas are checked without a copy first. Tables and data don't change. |
| A session idle in a transaction, a transaction open too long, a session blocking others | End this session, or cancel its query | pg_terminate_backend, or pg_cancel_backend for a running query. |
| An inactive replication slot holding back WAL | Remove the slot | pg_drop_replication_slot. |
| Most connections are in use, or almost none are left | Turn on pooling | Installs and starts PgBouncer, where allowed. See Connection pooling. |
| PgBouncer isn't answering | Start PgBouncer again | Rewrites its configuration and starts it. |
| Apps wait for a connection in PgBouncer | Raise the pool size, or Switch to transaction mode | Raises the pool by half, within max_connections, or switches to transaction mode. Nothing is disconnected. |
| Some changes didn't reach the second copy, or it has no full backup | Back up to the second copy | A full backup into the second copy. |
| Keeping one full backup fewer would save money, safely | Keep 1 full backup | Changes the setting; the oldest full backup goes at the next backup (after you confirm). |
| The disk is filling up | The fixes above that free space | Removing an inactive slot or unused indexes, cleaning up tables. |
| PostgreSQL uses default memory settings on a bigger server | Tune for this server | Applies the recommended settings, after a Mark. See Tuning. |
Query statistics are off (pg_stat_statements isn't loaded) | Turn on query statistics | Adds it to shared_preload_libraries and creates the extension; it starts after the next restart. |
| Changed settings wait for a restart | Restart PostgreSQL | Restarts it, where restarts are allowed. |
| A newer PostgreSQL minor release is available | Update to 16.10 | A Mark, the install, a restart, where updates are allowed. See Updates. |
| A newer release is installed but not running yet | Restart PostgreSQL | Restarts it, where restarts are allowed. |
| Security updates are waiting | Install security updates | A Mark, then the server's security updates (not PostgreSQL's packages), where allowed. |
| The server needs a reboot | Reboot the server | A Mark, then a reboot (you type the name), where allowed. |
| An upgrade keeps the old PostgreSQL version | Remove PostgreSQL 16 | Deletes the version kept for undo, freeing its disk. |
Connections aren't encrypted, weak password hashing, CREATE on schema public for everyone | Turn on encryption, Use modern password hashing, Remove CREATE from everyone | See the security check. |
Pulse also warns when your major version stops getting security fixes, or will within six months. Upgrading starts with a check and a rehearsal on the Updates tab (how).
No button for (the finding explains, with commands under Do it yourself): an offline agent, storage keys (they never leave your server), an untested index (only foreign-key indexes and indexes proven on a copy are created), a full table rebuild without pg_repack on the server (VACUUM FULL), your own PgBouncer, and replication lag.
What keeps it safe
- Only Rowsafe's own fixes. You pick a finding and a fix; Rowsafe rechecks and runs that exact fix with its own parameters. No SQL is sent.
- Rechecked on the server just before running: the table, index, session or slot is still the one found. Rowsafe, replication and autovacuum sessions are never ended. Indexes behind a primary key, unique or exclusion constraint are never removed.
- Owners and admins only. Read-only API keys can't apply fixes. Every fix is audited.
- You confirm what disrupts. Ending a session or removing an index or slot asks you to type the database name.
- A Mark first (
before-fix-...) before removing an index, so you can restore to just before it. - Gentle. Index changes wait at most 5 seconds for a lock, run
CONCURRENTLY, and vacuum runs with a cost delay. One fix at a time per database. - AI agents apply fixes with
apply_fixon the MCP server, as the person who connected them: at once for an owner or admin, refused otherwise.
Restart now or later
Some fixes and settings only take effect after a restart: a few seconds of refused connections. Restart (on the database's page, or the fix) asks when:
- Now.
- Later: a time you pick, up to 30 days ahead.
- In the next maintenance window, on Rowsafe Cloud.
You confirm with the database's name. A banner on the database's page shows the scheduled restart, with Cancel. The addresses that get your weekly Pulse get an email when it's scheduled, an hour before, and after.
At that time Rowsafe restarts it like Restart now, only where root allowed restarts. If the server isn't reporting, it keeps trying for an hour, then gives up and tells you. If it was restarted since, Rowsafe skips it and says so.
Disk forecast
From 14 days of disk use, Rowsafe tells you when the disk will be full ("in about 9 days"), ignoring one-off spikes. It needs about 2 days of history. disk_full_forecast warns 14 days ahead.
Table and index insights
About every 30 minutes, from catalog statistics (never your rows):
- largest tables and indexes, and their growth;
- unused and duplicate indexes;
- wasted space (bloat);
- tables with many dead rows;
- tables read by full scans (index recommendations test a fix);
- transaction and multixact ID age.
Buttons: Remove, Rebuild index, Clean up. CLI: rowsafe insights app.
Slow queries
With pg_stat_statements on (Turn on query statistics, then a restart), Rowsafe keeps 14 days of per-query timing. It shows the queries that took the most time and which got slower: average time more than doubled, adding at least 5 ms per call or a minute in all.
Query plans (PostgreSQL 12+): for the busiest queries, a plan in plain words, like "Reads the whole table orders: about 2.1 million rows." The agent asks PostgreSQL how it would run the query, without running it and without values (conditions show $1). They're estimates, not timings from real runs: Rowsafe never turns on auto_explain.
rowsafe top app # last 24 hours
rowsafe top app --since 168h --sort calls # last 7 days, by number of callsUptime
Each minute the agent reports whether it could query the database. You see uptime over 24 hours, 7 and 30 days, and each outage. Minutes with the agent offline don't count; agent_offline covers them.
Your weekly Pulse
Every Monday morning, per database: health score, backups and Proof, growth, storage cost, the second copy, disk forecast, top fixes, top recommendation, slowest queries and open alerts.
On for new organizations. It goes to the addresses you choose, or your email alert channels. Change it under Settings → Weekly Pulse email; every email has a one-click unsubscribe.
Alert rules
Rules are checked every 30 seconds. An alert fires once its condition has held for For.
| Rule | Severity | Default threshold | For | Fires when |
|---|---|---|---|---|
backup_stale | critical | 26 hours | 15m | No successful backup in the threshold, or an active database was never backed up after 12 hours. |
full_backup_stale | warning | 8 days | 15m | No successful full backup in the threshold. |
drill_failed | warning | The latest Proof failed. | ||
drill_overdue | warning | 8 days | 15m | No Proof passed in the threshold. |
wal_archiving_failing | critical | 10m | Archiving keeps failing. Point-in-time recovery stops at the last archived segment, and pg_wal grows. | |
postgres_unreachable | critical | 5m | The agent is up but can't query PostgreSQL. | |
agent_offline | critical | 5 minutes | 1m | No heartbeat from the agent. |
restore_point_unconfirmed | warning | 15m | A Mark was written but archiving hasn't moved past it. | |
awaiting_restart | warning | 24 hours | A plan was applied, but PostgreSQL wasn't restarted. | |
disk_free_low | warning | 15% | 5m | Free disk on the data directory's filesystem below the threshold. |
disk_free_critical | critical | 5% | 5m | Same, lower threshold. |
disk_full_forecast | warning (critical under 3 days) | 14 days | 30m | At the last two weeks' growth rate, the disk fills within the threshold. |
connections_high | warning | 80% | 5m | Connections above the share of max_connections. |
connections_critical | critical | 95% | 2m | Same, higher threshold. |
long_transaction | warning | 30 minutes | A transaction open longer than the threshold. | |
idle_in_transaction | warning | 10 minutes | A session idle in an open transaction longer than the threshold. | |
xid_wraparound | critical | 50% | Transaction ID headroom below the threshold. | |
multixact_wraparound | critical | 50% | Multixact ID headroom (row locks shared by several transactions) below the threshold. | |
replication_slot_inactive | warning | 1 GiB | 15m | An inactive replication slot holds back more WAL than the threshold. |
blocked_queries | warning | 60 seconds | 1m | A query waited longer than the threshold for another session's lock. |
replication_lag | warning | 5 minutes | 5m | A replica is further behind than the threshold. |
deadlocks | warning | 5 | More deadlocks than the threshold in 10 minutes. | |
cache_hit_low | info | 90% | 30m | Cache hit ratio below the threshold with real read traffic. |
pooler_down | critical | 2m | PgBouncer doesn't answer (only where pooling is on or monitored). | |
pooler_clients_waiting | warning | 2 seconds | 5m | Clients waited longer than the threshold for a connection in PgBouncer. |
second_copy_failing | warning | 30 minutes | 5m | Changes haven't reached the second copy for longer than the threshold, or some were left out. PostgreSQL and the first storage aren't affected. |
rls_drift | warning | A security scan found a new row-level security gap: a table your API reaches without it, a table with no policies, a policy that lets every row through, or a public storage bucket. PostgreSQL only. |
- Critical: recoverability or availability is degrading now. Warning: within a working day. Info: for the record.
- Backup, archiving, Proof and reachability rules wait until backups are on, so a new database doesn't page you.
- Admins can change or turn off any rule under Alerts (
rowsafe alerts rules).
How alerts behave
- One open alert per rule and target. It resolves when the condition clears, and you're told.
- A critical alert repeats every 4 hours until acknowledged.
- No data is not good news. Without fresh metrics an alert keeps its state; a silent agent raises
agent_offline. - Fix it on an alert applies the fix, or opens Health when there's a choice.
rowsafe alerts # firing alerts (--all adds resolved ones, --resolved shows only those)
rowsafe alerts ack alr_... # acknowledge: no more remindersNotification channels
Add and test channels under Settings → Alerts and notifications. Each has a minimum severity (default warning). Messages say what, where, since when, the value and threshold, and link to Fix it when there is one.
| Type | Destination | Notes |
|---|---|---|
email | Up to 20 addresses (repeat --address) | Plain text. |
slack | A Slack incoming webhook | |
discord | A Discord webhook | Mentions are off, so nobody is pinged. |
teams | A Microsoft Teams incoming webhook or Workflows URL | A card with the details. |
pagerduty | A service's Events API v2 integration key (routing key), US or EU account | Opens an incident and resolves it when the alert clears. |
opsgenie | An API integration's key, US or EU account | Priority from severity (critical P1). Closes the alert when it clears. |
telegram | A bot token (from @BotFather) and a chat ID (or @channel) | Add the bot to the chat first. |
ntfy | A server (default https://ntfy.sh), a topic and an optional access token | Priority from severity. On a public server, pick a topic nobody can guess. |
webhook | Any public https URL | JSON, signed. The signing secret is shown once. See Webhooks. |
Tests send at once, whatever the severity. URLs, keys, tokens and topics are secrets and never shown in full again. Up to 20 channels per organization.