Message storage is off
Weik.io 2026.2The message server of one hub server has turned its storage off, usually because its disk is full, and the hub runs without a spare copy.
| Kind | Severity | Who sees it |
|---|---|---|
message- |
Error | Platform administrators, in a list that isn’t narrowed to an environment or a project |
What it means#
The message server of one hub server has stopped keeping its copy of the hub’s storage, which a full disk does. The hub keeps working on the other copies, but it has no spare, and losing one more data server stops every change.
What raises it#
A data server’s message server has had its storage turned off for a minute. Each backend node checks this itself, without the metric store.
What to check#
On the hub server, see the free disk space:
$ df -hIn the stack’s folder, run the checks:
$ docker compose run --rm infra weikio-backend doctorHow to fix it#
Free disk space on that server, or grow its disk, then restart its message server in the stack’s folder:
$ docker compose restart natsThe server copies what it missed from the others when it starts.