Telemetry is waiting on a hub server
Weik.io 2026.2A hub server keeps telemetry on its own disk because a store didn't take it, and sends it with the original times once the store answers.
| Kind | Severity | Who sees it |
|---|---|---|
telemetry- |
Warning, and an error while the server accepts no more | Platform administrators, in a list that isn’t narrowed to an environment or a project |
The row’s figure gives the numbers in a few words, such as 1.4 GiB waiting, oldest 2 h 05 min.
What it means#
A hub server keeps logs, metrics or traces on its own disk because a store didn’t take them. Nothing is lost while it waits, and the server sends them with their own times once the store answers.
What raises it#
The oldest waiting batch has waited a minute. It becomes an error when the server has no room for more and refuses new batches, and the problem clears once nothing waits. Each server raises it from its own disk, so it shows even while the server that runs the stores is away.
What to check#
See the stores and what waits for each:
$ weikio platform healthOn a hub server, in the stack’s folder, check the stores, with a line for what waits for each:
$ docker compose run --rm infra weikio-backend doctorHow to fix it#
Start the store that doesn’t answer, in the stack’s folder of the server that runs it:
$ docker compose up -d victoria-logs victoria-metrics victoria-traces