Dead letters

An event that a reader still fails after its last attempt is copied into the environment’s dead letters, with the integration it failed in and the reason. The event waits there until you send it again or discard it, and the other readers of the channel go on with their own events.

See the dead letters#

$ weikio deadletters list --env production

The list has one row per dead letter: its position, the time, the reader it failed in, the channel, transfer or entity type it came from, how many deliveries it had and the reason. Narrow it to one channel or one reader:

$ weikio deadletters list finance/invoice-created --env production --since 24h
$ weikio deadletters list --env production --integration finance/invoice-archiver

weikio deadletters show --seq 1043 --env production prints one dead letter in full: the integration it failed in, what the event came from, its deliveries, the reason, its headers and its body.

In the admin UI, Events › Dead letters lists the same, with each dead letter’s reason, headers and body, and sends one or a selection again or discards them.

/events/dead-letters

Send them again#

Fix the cause first, such as the upstream that was down or the bug in the route, and deploy the fixed version. Then send the dead letters again:

$ weikio deadletters resend --seq 1043 --env production
$ weikio deadletters resend finance/invoice-created --env production --since 24h

Each dead letter goes again to the integration it failed in, and to no other reader. The CLI asks first and says how many it sends, and -y confirms. A dead letter whose integration can’t receive the event now, because it was undeployed for example, is refused and kept. One whose event the stream no longer keeps is refused too, because only the stream has the body.

Discard them#

$ weikio deadletters discard --seq 1043 --env production

Discard removes dead letters you don’t need: an event the reader should never have had, or one you handled by hand.

Skip ahead or rewind a reader#

When a reader is far behind, or you want it to read a part of the channel again, move the reader itself:

$ weikio reader skip finance/invoice-archiver --channel invoice-created --env production
$ weikio reader rewind finance/invoice-archiver --channel invoice-created --to-seq 48000 --env production

skip passes over every event the reader hasn’t finished, so it gets only what is published from now on. rewind delivers the events again from a position on. Both say first how many events they pass over or deliver again. The integration’s Event delivery section in the admin UI has the same as Skip to now and Rewind, beside each read’s delay, the age of its oldest event, its retries and its dead letters.

Send any stored event again#

Any event the stream still keeps can be published again, to every reader or to one:

$ weikio events resend finance/invoice-created --env production --seq 48213 -y

Target: https://hub.example, project finance (argument), environment production (--env)
event 48213 resent as 48397 (Weikio-Resend-Of: 48213)

--to finance/invoice-archiver sends it to that integration alone. The event keeps its ID, so a route that handles each event once with #weikioIdempotent skips one it already handled.

A dead letter stays in the environment for its retention, 30 days by default. When you delete an integration, its dead letters stay too, under its ID, and --record-id lists, sends or erases them.