Skip to content
Deedboxlatest

Wire QueueBox

This guide shows you how to send events to other systems with QueueBox. Deedbox writes a QueueBox outbox row in the append’s transaction, so a message exists exactly when its event commits. QueueBox delivers it, with retries, at least once.

  1. Run QueueBox against the same database, and let it create its outbox table.

  2. Add the package:

    Terminal window
    dotnet add package Deedbox.QueueBox
  3. Choose the events to publish:

    services.AddDeedbox(es => es
    .UsePostgres(connStr)
    .Keys(keys => keys.FromEnvironment("DEEDBOX_MASTER_KEY"))
    .Stream<Cart>(s => s.Events<ItemAdded, CheckedOut>())
    .Stream<Manuscript>(s => s.Events<ReviewerInvited, CoAuthorAdded>())
    .UseQueueBox(q => q
    .Publish<CheckedOut>("cart.checked_out")
    // Events with [PersonalData] need a payload you shape, so no personal data leaks by default.
    .Publish<ReviewerInvited>("review.invited", (e, info) => new { e.ManuscriptId, e.ReviewerId })
    // Tell downstream systems to erase too.
    .Publish<SubjectErased>("privacy.subject_erased")));
Column Value
id The event ID. A destination deduplicates on X-Message-Id.
topic The topic you chose.
key The stream ID. On QueueBox 0.4.0 and later, it keeps one stream’s messages in order.
payload The event’s JSON, or the payload you shaped.
headers X-Correlation-Id, traceparent, and the event’s type, stream and version, plus the headers you add. The stream version is in x-deedbox-stream-version.
aggregate_type The stream type, or the aggregate type that a message callback sets.

An event with [PersonalData] needs a payload you shape; start-up fails otherwise (DBX032).

If QueueBox runs with a custom table or column mapping, match it with UseTable and UseColumns.

On QueueBox 0.4.0 and later, with migration V11 applied, one stream’s messages reach a destination in stream order. QueueBox delivers the rows of one key in insert order, one row at a time. An append holds its stream’s lock until it commits, and it inserts its rows in version order. The next append of the stream inserts after that commit.

The cost of this order: one message of a stream is in flight at a time. A message that waits for a retry holds back the later messages of its stream, until QueueBox sends it or marks it dead.

On an older QueueBox, or with another relay, one stream’s messages can arrive out of order. A receiver that needs order then compares x-deedbox-stream-version and drops a message older than the one it holds.

Publish<TEvent>((e, pending) => new QueueBoxMessage(topic, payload) { Headers = ... }) builds the message for each event. pending gives the event ID, type, version, stream ID, time and metadata.

  • The topic can differ per event, such as one topic per stream.
  • Your headers join the default headers. A header with the name of a default replaces its value, in any letter case. You cannot remove a default.
  • Deedbox does not send EventMetadata.Headers. Copy pending.Metadata.Headers into Headers to send them.
  • AggregateType sets the aggregate_type column for that event. A destination that routes on it, such as with exchangeFrom: aggregate_type, can then send the events of one stream to different exchanges. Leave it null to write the stream type.
  • Return null to write no message for that event.
  • An empty topic or aggregate type, a topic or aggregate type over 255 characters, a header with an empty name or a null value, or an exception in the callback fails the append with DBX032. The events and their messages commit together or not at all.

The row ID and the key stay the event ID and the stream ID. The event ID makes delivery idempotent, and the stream ID keeps the stream’s order.

Deedbox has no CloudEvents type. Build the envelope in the message callback.

In structured mode, the payload is the whole CloudEvent:

// Structured mode: the payload is the whole CloudEvent.
q.Publish<CheckedOut>((e, p) => new QueueBoxMessage("cart.checked_out", new
{
specversion = "1.0",
id = p.EventId,
source = "/shop/carts",
type = p.EventType,
subject = p.StreamId,
time = p.OccurredAt,
datacontenttype = "application/json",
data = e,
})
{
Headers = new Dictionary<string, string> { ["content-type"] = "application/cloudevents+json" },
});

In binary mode, the attributes are headers and the payload is the event:

// Binary mode: the attributes are headers, and the payload is the event.
q.Publish<CheckedOut>((e, p) => new QueueBoxMessage("cart.checked_out", e)
{
Headers = new Dictionary<string, string>
{
["ce-specversion"] = "1.0",
["ce-id"] = p.EventId.ToString(),
["ce-source"] = "/shop/carts",
["ce-type"] = p.EventType,
["ce-subject"] = p.StreamId,
["ce-time"] = p.OccurredAt.ToString("O"),
["content-type"] = "application/json",
},
});

The HTTP binding names the headers ce-*. For Kafka, the binding uses ce_*; name the headers the way your destination expects.

A message is a contract with other teams; an event is not. Change them apart.

When you change an event’s shape, the callback gets the new shape, for old events too. Map it to the fields that consumers already read:

// cart.item_added is at version 3 in the store, and upcasting gives the callback that shape for old events too.
// The message keeps the fields consumers already read, and adds price as a new field they can ignore.
q.Publish<ItemPriced>((e, p) => new QueueBoxMessage("cart.item_added", new { sku = e.Sku, qty = e.Qty, price = e.Price }));

Add fields. Do not remove or rename them. A consumer that ignores unknown fields then needs no change.

A breaking change, such as a renamed or removed field, needs a new topic. Deedbox writes one message per event, so the app cannot write the old topic and the new topic in one transaction. Feed the old topic from the new one while consumers move:

  1. Start a translator: a consumer that reads the new topic and writes the old contract to the old topic. It keeps each message ID, so consumers still deduplicate.

  2. Release the app with the new publication:

    // A breaking change goes to a new topic. From this release, nothing in the app writes the old topic.
    q.Publish<ItemPriced>((e, p) => new QueueBoxMessage("cart.item_added.v2", new
    {
    sku = e.Sku,
    quantity = e.Qty,
    unitPrice = new { amount = e.Price, currency = "EUR" },
    }));
  3. Move each consumer to the new topic.

  4. Confirm that no consumer reads the old topic, from the broker’s consumer groups or the QueueBox metrics. Then stop the translator.