Wire QueueBox
This guide shows you how to send events to other systems with QueueBox. Deedbox writes a QueueBox outbox row in the append’s transaction, so a message exists exactly when its event commits. QueueBox delivers it, with retries, at least once.
-
Run QueueBox against the same database, and let it create its
outboxtable. -
Add the package:
Terminal window dotnet add package Deedbox.QueueBox -
Choose the events to publish:
services.AddDeedbox(es => es.UsePostgres(connStr).Keys(keys => keys.FromEnvironment("DEEDBOX_MASTER_KEY")).Stream<Cart>(s => s.Events<ItemAdded, CheckedOut>()).Stream<Manuscript>(s => s.Events<ReviewerInvited, CoAuthorAdded>()).UseQueueBox(q => q.Publish<CheckedOut>("cart.checked_out")// Events with [PersonalData] need a payload you shape, so no personal data leaks by default..Publish<ReviewerInvited>("review.invited", (e, info) => new { e.ManuscriptId, e.ReviewerId })// Tell downstream systems to erase too..Publish<SubjectErased>("privacy.subject_erased")));
What each row holds
Section titled “What each row holds”| Column | Value |
|---|---|
id |
The event ID. A destination deduplicates on X-Message-Id. |
topic |
The topic you chose. |
key |
The stream ID. On QueueBox 0.4.0 and later, it keeps one stream’s messages in order. |
payload |
The event’s JSON, or the payload you shaped. |
headers |
X-Correlation-Id, traceparent, and the event’s type, stream and version, plus the headers you add. The stream version is in x-deedbox-stream-version. |
aggregate_type |
The stream type, or the aggregate type that a message callback sets. |
An event with [PersonalData] needs a payload you shape; start-up fails otherwise (DBX032).
If QueueBox runs with a custom table or column mapping, match it with UseTable and UseColumns.
Message order
Section titled “Message order”On QueueBox 0.4.0 and later, with migration V11 applied, one stream’s messages reach a destination in stream order. QueueBox delivers the rows of one key in insert order, one row at a time. An append holds its stream’s lock until it commits, and it inserts its rows in version order. The next append of the stream inserts after that commit.
The cost of this order: one message of a stream is in flight at a time. A message that waits for a retry holds back the later messages of its stream, until QueueBox sends it or marks it dead.
On an older QueueBox, or with another relay, one stream’s messages can arrive out of order. A receiver that needs order then compares x-deedbox-stream-version and drops a message older than the one it holds.
Shape each message
Section titled “Shape each message”Publish<TEvent>((e, pending) => new QueueBoxMessage(topic, payload) { Headers = ... }) builds the message for each event. pending gives the event ID, type, version, stream ID, time and metadata.
- The topic can differ per event, such as one topic per stream.
- Your headers join the default headers. A header with the name of a default replaces its value, in any letter case. You cannot remove a default.
- Deedbox does not send
EventMetadata.Headers. Copypending.Metadata.HeadersintoHeadersto send them. AggregateTypesets theaggregate_typecolumn for that event. A destination that routes on it, such as withexchangeFrom: aggregate_type, can then send the events of one stream to different exchanges. Leave it null to write the stream type.- Return
nullto write no message for that event. - An empty topic or aggregate type, a topic or aggregate type over 255 characters, a header with an empty name or a null value, or an exception in the callback fails the append with DBX032. The events and their messages commit together or not at all.
The row ID and the key stay the event ID and the stream ID. The event ID makes delivery idempotent, and the stream ID keeps the stream’s order.
Send CloudEvents
Section titled “Send CloudEvents”Deedbox has no CloudEvents type. Build the envelope in the message callback.
In structured mode, the payload is the whole CloudEvent:
// Structured mode: the payload is the whole CloudEvent.q.Publish<CheckedOut>((e, p) => new QueueBoxMessage("cart.checked_out", new{ specversion = "1.0", id = p.EventId, source = "/shop/carts", type = p.EventType, subject = p.StreamId, time = p.OccurredAt, datacontenttype = "application/json", data = e,}){ Headers = new Dictionary<string, string> { ["content-type"] = "application/cloudevents+json" },});In binary mode, the attributes are headers and the payload is the event:
// Binary mode: the attributes are headers, and the payload is the event.q.Publish<CheckedOut>((e, p) => new QueueBoxMessage("cart.checked_out", e){ Headers = new Dictionary<string, string> { ["ce-specversion"] = "1.0", ["ce-id"] = p.EventId.ToString(), ["ce-source"] = "/shop/carts", ["ce-type"] = p.EventType, ["ce-subject"] = p.StreamId, ["ce-time"] = p.OccurredAt.ToString("O"), ["content-type"] = "application/json", },});The HTTP binding names the headers ce-*. For Kafka, the binding uses ce_*; name the headers the way your destination expects.
Change a message’s contract
Section titled “Change a message’s contract”A message is a contract with other teams; an event is not. Change them apart.
Keep the contract when an event changes
Section titled “Keep the contract when an event changes”When you change an event’s shape, the callback gets the new shape, for old events too. Map it to the fields that consumers already read:
// cart.item_added is at version 3 in the store, and upcasting gives the callback that shape for old events too.// The message keeps the fields consumers already read, and adds price as a new field they can ignore.q.Publish<ItemPriced>((e, p) => new QueueBoxMessage("cart.item_added", new { sku = e.Sku, qty = e.Qty, price = e.Price }));Add fields. Do not remove or rename them. A consumer that ignores unknown fields then needs no change.
Move consumers to a new contract
Section titled “Move consumers to a new contract”A breaking change, such as a renamed or removed field, needs a new topic. Deedbox writes one message per event, so the app cannot write the old topic and the new topic in one transaction. Feed the old topic from the new one while consumers move:
-
Start a translator: a consumer that reads the new topic and writes the old contract to the old topic. It keeps each message ID, so consumers still deduplicate.
-
Release the app with the new publication:
// A breaking change goes to a new topic. From this release, nothing in the app writes the old topic.q.Publish<ItemPriced>((e, p) => new QueueBoxMessage("cart.item_added.v2", new{sku = e.Sku,quantity = e.Qty,unitPrice = new { amount = e.Price, currency = "EUR" },})); -
Move each consumer to the new topic.
-
Confirm that no consumer reads the old topic, from the broker’s consumer groups or the QueueBox metrics. Then stop the translator.