Log drains are in private preview. Contact
support@unkey.com to request access.
How delivery works
Unkey sends events asynchronously in batches. A destination must accept the complete batch before Unkey marks the delivery as successful. HTTP destinations must return a2xx status within 30 seconds. Return a
non-2xx status if your endpoint cannot process the complete batch. Unkey then
retries the same batch.
Log drains provide at-least-once delivery. A destination can receive an event
more than once after a retry. Use id for audit logs or request_id
for key verifications and gateway requests. Use log_id for runtime logs.
Rate-limit checks in a multi-limit request share request_id, so it
does not uniquely identify each decision.
Deduplicate within each stream.
A drain starts at its creation time. Historical backfill is not supported.
After a pause or failure, delivery resumes from the last committed position,
subject to the stream’s retention period. Verification logs are retained for
90 days; gateway requests are retained for seven days. See audit log retention for audit
logs.
Configure a log drain
Create a log drain from your workspace settings.- Navigate to your workspace Settings.
- Select Log Drains.
- Click Create Log Drain.
- Select HTTP or Axiom.
- Enter a descriptive name.
- Select Audit logs, Key verifications, Gateway HTTP requests, Runtime logs, or Rate limits in Stream.
- Optional: Select the stream filters described below.
- Enter the destination settings.
- Click Create Log Drain.
Filter a stream
Leave a filter empty to send all values for that filter.
For audit logs, select Specific event types to open the category tree.
Categories use the event prefix, such as
key. Expand a category to select
individual actions, shown with their full event type, such as key.create.
Checking a category selects all its listed actions, even when search hides some
actions. All event types also includes event types added later.
When you select both keyspaces and outcomes, a verification must match both
filters. Unspecified matches an empty verification outcome.
For gateway requests and runtime logs, All sources sends events from all
sources in the workspace, including future resources. HTTP status and severity
filters still apply. Select Specific sources to restrict resources.
Expand a project in Sources to view
its apps and environments. Check a project or app to select its environments.
A partially checked parent has some, but not all, environments selected. Search
changes the visible rows, not the scope of a project or app checkbox.
Selections consisting entirely of whole projects or whole apps include future
environments in those resources. Mixed selections are saved as specific
environment IDs and do not automatically include future environments.
Select all checks the current tree and keeps Specific sources selected.
Clear all requires you to select at least one source before saving, or switch
to All sources. Switching modes retains your selections until you close the
form. Only Specific sources applies those selections when saving.
If saved source filters contain unavailable resources, changing the tree opens
Replace source filters?. Review the listed filters before confirming their
replacement. Keep current filters cancels that tree change. Editing the drain’s
name or destination alone preserves unavailable source filters.
Gateway requests must match the selected sources and HTTP status classes.
For example, select a production environment and 5xx to send its server errors.
Status filters select whole classes only. Exact codes and arbitrary ranges are
not supported.
Filter changes apply from the drain’s committed position. They do not replay
events skipped by an earlier filter. An in-flight request can still arrive
with the previous filters. To export another stream, create a separate drain.
Runtime filters follow the same resource selection rules. A log must match every
nonempty filter and any selected value within each filter. Severity matching is
exact, not a minimum threshold.
Rate-limit decisions must match every nonempty filter and any selected value
within each filter. For example, select the payments namespace and Blocked
to export only blocked checks in that namespace. Clearing Results includes
both passed and blocked checks.
Configure an HTTP destination
An HTTP log drain sends aPOST request to your endpoint for each batch. The
endpoint must use HTTPS and resolve to a public IP address. Unkey rejects
loopback, private, and link-local destinations. Do not include credentials in
the endpoint URL. Add authentication as a custom header.
Configure these fields in the dashboard:
Unkey stores the endpoint URL in plaintext. It encrypts each custom header
value at rest.
Handle HTTP requests
Process the complete request before you acknowledge it.- Accept
POSTrequests at the configured path. - Read the body according to the
Content-Typeheader. - Process every event in the batch.
- Return a
2xxstatus only after you accept the complete batch. - Return a non-
2xxstatus and a useful response body when processing fails.
To request a longer retry delay, return a standard
Retry-After header. Set
the value to a number of seconds or an HTTP date. Unkey applies the requested
delay when it is longer than the standard retry delay. The maximum requested
delay is 24 hours.
Read the HTTP payload
Choose the format that matches your destination. These examples contain an audit event. Each flat record includesstream, time, and the stream’s domain
fields. time is the occurrence instant, not insertion or delivery time, rendered
as an RFC 3339 UTC string with fixed millisecond precision. There is no event
wrapper or outer timestamp field.
occurred_at matches time.
Read a key verification payload
For a verification drain, each record contains the verification fields.
NDJSON uses the same record shape, with one record per line. Axiom uses the
same fields, with an additional
_time equal to time.
Read a gateway request payload
For a gateway request drain, each record contains these fields alongsidestream and time. The request, response, and latency objects stay nested.
Captured details follow your logging policy. The drain forwards stored values,
including any redaction already applied when the request was logged. Fields
that were not captured remain empty. Internal routing addresses are not exported.
Gateway batches use a conservative 16 MiB serialized-size budget in addition to
the drain’s event-count limit. A batch can contain fewer events when it reaches
that budget. Unkey continues from the last delivered event without truncating
captured fields. If one event, or a group with an identical cursor, cannot fit,
delivery stops before it. The cursor stays unchanged, and the drain follows its
retry and automatic-pause policy. Contact support to investigate an oversized
event before the logs expire.
HTTP JSON sends an array of flat records with
stream: "gateway_requests"
and time. NDJSON sends one such record per line. Axiom adds _time equal to time.
Only requests recorded in gateway request logs are eligible. Requests that do
not reach a deployment instance or are excluded from logging are not exported.
Read a runtime log payload
Runtime drains forward collected application output, including multiline messages and nested structured attributes. HTTP JSON sends an array such as:_time equal to time.
Messages and attributes contain the application’s stored output. Do not log
secrets that you do not want forwarded. Kubernetes pod names and platform metadata
are not exported. Runtime retention is unchanged: rows become eligible for deletion
seven days after their expires_at value, which defaults to 90 days after insertion.
ClickHouse deletes whole parts, so deletion can occur later.
Read a rate-limit payload
Rate-limit drains export recorded decisions fromratelimit.limit and
ratelimit.multiLimit, not audit events for configuration changes.
HTTP JSON sends an array such as:
time field is the recorded occurrence time, not the insertion
cursor or delivery time. reset_at is an absolute Unix timestamp in
milliseconds. tokens is the requested cost, including blocked checks.
override_id is present only when an override was used. Latency is not exported.
NDJSON sends the same record on one line. Axiom adds _time equal to time.
Identifiers are forwarded as stored and can contain sensitive customer data.
The drain cursor uses insertion time and request ID. If a batch ends between
multi-limit rows that share both values, the remaining tied rows can be skipped.
The stream includes only completed API decisions. Failed requests and
single-limit requests with X-Unkey-Metrics: disabled are not recorded.
Gateway policy decisions and SDK cache hits that do not call these API endpoints
are not included. Stored rate-limit logs become eligible for deletion one
calendar month after their occurrence time. Deletion is asynchronous, and
drains cannot recover logs that have expired.
Configure an Axiom destination
An Axiom log drain sends Axiom-compatible NDJSON to one dataset.- Create or select a dataset in Axiom.
- Create an Axiom API token with ingest permission for that dataset.
- In Unkey, select Axiom as the destination.
- Enter the dataset name and API token.
- Click Create Log Drain.
_time, which matches
time and tells Axiom when the event occurred. Unkey encodes each row
as one NDJSON line. This example is formatted across lines for readability:
Retry-After or X-RateLimit-Reset to request a longer retry
delay. Set Retry-After to a number of seconds or an HTTP date. Set
X-RateLimit-Reset to a Unix timestamp in seconds. Unkey applies the requested
delay when it is longer than the standard retry delay. The maximum requested
delay is 24 hours.
Unkey encrypts the Axiom token before storing it.
Retries
Unkey retries failed deliveries. This includes4xx responses, 5xx responses,
timeouts, DNS failures, and connection failures. A failed delivery does not
skip the affected events.
The first retry waits 1 minute. The delay doubles after each failure, with a
maximum wait of 4 hours between attempts. After 50 consecutive failures, Unkey
pauses the log drain. Retries continue for about 7 days.
A successful delivery resets the failure count. Changing the destination or
filters, or resuming the drain, also resets the failure count. Delivery
continues from the last successful batch.
Debug a log drain
Open a log drain from Settings > Log Drains. The detail page shows the drain’s status and these metrics for the past 24 hours:- Delivered, successful delivery attempts.
- Failed, attempts that did not return 2xx.
- Success rate, the share of attempts that succeeded.
- Last delivery, time since the last successful delivery.