> ## Documentation Index
> Fetch the complete documentation index at: https://docs.dnzwithazee.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Log push

> Stream DNZ's query log, audit log or DNZ Proxy's request log to a SIEM, Grafana Loki, S3-compatible storage or syslog as it is written, one JSON object per line.

DNZ can send its **query log**, its **audit log** or **DNZ Proxy's request log** somewhere else as
it is written, like Cloudflare Logpush: an HTTP endpoint (a SIEM, Vector, Fluent Bit, Logstash,
Splunk), Grafana Loki, S3-compatible storage or syslog. Set it up under **Settings → Integrations →
Log push** (superadmins). On a `--proxy-only` server only DNZ Proxy's request log can be pushed.

## What is sent

```json Query log theme={null}
{"dataset":"query_log","id":205840000000123,"time":"2026-10-28T09:15:02Z","server":"ns1","name":"www.example.com","type":"A","client":"10.0.4.21","rcode":"NOERROR","source":"cache","duration_ms":0,"zone_id":null,"division_id":2}
```

```json Audit log theme={null}
{"dataset":"audit_log","id":5123,"time":"2026-10-28T09:16:40Z","user":"ana","division":"IT","action":"record.update","target":"www.example.com A","details":{},"ip":"10.0.4.7"}
```

```json DNZ Proxy request log theme={null}
{"dataset":"proxy_log","id":48213,"time":"2026-11-05T09:15:02.481Z","proxy":"Edge","node":"jkt-1","host":"shop.example.com","method":"GET","path":"/search?q=1","status":403,"duration_ms":1,"bytes_in":0,"bytes_out":1520,"client_ip":"203.0.113.9","country":"ID","user_agent":"curl/8.5","referer":"","router":"shop","upstream":"","cache":"","firewall":"block waf:sqli-union","tls_version":"TLSv1.3","http_version":"HTTP/2.0","request_id":"8f2a1b3c4d5e6f70"}
```

A query log job can take only some answers: local, forwarded, cached, blocked, error.

## DNZ Proxy's request log

**The nodes push their own lines.** The log stays on each node and nothing goes through DNZ, so proxy
traffic never loads it. A job goes to the nodes of the proxies it names (none named: every DNZ Proxy)
with their configuration; each node reads its own log from a cursor it keeps, batches and sends.

> [!WARNING]
>
> Each node has to reach the destination itself, and `localhost` is each node's own machine. **Test**
> sends its line from DNZ, so a node that can't reach the destination shows on the job's row, not there.

A job's passwords and keys reach the node over its authenticated stream and stay in its memory: they are
never written to its state directory. A node that restarts while DNZ is away keeps serving and logging,
and pushes again once it is connected, from where it stopped.

| Lines | What | What it costs |
| - | - | - |
| **Security events only** | Lines the WAF or a firewall rule matched: blocks, challenges, log-only matches | Next to nothing: the nodes log these anyway |
| **What the nodes log** | The sample each node keeps: up to the proxy's *Request log lines per second*, with ten times the room for errors, blocks and challenges | Next to nothing: the same lines, sent on |
| **Every request** | A line for every visit, past the proxy's budget | **It costs.** Every request is written to the node's database and sent on: in our tests about 27 µs more CPU per request on one CPU (a fifth fewer requests a second at the most a node handles), about 230 bytes of disk per line and up to 25 MiB more memory; at 5,000 requests a second the slowest 1% took 2.7 ms instead of 1 ms. Its log grows with its traffic for as long as logs are kept. Off by default |

A push sends at most 50,000 lines, so with **Every request** the job's interval decides how much traffic it keeps up
with: 1,666 requests a second at the default 30 seconds, 10,000 at 5 seconds. Measurements:
[Proxy benchmarks](https://docs.dnzwithazee.com/proxy/benchmarks.md#what-each-feature-costs).

While a proxy has no such job a visit costs nothing more. How it goes shows on the job's row: how many
of its nodes push, fail or don't report, the longest lag and the last error, and **Nodes** lists them.
**Nodes older than this version skip the job**, serving and logging as before: the row names them, so you
know which to [update](https://docs.dnzwithazee.com/proxy/dnz-proxy.md).

## Destinations

> [card] HTTP
>
> The batch as NDJSON (optionally gzipped) with your headers. Vector, Fluent Bit, Logstash and
> Splunk HEC (`/services/collector/raw`) take it.

> [card] Grafana Loki
>
> `/loki/api/v1/push` with labels `job=dnz`, `dataset`, `server` and yours; tenant and basic auth.

> [card] S3-compatible
>
> One gzipped NDJSON object per batch, by day: AWS S3, Cloudflare R2, Backblaze B2, MinIO.

> [card] Syslog
>
> RFC 5424 over UDP, or TCP with octet counting; the JSON is the message.

> [!WARNING]
>
> `localhost` or `127.0.0.1` is DNZ's own machine (in Docker, its container). Use the address DNZ
> reaches the destination on; the form warns otherwise.

## Try it with Vector

**1. Run a receiver**

```yaml vector.yaml theme={null}
sources:
  dnz:
    type: http_server
    address: 0.0.0.0:9000
    decoding: { codec: json }
    framing: { method: newline_delimited }
sinks:
  out:
    type: console
    inputs: [dnz]
    encoding: { codec: json }
```

```bash theme={null}
docker run -d --name vector -p 9000:9000 -v $PWD/vector.yaml:/etc/vector/vector.yaml:ro timberio/vector:0.41.1-alpine
```

**2. Point a job at it**

```bash theme={null}
dnz logpush add try-it --dataset query-log --to http --url http://<that machine>:9000/
```

`add` sends a test line right away; `docker logs -f vector` shows the queries as they come.

## Delivery

* Every 30 seconds by default (5 to 3600), in batches of 5,000 lines.
* A new job starts with what is logged from then on.
* Each job keeps a cursor: a push that fails is sent again from the same place, so nothing is lost
  or sent twice while the log still has it.
* Every server pushes its own query log (`"server"` tells them apart); only the cluster's leader
  pushes the shared audit log.
* Every DNZ Proxy node pushes its own request log, with its own cursor. As a Loki `server` label, in an
  S3 object's name and as the syslog host it is `<proxy>-<node>`.

## CLI

```bash theme={null}
dnz logpush ls
dnz logpush add siem --dataset query-log --to http --url https://siem.example.com/ingest \
  --header "Authorization=Bearer …" --source blocked
dnz logpush add loki --dataset audit-log --to loki --url http://loki:3100 --label env=prod
DNZ_LOGPUSH_SECRET_KEY=… dnz logpush add archive --to s3 --endpoint https://s3.eu-central-1.amazonaws.com \
  --bucket logs --prefix dnz --access-key AKIA…
dnz logpush add syslog --to syslog --address 10.0.0.5:514 --protocol tcp
dnz logpush add waf --source proxy --security-only --to http --url https://siem.example.com/ingest
dnz logpush add edge --source proxy --proxy Edge --every-request --to loki --url http://loki:3100
dnz logpush nodes waf
dnz logpush test siem
dnz logpush disable siem
```

`dnz_logpush_lines_total` and `dnz_logpush_failures_total` are on the [metrics](https://docs.dnzwithazee.com/dashboard/prometheus.md)
endpoint, per server and with `{source="proxy",job,proxy,node}` per node, with
`dnz_logpush_lag_seconds`, `dnz_logpush_last_push_timestamp_seconds` and `dnz_logpush_node_pushing`.
The MCP tool `logpush_jobs` lists the jobs with what their nodes report.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.