# Configuration

BanyanD is the BanyanDB server. There are two ways to configure BanyanD: using a bootstrap flag or using environment variables. The environment variable name has a prefix `BYDB_` followed by the flag name in uppercase. For example, the flag `--port` can be set using the environment variable `BYDB_PORT`.

> BanyanDB supports both absolute and relative paths for directory and file configurations (such as data directories, TLS certificates, keys, etc.). Relative paths are resolved against the current working directory where the BanyanD process is started. Available as of v0.10.0.

## Commands

### Bootstrap commands

There are three bootstrap commands: `data`, `liaison`, and `standalone`. You could use them to boot different roles of BanyanD.

- `data`: Run as the data server. It stores the data and processes the data requests.
- `liaison`: Run as the liaison server. It is responsible for the communication between the data servers and clients.
- `standalone`: Run as the standalone server. It combines the data and liaison server for development and testing.

### Other commands

- `completion`: Generate the autocompletion script for the specified shell.
- `help`: Help about any command.

## Flags

Below are the available flags for configuring BanyanDB:

### Service Discovery

BanyanDB supports three node-discovery modes: `none` (default, standalone only), `dns` (DNS SRV based), and `file` (static YAML list). For conceptual details, TLS configuration, cluster bring-up, and operational guidance, see the [node discovery documentation](node-discovery.md).

- `--node-discovery-mode string`: Node discovery mode: `none` (default, standalone), `dns`, or `file`.
- `--node-registry-timeout duration`: Timeout for the node registry (default: 2m).
- `--node-discovery-grpc-timeout duration`: Timeout for gRPC calls to fetch node metadata from discovered nodes.

#### DNS Mode Flags

- `--node-discovery-dns-srv-addresses strings`: Comma-separated DNS SRV addresses to query (e.g., `_grpc._tcp.banyandb.default.svc.cluster.local`).
- `--node-discovery-dns-fetch-init-interval duration`: Query interval during the initialization phase (default: 5s).
- `--node-discovery-dns-fetch-init-duration duration`: How long the initialization phase lasts before falling back to the steady-state interval (default: 5m).
- `--node-discovery-dns-fetch-interval duration`: Query interval after the initialization phase (default: 15s).
- `--node-discovery-dns-tls`: Enable TLS for DNS discovery gRPC connections.
- `--node-discovery-dns-ca-certs strings`: Comma-separated CA certificate files, one per SRV address in the same order, used to verify DNS-discovered nodes.

#### File Mode Flags

- `--node-discovery-file-path string`: Path to the static YAML file that lists cluster nodes (required when the mode is `file`).
- `--node-discovery-file-fetch-interval duration`: Polling interval to reread the discovery file as a fallback to fsnotify-based reloads (default: 5m).
- `--node-discovery-file-retry-initial-interval duration`: Initial retry delay for nodes whose metadata fetch failed (default: 1s).
- `--node-discovery-file-retry-max-interval duration`: Upper bound for the retry backoff (default: 2m).
- `--node-discovery-file-retry-multiplier float`: Multiplicative factor applied between retries (default: 2.0).

#### Node Host Registration

Each node advertises itself with a host part resolved according to `--node-host-provider`:

`node-host-provider`: the node host provider, can be "hostname", "ip" or "flag", default is hostname.

If the `node-host-provider` is "flag", you can use `node-host` to configure the node host:

```sh
./banyand liaison --node-host=foo.bar.com --node-host-provider=flag
```

If the `node-host-provider` is "hostname", BanyanDB will use the hostname of the server as the node host. The hostname is parsed from the go library `os.Hostname()`.

If the `node-host-provider` is "ip", BanyanDB will use the IP address of the server as the node host. The IP address is parsed from the go library `net.Interfaces()`. BanyanDB will use the first non-loopback IPv4 address as the node host.

The official Helm chart uses the `node-host-provider` as "ip" as the default value.

### Liaison & Network

BanyanDB uses gRPC for communication between the servers. The following flags are used to configure the network settings.

- `--grpc-host string`: The host BanyanDB listens on.
- `--grpc-port uint32`: The port BanyanDB listens on (default: 17912).
- `--http-grpc-addr string`: HTTP server redirects gRPC requests to this address (default: "localhost:17912").
- `--http-host string`: Listen host for HTTP.
- `--http-port uint32`: Listen port for HTTP (default: 17913).
- `--max-recv-msg-size bytes`: The size of the maximum receiving message (default: 10.00MiB).

The following flags are used to configure access logs for the data ingestion:

- `--access-log-root-path string`: Access log root path.
- `--enable-ingestion-access-log`: Enable ingestion access log.
- `--access-log-sampled`: if true, requests may be dropped when the channel is full; if false, requests are never dropped

The following flags are used to configure the timeout of data sending from liaison to data servers:

- `--stream-write-timeout duration`: Stream write timeout (default: 1m).
- `--measure-write-timeout duration`: Measure write timeout (default: 1m).
- `--trace-write-timeout duration`: Trace write timeout (default: 1m).

The following flags tune the BydbQL prepared-statement cache on the query path. The cache stores parsed statements keyed by query text so repeated, parameterized (`?`) queries skip re-parsing; literal queries without placeholders are never cached. It is bounded by both an entry count and the estimated in-memory size of the cached statements, evicting least-recently-used entries when either bound is exceeded. Effectiveness is observable via the `bydbql_prepared_cache_*` metrics (a `hit`/`miss`/`bypass` counter plus hit-ratio, entry-count, and byte-size gauges).

- `--bydbql-prepared-cache-size int`: Max number of prepared BydbQL statements cached on the query path; `0` disables the cache (default: 4000).
- `--bydbql-prepared-cache-max-bytes int`: Max total estimated size (in bytes) of the cached prepared statements; `0` removes the byte bound (default: 10485760, i.e. 10MiB).

These flags surface the queries behind cache misses and slow responses without exposing high-cardinality query text as metric labels: Prometheus gets only two counters (`bydbql_prepared_cache_total{result="miss"}` and `bydbql_slow_query_total`), while the specific hot queries are logged.

- `--bydbql-slow-query-threshold duration`: End-to-end latency above which a BydbQL query is counted as slow (increments `bydbql_slow_query_total`) and tracked in the slow-query top-K; `0` disables slow-query tracking (default: `1s`).
- `--bydbql-topk-log-interval duration`: How often to log the hottest cache-miss and slow queries. The cache-miss list holds only templates that were compiled more than once — either evicted and compiled again (thrashing), or too large for the byte bound and therefore re-compiled on every single request; a template's unavoidable first-ever compile is excluded at the source, so every entry is actionable. The slow-query list is ranked by peak latency (`max_latency`) so a rarely-but-catastrophically slow query is not buried under frequently-mildly-slow ones. Uses a bounded approximate heavy-hitters tracker (128 entries), so it costs O(1) and never grows unbounded; `0` disables the top-K log (default: `5m`).
- `--bydbql-topk-slow-ttl duration`: Drop a slow-query top-K entry whose query has not been slow again for this long; `0` keeps entries for the process lifetime (default: `24h`).
- `--bydbql-topk-reparse-ttl duration`: Drop a cache-miss top-K entry whose template has not been re-parsed again for this long; `0` keeps entries for the process lifetime (default: `24h`).
- `--bydbql-topk-param-mode string`: How much of a slow query's bound parameters to include in the slow-query top-K log (default: `fingerprint`). The reported sample is that of the **most recent** slow occurrence of the template, logged as `last_params` — it is not necessarily the occurrence that produced `max_latency`. Numeric, timestamp and null parameters are always rendered verbatim in every mode above `none`: window width, `LIMIT` and thresholds are what explain a slow query, and they carry no user-identifying content. Only `str` and `str_array` parameters are subject to the mode; binary is always digested, never rendered verbatim:
  - `none` — omit parameters entirely. The log line carries no `last_params` field at all. Choose this when the log destination is untrusted or when any parameter metadata is unacceptable; you lose the ability to tell which parameters a slow query ran with.
  - `fingerprint` (default) — report each string's length plus a stable digest (`str(len=12):fp=1a2b3c4d`). The value itself never appears. The same value digests identically across restarts and across liaison nodes, which is what distinguishes "one particular service is always slow" from "every service is slow". The digest is **unsalted**: it hides values from a casual reader of the log, but a low-cardinality value such as a service name can be recovered by digesting a candidate dictionary. Do not treat it as a cryptographic guarantee.
  - `raw` — log string values verbatim (`"checkout-svc"`). This is the only mode that reveals parameter values. Enable it only when the log destination is trusted, and prefer turning it back off once the investigation is done; parameters may contain user data such as trace IDs or account identifiers. Binary parameters are digested even in this mode — raw bytes would corrupt the log line and carry no diagnostic value beyond their size.

  In short: `none` reveals nothing, `fingerprint` reveals *whether two slow queries used the same value*, `raw` reveals *the value*. Non-string parameters (numeric, timestamp, null) render verbatim in `fingerprint` and `raw` alike, because window width, `LIMIT` and thresholds are what explain a slow query and carry no user-identifying content.

  Array parameters (`IN (?)` bound to a `str_array`/`int_array`) render at most 8 elements plus a `+N more` marker, so a large value set cannot bloat the log line. If you need the complete, unredacted parameters of a query, enable the query access log (`--enable-query-access-log`) instead: it already records every request in full, in its own file, with its own retention and permissions.

Each logged entry carries `last_seen`, and slow entries additionally carry `max_latency_at`. Both trackers accumulate, so `max_latency` is a running peak that can long outlive the incident that produced it: without these timestamps a one-off startup spike keeps being reported as if it were current, and a reader cannot tell a live problem from a stale one. The TTLs bound how long that can happen — an entry whose query stops recurring disappears — while `max_latency_at` dates the peak itself for entries that do keep recurring.

Keep each TTL comfortably above `--bydbql-topk-log-interval`. An entry is only ever reported by a dump, so a TTL shorter than the interval lets an entry expire in the gap between two dumps and never be logged at all — the tracker would go quiet not because nothing is wrong but because it forgets faster than it reports. The defaults (`24h` against `5m`) leave a wide margin.

#### Diagnosing an ineffective BydbQL cache

A cache hit costs about 25 ns and 0 allocations, so the cache is effectively free when it works. When it does not, every request re-parses the query — for a wide trace/log query that is roughly **220 µs and ~147 KB / ~2,100 allocations per request** (about four orders of magnitude more CPU and heap churn than a hit). Watch these signals; several going abnormal together points at an ineffective cache:

- **`bydbql_prepared_cache_hit_ratio` drops toward 0** and the **`miss` rate of `bydbql_prepared_cache_total{result="miss"}` climbs** — the primary indicator.
- **BydbQL query latency rises** (the liaison gRPC `query` latency series) because the parse cost is added back to every request.
- **Process CPU and Go GC pressure rise** — each miss allocates a fresh parse tree, so the allocation rate, GC frequency, GC pause, and GC CPU fraction all increase.
- **`bydbql_prepared_cache_count` / `bydbql_prepared_cache_bytes` sit pinned at the cap while the miss rate stays high** — the set of distinct query templates exceeds the cache capacity and entries are constantly evicted.
- **The `bypass` rate of `bydbql_prepared_cache_total{result="bypass"}` is high** — many queries are literal (contain no `?`) and are never cached. Bypasses are excluded from the hit ratio.

To find the exact queries behind a high miss/bypass rate, enable the query access log (`--enable-query-access-log`). Each BydbQL entry is tagged with its cache outcome via the log's service field: `bydbql-hit`, `bydbql-miss` (cacheable but not in the cache), and `bydbql-bypass` (literal, never cached). Filtering the `bydbql-query-*` log for `bydbql-miss` / `bydbql-bypass` yields exactly the queries that did not hit the cache, together with their full text.

Remedies:

- **High miss rate with the cache pinned at capacity** → the working set of distinct query templates is larger than the cache. Raise `--bydbql-prepared-cache-size` (and `--bydbql-prepared-cache-max-bytes` if the byte bound binds first) until the hit ratio recovers.
- **High bypass rate (near-empty cache)** → queries are not parameterized. Literal queries (no `?`) are never cached, and inlining values (e.g. `id = 'a' OR id = 'b' …`) explodes the template count. Use the `bydbql-bypass` access-log entries to find them, then parameterize the varying values with `?`, and express value sets as `IN (?)` bound to a `str_array`/`int_array` so a query is one reusable template regardless of the number of values.

### TLS

If you want to enable TLS for the communication between the client and liaison/standalone, you can use the following flags:

- `--tls`: gRPC connection uses TLS if true, else plain TCP.
- `--http-tls`: http connection uses TLS if true, else plain HTTP.
- `--key-file string`: The TLS key file.
- `--cert-file string`: The TLS certificate file.
- `--http-grpc-cert-file string`: The gRPC TLS certificate file if the gRPC server enables TLS. It should be the same as the `cert-file`.
- `--http-key-file string`: The TLS key file of the HTTP server.
- `--http-cert-file string`: The TLS certificate file of the HTTP server.

#### Internal queue TLS (Liaison ↔ Data)

Enable TLS on the internal gRPC queue that the Liaison uses to push data into every Data‑Node:

- `--internal-tls`: enable TLS on the queue client inside Liaison; if false the queue uses plain TCP.
- `--internal-ca-cert <path>`: PEM‑encoded CA (or bundle) that the queue client uses to verify Data‑Node server certificates.

#### Server certificates

Each Liaison/Data process still advertises its certificate with the public flags shown above (`--tls`, `--cert-file`, `--key-file`).
The same certificate/key pair can be reused for both external traffic and the internal queue.

### Data & Storage

The following flags are used to configure the measure storage engine:

- `--measure-flush-timeout duration`: The memory data timeout of measure (default: 5s).
- `--measure-root-path string`: The root path of the measure database (default: "/tmp").
- `--measure-data-path string`: The data directory path of measure. If not set, `<measure-root-path>/measure/data` is used.
- `--measure-max-fan-out-size bytes`: the upper bound of a single file size after merge of measure (default 8.00EiB)
- `--measure-lifecycle-receive-mem-wait-timeout duration`: Max time a lifecycle-migration receiver waits for memory to recover before introducing an external segment (default: 5m).

The following flags are used to configure the stream storage engine:

- `--stream-flush-timeout duration`: The memory data timeout of stream (default: 1s).
- `--stream-root-path string`: The root path of the stream database (default: "/tmp").
- `--stream-data-path string`: The data directory path of stream. If not set, `<stream-root-path>/stream/data` is used.
- `--stream-max-fan-out-size bytes`: the upper bound of a single file size after merge of stream (default 8.00EiB)
- `--element-index-flush-timeout duration`: The element index timeout of stream (default: 1s).
- `--stream-lifecycle-receive-mem-wait-timeout duration`: Max time a lifecycle-migration receiver waits for memory to recover before introducing an external segment (default: 5m).

The following flags are used to configure the trace storage engine:

- `--trace-flush-timeout duration`: The memory data timeout of trace (default: 1s).
- `--trace-root-path string`: The root path of the database (default: "/tmp").
- `--trace-max-fan-out-size bytes`: the upper bound of a single file size after merge of trace (default 8.00EiB)
- `--trace-lifecycle-receive-mem-wait-timeout duration`: Max time a lifecycle-migration receiver waits for memory to recover before introducing an external segment (default: 5m).

The following flags configure the remaining per-catalog storage roots:

- `--property-root-path string`: The root path of the property database (default: "/tmp").
- `--trace-root-path string`: The root path of the trace database (default: "/tmp").

The following flags are used to configure the memory protector:

- `--allowed-bytes bytes`: Allowed bytes of memory usage. If the memory usage exceeds this value, the query services will stop. Setting a large value may evict data from the OS page cache, causing high disk I/O. (default 0B)
- `--allowed-percent int`: Allowed percentage of total memory usage. If usage exceeds this value, the query services will stop. This takes effect only if `allowed-bytes` is 0. If usage is too high, it may cause OS page cache eviction. (default 75)

### Snapshot Retention

Each service supports a minimum snapshot age configuration to prevent recently created snapshots from being deleted prematurely:

- `--measure-min-file-snapshot-age duration`: Minimum age for measure snapshots (default: 1h).
- `--stream-min-file-snapshot-age duration`: Minimum age for stream snapshots (default: 1h).
- `--trace-min-file-snapshot-age duration`: Minimum age for trace snapshots (default: 1h).
- `--property-min-file-snapshot-age duration`: Minimum age for property snapshots (default: 1h).
- `--schema-server-min-file-snapshot-age duration`: Minimum age for schema server snapshots (default: 1h).

During normal snapshot cleanup (triggered when a new snapshot is created), a snapshot is only deleted if it is older than the configured minimum age **and** the total snapshot count exceeds `*-max-file-snapshot-num` (default: 10). This ensures that recent snapshots remain available for backup and recovery operations.

### Observability

- `--observability-listener-addr string`: Listen address for observability (default: ":2121").
- `--observability-modes strings`: Modes for observability (default: [prometheus]).
- `--pprof-listener-addr string`: Listen address for pprof (default: ":6060").
- `--dst-slow-query duration`: distributed slow query threshold, 0 means no slow query log. This is only used for the liaison server (default: 0).
- `--slow-query duration`: slow query threshold, 0 means no slow query log. This is only used for the data and standalone server (default: 0).

### Schema Registry

BanyanDB stores cluster metadata in a property-based schema registry. Data nodes that carry the meta role host the schema server; every node (data and liaison) runs a schema client that connects to the schema server and keeps its local cache in sync.

- `--schema-registry-mode string`: Schema registry mode (default: "property"). Only `property` is supported.

#### Schema Server (data node)

These flags are only effective on the `data` command and configure the property-based schema server embedded in the data node. They have no effect on liaison nodes.

- `--has-meta-role`: Whether this data node runs the embedded schema server (default: true). Set to false to deploy a pure data node that only serves time-series data and reads schemas from other data nodes. When false, every other flag in this subsection is inert on that node.
- `--schema-server-root-path string`: Root storage path for the schema property data (default: "/tmp").
- `--schema-server-grpc-host string`: Host the schema server listens on.
- `--schema-server-grpc-port uint32`: Port the schema server listens on (default: 17916).
- `--schema-server-flush-timeout duration`: Memory flush interval (default: 5s).
- `--schema-server-expire-delete-timeout duration`: Soft-delete expiration for deleted schemas (default: 168h / 7d).
- `--schema-server-tls`: Enable TLS on the schema server.
- `--schema-server-cert-file string`: The TLS certificate file of the schema server.
- `--schema-server-key-file string`: The TLS key file of the schema server.
- `--schema-server-max-recv-msg-size bytes`: Max gRPC receive message size for the schema server.
- `--schema-server-max-file-snapshot-num int`: Maximum number of file snapshots retained (default: 10).
- `--schema-server-min-file-snapshot-age duration`: Minimum age before a file snapshot is eligible for deletion (default: 1h).

##### Schema repair (gossip)

The following flags tune the gossip-based repair protocol that reconciles schema state between data nodes.

- `--schema-property-repair-trigger-cron string`: Cron expression that triggers a repair gossip round (default: `@every 10m`).
- `--schema-server-repair-tree-slot-count int`: Repair merkle tree slot count (default: 32).
- `--schema-server-repair-build-tree-cron string`: Cron for periodic repair tree rebuilding (default: `@every 1h`).
- `--schema-server-repair-quick-build-tree-time duration`: Minimum delay between on-demand quick repair tree rebuilds (default: 10m).

#### Schema Client (all nodes)

These flags configure how every node talks to the schema server.

- `--schema-property-client-sync-interval duration`: Polling interval for property-based schema sync (default: 30s).
- `--schema-property-client-health-check-interval duration`: Interval for periodic connection health checks to schema servers. Set to 0 to use the default interval (10s), or to a negative value to disable.
- `--schema-property-client-max-recv-msg-size bytes`: Max gRPC receive message size for property schema client.
- `--schema-property-client-tls`: Enable TLS for property schema client connections.
- `--schema-property-client-ca-cert string`: CA certificate file to verify the property schema server.

### Queue client (liaison to data nodes)

These flags configure the connections a liaison holds to the data nodes it queries and writes to. `<prefix>` is `data` or `liaison` depending on which peer set the client serves.

- `--<prefix>-client-health-check-interval duration`: How often to re-check the nodes the client currently considers active, evicting any that no longer answer; `0` disables the periodic check (default: `10s`). The active set is otherwise only validated when a node is admitted and, after that, when a request happens to fail on it — so a node that dies stays routable until some query picks it and pays that query's full timeout to discover it. Keep this well below the distributed query timeouts (`--dst-broadcast-timeout`, default `15s`), since the interval bounds how long a dead node can keep absorbing queries.
- `--<prefix>-client-tls`: Enable client TLS.
- `--<prefix>-client-ca-cert string`: CA certificate file to verify the server.

### Other

- `-n, --name string`: Name of this service.
- `-h, --help`: Help for standalone.
- `--show-rungroup-units`: Show rungroup units.
- `-v, --version`: Version for standalone.

### Global Flags

- `--logging-env string`: The logging environment (default: "prod").
- `--logging-level string`: The root level of logging (default: "info").
- `--logging-levels strings`: The level logging of logging.
- `--logging-modules strings`: The specific module for logging.
- `--node-host string`: The node host of the server, only used when `node-host-provider` is "flag".
- `--node-host-provider nodeIDProvider`: The node host provider, can be hostname, IP, or flag (default: Hostname).

## Example Command

```sh
banyand standalone --logging-env=dev --logging-level=debug --grpc-port=18913
```

If you want to use environment variables to configure BanyanD, you can set the environment variables like this:

```sh
export BYDB_LOGGING_ENV=dev
export BYDB_LOGGING_LEVEL=debug
export BYDB_GRPC_PORT=18913
```

Then you can run BanyanD without any flags:

```sh
banyand standalone
```
