Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DOC-11515 PR #132742 - logconfig: update file sink log validation config #19053

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion src/current/v24.3/configure-logs.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ All supported sink types use the following common sink parameters:
| `redact` | When `true`, enables automatic redaction of personally identifiable information (PII) from log messages. This ensures that sensitive data is not transmitted when collecting logs centrally or over a network. For details, see [Redact logs](#redact-logs). |
| `redactable` | When `true`, preserves redaction markers around fields that are considered sensitive in the log messages. The markers are recognized by [`cockroach debug zip`]({% link {{ page.version.version }}/cockroach-debug-zip.md %}) and [`cockroach debug merge-logs`]({% link {{ page.version.version }}/cockroach-debug-merge-logs.md %}) but may not be compatible with external log collectors. For details on how the markers appear in each format, see [Log formats]({% link {{ page.version.version }}/log-formats.md %}). |
| `exit-on-error` | When `true`, stops the Cockroach node if an error is encountered while writing to the sink. We recommend enabling this option on file sinks in order to avoid losing any log entries. When set to `false`, this can be used to mark certain sinks (such as `stderr`) as non-critical. |
| `auditable` | If `true`, enables `exit-on-error` on the sink. Also disables `buffered-writes` if the sink is under `file-groups`. This guarantees [non-repudiability](https://wikipedia.org/wiki/Non-repudiation) for any logs in the sink, but can incur a performance overhead and higher disk IOPS consumption. This setting is typically enabled for [security-related logs]({% link {{ page.version.version }}/logging-use-cases.md %}#security-and-audit-monitoring). |
| <a id="auditable"></a>`auditable` | If `true`, enables `exit-on-error` on the sink. Also disables `buffered-writes` if the sink is under `file-groups`. This guarantees [non-repudiability](https://wikipedia.org/wiki/Non-repudiation) for any logs in the sink, but can incur a performance overhead and higher disk IOPS consumption. This setting is typically enabled for [security-related logs]({% link {{ page.version.version }}/logging-use-cases.md %}#security-and-audit-monitoring).<br><br>File-based audit logging cannot coexist with the buffering configuration, so disable either [`buffering`](#file-buffering) or `auditable`. |

If not specified for a given sink, these parameter values are inherited from [`file-defaults`](#set-file-defaults) (for file sinks), [`fluent-defaults`](#set-fluentd-defaults) (for Fluentd sinks), and [`http-defaults`](#set-http-defaults) (for HTTP sinks).

Expand Down Expand Up @@ -153,6 +153,7 @@ Along with the [common sink parameters](#common-sink-parameters), each file grou
| `max-group-size` | Approximate maximum combined size of all files to be preserved for this sink. Configures the maximum size for a logging group (for example, `cockroach`, `cockroach-sql-audit`, `cockroach-auth`, `cockroach-sql-exec`, `cockroach-pebble`), after which the oldest log file is deleted. An asynchronous garbage collection removes files that cause the file set to grow beyond this specified size. Accepts a valid file size, such as `1GiB`.<br/><br/> For high-traffic deployments, or to ensure log retention over longer periods of time, consider raising this value to `500MiB` or `1GiB`.<br/><br/>**Default:** `100MiB` |
| `file-permissions` | The `chmod`-style permissions on generated log files, formatted as a 3-digit octal number. The executable bit must not be set. <br><br>**Default:** `640` (readable by the owner or members of the group, writable by the owner). |
| `buffered-writes` | When `true`, enables buffering of writes. Set to `false` to flush every log entry (i.e., propagate data from the `cockroach` process to the OS) and synchronize writes (i.e., ask the OS to confirm the log data was written to disk). Disabling this setting provides [non-repudiation](https://wikipedia.org/wiki/Non-repudiation) guarantees, but can incur a performance overhead and higher disk IOPS consumption. This setting is typically disabled for [security-related logs]({% link {{ page.version.version }}/logging-use-cases.md %}#security-and-audit-monitoring). |
| <a id="file-buffering"></a>`buffering` |`buffering` is disabled by default for file log sinks. Default: `buffering: NONE`. Note that enabling asynchronous buffering of file log sinks is in [preview]({% link {{ page.version.version }}/cockroachdb-feature-availability.md %}#features-in-preview).<br><br>To configure buffering of log messages for the sink, use the following sub-parameters:<br><br><ul><li>`max-staleness`: The maximum time a log message will wait in the buffer before a flush is triggered. Set to `0` to disable flushing based on elapsed time.</li><li>`flush-trigger-size`: The number of bytes that will trigger the buffer to flush. Set to `0` to disable flushing based on accumulated size.</li><li>`max-buffer-size`: The maximum size of the buffer: new log messages received when the buffer is full cause older messages to be dropped.</li></ul>When `max-staleness` and `flush-trigger-size` are used together, whichever is reached first will trigger the flush. This setting is typically disabled for [security-related logs]({% link {{ page.version.version }}/logging-use-cases.md %}#security-and-audit-monitoring). For a usage example, refer to [Enable WAL failover]({% link {{ page.version.version }}/cockroach-start.md %}#enable-wal-failover).<br><br>File-based audit logging cannot coexist with this buffering configuration, so disable either `buffering` or [`auditable`](#auditable). |

If not specified for a given file group, the parameter values are inherited from [`file-defaults`](#configure-logging-defaults) (except `channels`, which uses the [default configuration](#default-logging-configuration)).

Expand Down
Loading