Splunk logging driver

Estimated reading time: 8 minutes

The splunk logging driver sends container logs to HTTP Event Collector in Splunk Enterprise and Splunk Cloud.

Usage

You can configure Docker logging to use the splunk driver by default or on a per-container basis.

To use the splunk driver as the default logging driver, set the keys log-driver and log-opts to appropriate values in the daemon.json configuration file and restart Docker. For example:

{
  "log-driver": "splunk",
  "log-opts": {
    "splunk-token": "",
    "splunk-url": "",
    ...
  }
}

The daemon.json file is located in /etc/docker/ on Linux hosts or C:\ProgramData\docker\config\daemon.json on Windows Server. For more about configuring Docker using daemon.json, see daemon.json.

Note

log-opts configuration options in the daemon.json configuration file must be provided as strings. Boolean and numeric values (such as the value for splunk-gzip or splunk-gzip-level) must therefore be enclosed in quotes (").

To use the splunk driver for a specific container, use the commandline flags --log-driver and log-opt with docker run:

  docker run --log-driver=splunk --log-opt splunk-token=VALUE --log-opt splunk-url=VALUE ...

Splunk options

The following properties let you configure the splunk logging driver.

• To configure the splunk driver across the Docker environment, edit daemon.json with the key, "log-opts": {"NAME": "VALUE", ...}.
• To configure the splunk driver for an individual container, use docker run with the flag, --log-opt NAME=VALUE ....

If there is collision between the label and env keys, the value of the env takes precedence. Both options add additional fields to the attributes of a logging message.

Below is an example of the logging options specified for the Splunk Enterprise instance. The instance is installed locally on the same machine on which the Docker daemon is running.

The path to the root certificate and Common Name is specified using an HTTPS scheme. This is used for verification. The SplunkServerDefaultCert is automatically generated by Splunk certificates.

$ docker run \
    --log-driver=splunk \
    --log-opt splunk-token=176FCEBF-4CF5-4EDF-91BC-703796522D20 \
    --log-opt splunk-url=https://splunkhost:8088 \
    --log-opt splunk-capath=/path/to/cert/cacert.pem \
    --log-opt splunk-caname=SplunkServerDefaultCert \
    --log-opt tag="{{.Name}}/{{.FullID}}" \
    --log-opt labels=location \
    --log-opt env=TEST \
    --env "TEST=false" \
    --label location=west \
    your/application

The splunk-url for Splunk instances hosted on Splunk Cloud is in a format like https://http-inputs-XXXXXXXX.splunkcloud.com and does not include a port specifier.

Message formats

There are three logging driver messaging formats: inline (default), json, and raw.

The default format is inline where each log message is embedded as a string. For example:

{
    "attrs": {
        "env1": "val1",
        "label1": "label1"
    },
    "tag": "MyImage/MyContainer",
    "source":  "stdout",
    "line": "my message"
}

{
    "attrs": {
        "env1": "val1",
        "label1": "label1"
    },
    "tag": "MyImage/MyContainer",
    "source":  "stdout",
    "line": "{\"foo\": \"bar\"}"
}

Note: If your messages are JSON objects, you may want to embed them in the message we send to Splunk.

To format messages as json objects, set --log-opt splunk-format=json. The driver trys to parse every line as a JSON object and send it as an embedded object. If it cannot parse the message, it is sent inline. For example:

{
    "attrs": {
        "env1": "val1",
        "label1": "label1"
    },
    "tag": "MyImage/MyContainer",
    "source":  "stdout",
    "line": "my message"
}

{
    "attrs": {
        "env1": "val1",
        "label1": "label1"
    },
    "tag": "MyImage/MyContainer",
    "source":  "stdout",
    "line": {
        "foo": "bar"
    }
}

To format messages as raw, set --log-opt splunk-format=raw. Attributes (environment variables and labels) and tags are prefixed to the message. For example:

MyImage/MyContainer env1=val1 label1=label1 my message
MyImage/MyContainer env1=val1 label1=label1 {"foo": "bar"}

Advanced options

Splunk Logging Driver allows you to configure few advanced options by specifying next environment variables for the Docker daemon.