Setting up fluentd to Parse Nested JSON from Docker

As I wrote about in Migrating Your Spring Boot Application to use Structured Logging, structured logging is pretty great.

I've recently set up fluentd for Logz.io on Kubernetes, via their super handy configuration, but wanted to make it work for the services I've got that produce JSON logs.

As these are running on Docker, the default fluentd configuration worked to pick up the logs themselves, it doesn't auto-parse the logs themselves as JSON.

Fortunately okkez's answer on StackOverflow managed to solve this for me, with the following addition to the configuration.

     # diff is based from https://github.com/logzio/logzio-k8s/blob/c9fbf432e5aeeee333717b0c414487c55043a28b/configmap.yaml

     # This adds type to the log && change key log to message
     <filter **>
       @type record_modifier
       <record>
         type  k8s
         message ${record["log"]}
       </record>
       remove_keys log
     </filter>

+   # https://stackoverflow.com/a/56388980
+   <filter **>
+     @type parser
+     key_name message
+     reserve_data true
+     remove_key_name_field true
+     <parse>
+       @type multi_format
+       <pattern>
+         format json
+       </pattern>
+       <pattern>
+         format none
+       </pattern>
+     </parse>
+   </filter>

This now allows Logz.io to show a full log message, with top-level keys parsed into the log message itself.

And because we're using the Multi format parser plugin, we're able to handle both plain-formatted logs and JSON ones, which is useful for cases that we've got a mix of log formats across our services being monitored.

Written by Jamie Tanna's profile image Jamie Tanna on , and last updated on .

Content for this article is shared under the terms of the Creative Commons Attribution Non Commercial Share Alike 4.0 International, and code is shared under the Apache License 2.0.

#blogumentation #fluentd #logs #docker #kubernetes #logz.io.

This post was filed under articles.

Interactions with this post

Interactions with this post

Below you can find the interactions that this page has had using WebMention.

Have you written a response to this post? Let me know the URL:

Do you not have a website set up with WebMention capabilities? You can use Comment Parade.