It's critical to monitor what happens within your apps in the modern world. It assists you in monitoring the situation and resolving any issues that may arise. We have an excellent setup of our apps on Google's Kubernetes Engine (GKE), where we have used nginx as a reverse proxy which provides a robust foundation for our applications. However, despite this solid infrastructure, we've faced difficulties in efficiently parsing and comprehending nginx logs. In nginx, “combined” format is used as the original log format if the format is not specified.

Even though this format might be sufficient for simple logging requirements, it may become complicated and challenging to understand, particularly in complex systems. It has been determined that switching to JSON as our logging format is necessary to resolve this problem. We need to switch over to JSON for our logging format to make logs more readable and understandable.
Our goal in moving to JSON format is to increase readability and optimize the logging procedure. JavaScript Object Notation, or JSON, provides an organized and consistent style for event logging that makes it simple to extract and analyze relevant information.

The purpose of this blog is to convert the nginx default log format to JSON format.

Nginx as a reverse proxy:

Your application can use Nginx as a reverse proxy, which will handle incoming requests and route them to the relevant backend services. You can deploy Nginx in your Kubernetes cluster as part of your application deployment or as a stand-alone pod. For several reasons, Kubernetes deployments frequently use Nginx as a reverse proxy. To effectively handle incoming requests and forward them to the appropriate directions, Nginx serves as a middleman between clients and backend services. You can make use of functions like SSL termination, caching, load balancing, and security improvements by implementing Nginx as a reverse proxy. You can also manage traffic flow and optimize resource utilization in your Kubernetes cluster with more flexibility when you use Nginx.

ConfigMap: Configure Nginx Logging

In Kubernetes, a ConfigMap is a resource object that lets you keep configuration data apart from your containerized apps. It's particularly useful for storing non-sensitive & configuration-related data such as environment variables, command-line arguments, or configuration files. You can find more about ConfigMap in my previous blog 8 Key Kubernetes Components You Need to Know for Successful Application Deployment

In our setup, we use ConfigMap to dynamically configure Nginx's logging format. By default, as we know, Nginx logs in the 'combined' format, which provides various details into a single line. However, for better log parsing and understanding, we decided to switch to JSON formatting.

To accomplish this, we specify a unique log format in the Nginx configuration, which is kept in a ConfigMap. This enables us to override the default logging behavior to JSON format.

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-nginx
data:
  app-nginx: |
    log_format main_json escape=json '{'
      '"timestamp":"$time_local",'
      '"remote_addr":"$remote_addr",'
      '"remote_user":"$remote_user",'
      '"request":"$request",'
      '"status":$status,'
      '"body_bytes_sent":$body_bytes_sent,'
      '"http_referer":"$http_referer",'
      '"http_user_agent":"$http_user_agent",'
      '"http_x_forwarded_for":"$http_x_forwarded_for"'
    '}';

    server {
      listen 80 default_server;
      server_name example.dev;

      access_log /var/log/nginx/access.log main_json;
    }

We have configured the access_log directive within the server block of the ConfigMap to ensure that the custom JSON logging format is applied only to the specific server where it is needed.

To implement the updated Nginx configuration, you need to apply the updated Kubernetes ConfigMap into the cluster. Now you can verify whether the new updated configuration has been applied or not by viewing logs from the nginx pod. To view the log you can follow the below commands,

#Get the nginx pod name
kubectl get pods -n namespace

#Vew nginx logs
kubectl logs nginx_pod -n namespace

If the pod has multiple containers then use this command

kubectl logs nginx_pod -n namespace -c nginx_container_name

To get & understand more kubectl commands, please check my previous blog 12 Kubectl Commands to Master Kubernetes Deployments

Conclusion:

We successfully improve logging capabilities in our GKE environment by using ConfigMaps in Kubernetes to change Nginx's default logging configuration. The use of JSON formatting for Nginx logs improves our log readability and helps us to analyze logs more effectively, ultimately contributing to better monitoring and troubleshooting of our containerized applications.

---

I appreciate you taking the time to read this. Your support is much appreciated! If you found this article valuable, please consider clicking the 👉 Follow button and giving it a few claps by clicking the ❤️ like button to help me create more informative content like this. Thank you for your time! 🖤
Also, follow me on Medium, Twitter & LinkedIn.