Skip to main content
Hello and welcome back to this lesson. In our previous lesson, we deployed the login application and observed how user interactions on the frontend were accurately recorded in our logs. We also discussed the importance of configuring logging within the application code to capture all relevant details. Now that our application is generating logs, the next step is to use Fluent Bit to transport these logs to Elasticsearch. Fluent Bit offers an efficient method for forwarding logs, and in earlier demonstrations, we configured it for our login app. In this lesson, we will cover how a minimal change in the Fluent Bit configuration allows us to reuse the same setup to collect Python application logs.

​
Verifying the Kubernetes Environment

Before making any changes, ensure that your application is running correctly within your Kubernetes cluster. Verify your pods with:
You should see an output similar to:
Next, verify the services by running:
This command should produce a result like:
Ensure that your Kubernetes cluster is healthy and all related pods are running before applying configuration changes.

​
Reviewing Fluent Bit Deployment Files

Within the Kubernetes deployments directory, you will find several Fluent Bit-related files that were deployed for the login application. These files include:
  • fluent-bit.yaml
  • Service account configuration
  • ConfigMap
  • ClusterRole
  • ClusterRoleBinding
For example, you might see the following files when listing the directory contents:
In this lesson, we will modify a single line in the Fluent Bit configuration file (fluent-bit-config.yaml) to monitor the logs of our Python application.

​
Updating the Fluent Bit Configuration

To adapt Fluent Bit to collect Python application logs, update the [INPUT] section in fluent-bit-config.yaml. Change the Path to point to the log file generated by the Python application as shown below:
This update ensures that Fluent Bit captures the logs using the proper metadata labels for our Python application. Below is the updated configuration file with the corrected settings. Notice that only the [INPUT] section has been changed:
If you need to collect logs from multiple sources within the same namespace, you can simply add another input section without redeploying a separate Fluent Bit instance. For example:
Fluent Bit will automatically detect and process multiple inputs defined in the same configuration file.
When updating your Fluent Bit configuration, always verify that your changes are correctly applied to prevent disruptions in log collection.

​
Deploying the Updated Configuration

After updating the configuration, clear your terminal and run the following command to deploy the changes:
Verify that all pods are running by using:
You should see an output similar to:
Next, check the logs of the Fluent Bit pod to confirm that it is picking up your application logs:
A portion of the log output might look like this:
The Fluent Bit pod is now successfully collecting logs from your Python (or login) application. The subsequent step involves pushing these logs to Elasticsearch where an index (commonly the Logstash index) will be created. In a future lesson, we will discuss building an effective dashboard in Kibana using these logs.
After confirming that the logs are being collected correctly, explore further enhancements such as filtering and parsing to improve log analytics.
That’s it for this lesson. Thank you for following along, and see you in the next lesson!

​
Additional Resources

Watch Video

Practice Lab