# Fluent-Bit log routing by namespace in Kubernetes

- Link: https://www.agilicus.com/routing-to-multiple-outputs-by-kubernetes-namespace-in-fluent-bit/
- Published: 2019-04-12T08:40:25+00:00
- Author: Don Bowman

You’ve got a mixed-used Kubernetes cluster. You have a very nifty security [logging](https://www.agilicus.com/increasing-the-usefulness-of-your-kubernetes-ingress-logging/)
system that wants to audit everything from certain namespaces, and of course you
want general logs from all. How would you go about configuring [Fluent-bit](https://github.com/fluent/fluent-bit)
to route all logs to one output, and only a single namespace to another, simultaneously?
Read on to learn how to perform Fluent-Bit rlog outing by namespace.

First, let’s understand the flow of information. Fluent-bit operates with a set 
of concepts (Input, Output, Filter, Parser). Inputs consume data from an external
source, Parsers modify or enrich the log-message, Filter’s modify or enrich the 
overall container of the message, and Outputs write the data somewhere.

The typical flow in a Kubernetes Fluent-bit environment is to have an Input of type`
Tail`, which conceptually does a `tail -f` on all your log files. It relies on the
fact the files have a magic name (incorporating the pod/namespace/container information).

From the Input we then go through a `Filter` of type `Kubernetes`. This takes the
input parsed from the filename (Pod/Namespace/Container), does a lookup to the Kubernetes
API (e.g. to look at annotations). From here we go to the output stage.

In the example use case here, we want to send *all* records to the default Output,
and *some* (e.g. all from a namespace) to a secondary Output. Let’s dig in.

    ```wp-block-code
    [SERVICE]
        Flush        1
        Daemon       Off
        Log_Level    info
        Parsers_File parsers.conf

    [INPUT]
        Name             tail
        Path             /var/log/containers/*log
        Parser           docker_no_time
        Tag              kube...
        Tag_Regex        (?[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*)_(?[^_]+)_(?.+)-
        Refresh_Interval 5
        Mem_Buf_Limit    50KB
        Skip_Long_Lines  On

    [FILTER]
        Name                kubernetes
        Match               kube.*
        Merge_Log           Off
        Regex_Parser        kube-tag
        K8S-Logging.Parser  On
        K8S-Logging.Exclude On

    [OUTPUT]
        Match *
        Name  stdout
        Format json_lines
        JSON_Date_Format iso8601

    [OUTPUT]
        Name file
        Match kube.ns2.*
        Path ns2-logs-only.txt
    ```

OK, we added two different outputs. One matches all, one only namespace ns2. This
Match parameter is how we do Fluent-Bit routing by namespace.

    ```wp-block-code
    [PARSER]     Name   json     Format json     Time_Key time     Time_Format %Y-%m-%dT%H:%M:%S.%L [PARSER]     Name    kube-tag     Format  regex     Regex    (?[^.]+).(?[^.]+).(?[^.]+).(?[^.]+) [PARSER]     Name         docker_no_time     Format       json     Time_Key     time     Time_Format  %Y-%m-%dT%H:%M:%S.%L     Time_Keep    Off     Decode_Field_As   escaped    log     Decode_Field_As   escaped    stream
    ```

OK what have we achieved? Well, for namespace `ns2` we route that output to a file.
For all logs (including `ns2`) we route that to stdout. We have done Fluent-Bit 
log routing by namespace in Kubernetes.

Now, replace `stdout` and `file` with your outputs. And profit!

You may use this tag match routing for other users. I use it with 2 different clusters,
each running fluent-bit. 1 of them is configured to forward to the other, and the
logs are routed by cluster source.

I also use this technique to route the Node logs to one Elasticsearch Index, and
the Pod logs to another.