While the norm is for applications to write their logs to file or to stdout (console) or a file, this isn’t the most efficient way to handle logs (particularly given I/O performance for the storage devices). Many logging frameworks have addressed this by providing more direct outputs to commonly used services such as ElasticSearch and OpenSearch. This is fine, but the only downside is that there is no means for an intermediary layer to preprocess, filter, and route (potentially to multiple services). These constraints can be overcome by using an intermediary service such as Fluent Bit or Fluentd.
Many logging frameworks can work with Fluentd by supporting the HTTP or Forward protocols Fluentd supports out of the box. However, as both Fluent Bit and Fluentd are interchangeable with these protocols and logging frameworks that support Fluentd, by implication, Fluent Bit also supports OpenTelemetry.
The following table identifies a range of frameworks that can support communicating directly with Fluent Bit. It is not exhaustive, but we’re working on it.
Feel free to contact us with more details
| Language | Framework / Library | Protocol(s) | Commentary |
|---|---|---|---|
| Java | Log4J2 | HTTP Appender | Send JSON payloads over HTTP (use HTTP input plugin) |
| Java | fluent-logger-java | Forward | |
| Python | core language | HTTP Handler | Provides the means to send logs over HTTP – means Fluent Bit input handler can manage |
| Python | fluent-logger-python Fluent Logger | Forward | Uses the Forward protocol meaning it can gain the efficiencies from msgpack. Maintained by the Fluent community |
| Node.js | fluent-logger-node | Forward | It uses the Forward protocol, meaning it can gain efficiencies from msgpack. Maintained by the Fluent community |
| Node.js | Winston | HTTPForward | Winston is designed as a simple and universal logging library supporting multiple transports. Winston includes transport support for HTTP in its core. There is also a Transport implementation for native Fluent https://github.com/sakamoto-san/winston-fluent |
| Node.js | Pino (Pino-fluent extension) | Logger integrated into the Pino logging framework More on Pino https://getpino.io/ | |
| Go (Golang) | fluent-logger-golang | Forward | It uses the Forward protocol, meaning it can gain efficiencies from msgpack. Maintained by the Fluent community |
| Go (Golang) | Zap | TBD | TBD |
| .Net (C# VB.Net etc) | NLog (NLog.Targets.Fluentd) | An NLog target – works with .Net | |
| .Net (C# VB.Net etc) | Log4Net | Log4Net Appender | |
| .Net | Serilog (Fluent Sink) | Forward and HTTP | Supports both HTTP and nativbe Fluentd/FluentBit |
| Ruby | fluent-logger-ruby | Forward | It uses the Forward protocol, meaning it can gain efficiencies from msgpack. Maintained by the Fluent community |
| PHP | fluent-logger-php | Forward | It uses the Forward protocol, meaning it can gain efficiencies from msgpack. Maintained by the Fluent community |
| Perl | fluent-logger-perl | Forward | It uses the Forward protocol, meaning it can gain efficiencies from msgpack. Maintained by the Fluent community |
| Scala | fluent-logger-scala | Forward | It uses the Forward protocol, meaning it can gain efficiencies from msgpack. Maintained by the Fluent community |
| swift | list of Swift logging libraries | swift has a number of logging frameworks that focus on console or file output. But several can be easily extended to send logs over HTTP | |
| Erlang | fluent-logger-erlang | Forward | It uses the Forward protocol, meaning it can gain efficiencies from msgpack. Maintained by the Fluent community |
| OCAML | fluent-logger-ocaml | Forward | It uses the Forward protocol, meaning it can gain efficiencies from msgpack. Maintained by the Fluent community |
| Rust | Rust Logging framework extension for Fluent Bit | Rust crate for logging to Fluent Bit | |
| Delphi | Quicklogger | HTTP |
We’ve not included a comments form directly to avoid being spammed.
other resources on the subject of logging frameworks:
Phil (aka MP3Monster)'s Blog 