Tags
.net, Erlang, Fluentbit, Go, Golang, java, languages, libraries, loggiong, node.js, OCAML, perl, PHP, python, Ruby, Scala
While the typical norm is for applications to write their logs to file or to stdout (console), this isn’t the most efficient way to handle logs (particularly given I/O performance for the storage devices). Many logging frameworks have addressed this by providing more direct outputs to commonly used services such as ElasticSearch and OpenSearch. This is fine, but the only downside is that there is no means for an intermediary layer to preprocess, filter, and route (potentially to multiple services). These constraints can be overcome by using an intermediary service such as Fluent Bit or Fluentd.
Many logging frameworks can work with Fluentd by supporting the HTTP or Forward protocols Fluentd supports out of the box. But as both Fluent Bit and Fluentd are interchangeable with these protocols and logging framework that supports Fluentd, by implication also supports Fluent Bit, not to mention Fluent Bit supports OpenTelemetry.
The following table identifies a range of frameworks that can support communicating directly with Fluent Bit. It is not exhaustive but does provide broad coverage. We’ll update the table as we discover new frameworks that can communicate directly.
Language | Framework / Library | Protocol(s) | Commentary |
---|---|---|---|
Java | Log4J2 | HTTP Appender | Send JSON payloads over HTTP (use HTTP input plugin) |
Java | fluent-logger-java | Forward | |
Python | core language | HTTP Handler | Provides the means to send logs over HTTP – means Fluent Bit input handler can manage |
Python | fluent-logger-python Fluent Logger | Forward | Uses the Forward protocol meaning it can gain the efficiencies from msgpack. Maintained by the Fluent community |
Node.js | fluent-logger-node | Forward | It uses the Forward protocol, meaning it can gain efficiencies from msgpack. Maintained by the Fluent community |
Node.js | Winston | HTTP Forward | Winston is designed as a simple and universal logging library supporting multiple transports. Winston includes transport support for HTTP in its core. There is also a Transport implementation for native Fluent https://github.com/sakamoto-san/winston-fluent |
Node.js | Pino (Pino-fluent extension) | Logger integrated into the Pino logging framework | |
Go (Golang) | fluent-logger-golang | Forward | It uses the Forward protocol, meaning it can gain efficiencies from msgpack. Maintained by the Fluent community |
.Net (C# VB.Net etc) | NLog (NLog.Targets.Fluentd) | An NLog target – works with .Net | |
.Net (C# VB.Net etc) | Log4Net | Log4Net Appender | |
.Net | Serilog (Fluent Sink) | Forward and HTTP | Supports both HTTP and nativbe Fluentd/FluentBit |
Ruby | fluent-logger-ruby | Forward | It uses the Forward protocol, meaning it can gain efficiencies from msgpack. Maintained by the Fluent community |
PHP | fluent-logger-php | Forward | It uses the Forward protocol, meaning it can gain efficiencies from msgpack. Maintained by the Fluent community |
Perl | fluent-logger-perl | Forward | It uses the Forward protocol, meaning it can gain efficiencies from msgpack. Maintained by the Fluent community |
Scala | fluent-logger-scala | Forward | It uses the Forward protocol, meaning it can gain efficiencies from msgpack. Maintained by the Fluent community |
Erlang | fluent-logger-erlang | Forward | It uses the Forward protocol, meaning it can gain efficiencies from msgpack. Maintained by the Fluent community |
OCAML | fluent-logger-ocaml | Forward | It uses the Forward protocol, meaning it can gain efficiencies from msgpack. Maintained by the Fluent community |
Rust | Rust Logging framework extension for Fluent Bit | Rust crate for logging to Fluent Bit | |
Delphi | Quicklogger | HTTP |