Tags

, , , , ,

August saw the the release of Fluent Bit 5.1, which has introduced a range of improvements including:

  • Performance improvements and configuration controls available, such as HTTP workers.
  • Support for NVIDIA GPUs via the NVML (NVIDIA Management Library)
  • A FIPS safe mode to ensure things are safe during start and restart.
  • Ensuring timestamps will work beyond 2038.
  • TLS handling refinements such as certificate reloading
  • Management of the internal buffers can be made more dynamic now.
  • Improvements across a number of plugins for input and output.
  • Dynamic flushing to allow dynamic optimization within configured boundaries.
  • Packaging improvements for Windows Nano and Debian deployments.
  • New root section called Extensions.

A lot of these gains are great, and will really pay off for high volume deployments, where small savings, cumulative can really payoff economically. It is the new extensions feature that I’m most interested in as it open up some capabilities when managing the observability domain.

Extensions

The extension structure (first identified as a useful feature with issue 11863) doesn’t impact Fluent Bit behavior but is retained and accepted as YAML configuration values. This means that any tools being used to manage Fluent Bit configuration can now add YAML key value pairs (including nested values) without causing Fluent Bit a problem, and can therefore be used by management tools. For example we can track what version of a configuration is being handled by Fluent Bit by injecting a Git version identifier (just as you see in the Fluent Bit output at startup). This simple change means that it is now possible to understand which version of configuration is being executed. When you’re rolling out to a large estate (particularly outside of containerized environments) it is easy to know what has been rolled out.

extensions:
  opamp:
    endpoint: 127.0.0.1:4318
    insecure: true
  deployment:
    config_version: 12345

If you’re using Fleet Management capabilities such as OpAMP (a specification developed as part of the Open Telemetry project) you can embed the client side controller/supervisor configuration within the Fluent Bit configuration (as long as the controller supervisor understand where to get the values from within a YAML file).

It also means that you’re allowing more dynamic changes, it becomes easy to notate such dynamic changes., for example:

extensions:
  deployment:
    changeDTG: 2026.08.05.16.25
    notes:
      - changed flush interval
      - moved plugin x output

Personally I’d not advocate changes without using configuration control tooling, but it certainly is better than no record.

Next steps

If you follow this blog regularly, they’ll be aware of the OpAMP project which implements the protocol, and provides a server that includes fleet management and configuration editing and validation for Fluent Bit server side, and a client supervisor/observer on client client/consumer side which can manage Fluent Bit instances. These changes mean that the project will need:

  • New configuration and validation settings (the UI is entirely meta-data driven) for the new properties.
  • The configuration catalogue viewer will need to be updated to exploit the extensions feature for showing versioning.
  • The client/consumer side can be extended to become Fluent Bit aware and exploit the embeddable extensions capability.
  • We may also offer the dynamic change recording possibility as an optional feature (from a product view – we shouldn’t force specific ways of working, but from a personal view it isn’t a way I’d recommend working)

Resources