Today we got the official notification that our book has been published …
As you can see, the eBook is now available. The print edition can be purchased from Thursday (24th Oct). If you’ve been a MEAP subscriber, you should be able to download the complete book. The book will start showing up on other platforms in the coming weeks (Amazon UK has set an availability date, and Amazon.com you can preorder).
There are some lovely review quotes as well:
And there is more …
I hadn’t noticed until today, but the partner book Logging in Action, which covers Fluentd, is available in ebook and print as well as audio and video editions. As you can see, these are available on Manning and platforms like O’Reilly/Safari…
I have to say that my digital twin, who narrated the book, sounds pretty intelligent.
Update
Amazon UK is correct now, and has an availability date
I thought you might like to know that last week, the production process on the book (Logs and Telemetry with Fluent Bit, written with the working title of Fluent Bit with Kubernetes) was completed, and the book should be on its way to the printers. In the coming weeks, you’ll see the MEAP branding disappear, and the book will appear in the usual places.
If you’ve been brilliant and already purchased the book – the finished version will be available to download soon, and for those who have ordered the ‘tree’ media version – a few more weeks and ink and paper will be on their way.
Earlier in the year, I made a utility available that supported the migration from Fluent Bitclassic configuration format to YAML. I also mentioned I would explore the migration of Fluentd to Fluent Bit. I say explore because while both tools have a common conceptual foundation, there are many differences in the structure of the configuration.
We discussed the bigger ones in the Logs and Telemetry book. But as we’ve been experimenting with creating a Fluentd migration tool, it is worth exploring the fine details and discussing how we’ve approached it as part of a utility to help the transformation.
Routing
Many of the challenges come from the key difference in terms of routing and consumption of events from the buffer. Fluentd assumes that an event is consumed by a single output; if you want to direct the output to more than one output, you need to copy the event. Fluent Bit looks at things very differently, with every output plugin having the potential to output every event – the determination of output is controlled by the match attribute. These two approaches put a different emphasis on the ordering of declarations. Fluent Bit focuses on routing and the use of tags and match declarations to control the rounding of output.
We can also see that Fluentd’s directives are more hierarchical (e.g., buffer, and format are within the store) than the structures used by Fluentd Bit, so we need to be able to ‘flatten’ the hierarchy. As a result, it makes sense that where the copy occurs, we’ll define both outputs in the copy declaration as having their own output plugins.
Buffering
There is a notable difference between the outputs’ buffer configurations: in Fluent Bit, the output can only control how much storage in the filesystem can be used. As you can see in the preceding example, we can set the flushing frequency, control the number of chunks involved (regardless of storage type).
Pipelines
Fluentd allows us to implicitly define multiple pipelines of sources and destinations, as ordering of declarations and event consumption is key. ~In addition to this, we can group plugin behavior with the use of the Fluentd label attribute. But the YAML representation of a Fluent Bit doesn’t support this idea.
Fluentd also supports the idea of a secondary output as the following fragment illustrates. If the primary output failed, you could write the event to an alternate location. Fluent Bit doesn’t have an equivalent mechanism. To create a mapping tool, we’ve taken the view we should create a separate output.
The reworked structure requires consideration for the matching configuration, which isn’t so easily automated and can require manual intervention. To help with this, we’ve included an option to add comments to link the new output to the original configuration.
Configuration differences
While the plugins have a degree of consistency, a closer look shows that there are also attributes and, as a result, features of plugins that don’t translate. To address this, we have commented out the attribute so that the translated configuration can be seen in the new configuration to allow manual modification.
Conclusion
While the tool we’re slowly piecing together will do a lot of the work in converting Fluentd to Fluent Bit, there aren’t exact correlations for all attributes and plugins. So the utility will only be able to perform the simplest of mappings without developer involvement. But we can at least help show where the input is needed.
With some recent good news from work, I decided to treat myself to a speaker upgrade – Acoustic Energy 500s sat on some IsoAcoustic Aperta stands. While these would be considered audiophile – they’re still at the lower end – we’re not talking audio exotica like B& Nautilus at nearly hundred thousand pounds or the Cosmotron 130 at around the million pound mark.
Bowers & Wilkins – Nautilus Speaker – a snip at £90,000
So how can I decide and justify the expenditure, even if it’s a fraction of the loose change from the back of the sofa from buying these monsters? As friends have said to me in the past, the Samsung speakers on my stereo are just as good. Well there are a raft of things that will prevent speakers from performing well, from positioning, to the quality of their source.
Cosmotrom priced at £1M
The source material is often one of the biggest issues, particularly for rock and pop pushing the envelope with CDs. We saw what has become known as the loudness wars – where the dynamic range of the music was reduced. But music with a wide dynamic range with good speakers is great. A couple characteristics of good speakers is the containment of distortion – so if you have a song that is often quiet with occasional moments of loudness, the speaker drivers (cones) will be able to react properly to another sudden spike in signal occurs the sudden movement in the magnet moving the cone is handled rather than causing the speaker surface straining against its mounts.
Better speakers will result in better control of the cone (the visible bit of the speaker), making the cone’s movements more precisely revealing detail in the music. You’ll go from hearing a cymbal, to being able to tell how the cymbal was struck, a drum is no long a thump, but you’ll start to hear it resonate.
The cone moves backward and forwards to move the air, which affects air inside the speaker, not just outside. We don’t want the speaker casing to behave as a suction cup, preventing air movement and inhibiting the cone’s movement.
Improvements in speaker performance can help you recognize little details. For example, with a vocal performance, you’ll start to hear fine details, such as air drawn over the microphone as the singer inhales. You can also hear changes as a singer moves close to or away from the microphone, even if they alter their vocal volume.
I was experimenting with a loaned hi-fi kit once, listening to a Jamie Cullum live performance, and a detail that leapt out as I swapped in and out a piece of equipment was what sounded like background ambient noise, such as air conditioning. But suddenly, it became clear I wasn’t picking up ambient noise but the fan that was positioned behind Jamie.
It is always useful to have some good go-to pieces of music for trying out hi-fi. Being familiar with the music and knowing the production values applied means that if there are improvements, you’ll pick them up. So, what are my go-to pieces at the moment?
Tori Amos – Me and a Gun — although any part of Little Earthquakes is good. This song is an acapella performance, recounting a rape. With just a voice, the miking of the vocal is very close, and you can hear the inhalation and the rawness of the performance.
Beth Orton – Weather Alive — probably Beth’s best album to date. Here is another incredible voice, but also more delicate than Tori Amos, so the better the HiFi, the purer the performance will sound.
GoGo Penguin – Branches Break from Man Made Objects – although just about any of their work will be good. This is a trio of piano, bass, and drums in a jazz/minimalist classical/chill beat crossover. This is a recording that should feel like it’s being performed in a big live sounding room. But you’ll hear each instrument clearly, particularly down to recognizing the loudness, varying attack, and decay of each note played.
Rush – Red Sector A from Grace Under Pressure, perhaps not the best-produced album in the world, but before the loudness wars really took hold. Rush were a real bunch of prog rock musos with the late Neil Peart, who many considered to be one of the best ever drummers. This track will test the HiFi in terms of control – the drumming has a huge range of very fine cymbal work, some really deep bass drums, and tom-tom runs that make Phil Collin’s In The Air Tonight sound like child’s play.
Elbow – One Day Like This – The Seldom Seen Kid (Live At Abbey Road Studios) — with a high-quality recording (Abbey Road’s special Half Speed Mastered edition), you’ll get a sense of staging and as the song grows scale with the choir. The strings will be natural and nuanced, in the early parts of the performance of the performance you’ll hear how dry Guy’s voice is – not a hint of vibrato or sibilance.
Peter Gabriel – the Book Of Love — from Scratch My Back — another performance that should give a sense of staging and breadth with great dynamics and the strings swell and subside. Fronted by Peter’s voice which should weathered and world warn.
The list of music could go on. But, ultimately, it’s a very individual choice.
Final anecdote
Buying Hi-Fi is a law of diminishing returns. As you get better and better, the parts needed are more expensive and produced in fewer numbers, making the R&D more expensive, with costs to be covered by a small number of sales. But still, these esoteric, bank-crushing systems are amazing.
Some years back, I went to a HiFi show; if you’ve never been to such a show then picture this. A corridor of rooms is stripped of the beds and furnishings other than some chairs. Each company has a room and typically sets up its demo kit where the head of the bed would usually be. Everything would be positioned and mounted on professional hi-fi tables, etc, for the absolute best performance. The classic layout for a hotel room means as you walk into the room, you won’t see what is set, so the seconds it takes to walk past what is normally the bathroom is almost a blind test as you can’t see the HiFi, but you’ll be able to hear it.
So here we are, as we start to walk into a room that was pretty busy, so you didn’t see the main space for a minute or so, and we hear a performance of a beautifully played unaccompanied double bass. I could have sworn there was a musician in the room performing – the performance had that warmth, depth, and volume you’d expect. No hint of any recording artifacts. When we got to the main part of the room, we were stunned to see two speakers, big and rather boxy – no audio exotica beauty like Nautilus or Cosmotron — definitely all function, and little thought to form. With them, 3 large pieces of silver HiFisat on big chunky slabs of marble on the floor – what I assume to be a pre-amp and a power amp for each speaker. Plus a source – which might have been a turntable – but honestly, I can’t remember – whatever it was, the sound was breathtakingly natural sounding.
Chord Monobloc Power Amplifier £350,000 per bloc – you’d need two, plus a pre-amp for a basic arrangement.
I do remember the price tags, and at the time, prices were around 50k a component- so little change out of a quarter of a million. It left me wishing I’d won the national lottery.
You must be logged in to post a comment.