12th April Update – The last chapter, a use case Appendix, and a couple of chapter updates are heading to the MEAP release.
We’ve not been blogging too much as we’ve been very focused on the book. For the keen readers who have signed up for the MEAP (Manning Early Access Programme) of the book, another 2 chapters are in the process of being made available.
The last chapter has been submitted to our editor along with the appendix, which includes an enterprise use case that outlines a business scenario and illustrates how Fluent Bit can be applied.
We’ve received the feedback from the second peer review and have started to address it. I’m sure that every Manning author will testify as to how helpful the process is. While I recommended some of the reviewers to my editor, I didn’t know others. All the feedback comes back anomalously. So publicly, thank you to the reviewers. Constructive feedback is key to how we ensure that we are getting our points across, but also how details we may have overlooked or thought obvious get put right.
Unfortunately, authors can’t always address every comment. Sometimes, that is down to the fact that the layout has to work within the publisher’s guidelines. Sometimes, we simply can’t fit in suggested content, as we’re ultimately working to an agreed timeline, and people can be put off by 800-page books. For me, and I suspect other authors, those extras aren’t ignored; they’re fuel for blog ideas and content.
We’ve one more peer review cycle where the reviewers get pretty much the entire book, and once any edits for that are needed, we move into the copy editing, which is done by Manning, and I just need to confirm edits don’t accidentally change the meaning and emphasis. This will be a time when we can start blogging and sharing more.
A quick update on the book – very early this morning or late last night (depending on your perspective), we sent our development editor the final chapter of the Fluent Bit with Kubernetes book. There is still a way to go before we’re completed (with multiple reviews to happen, appropriate edits to be made, copy editing, etc. Still, it is an important milestone from an author’s perspective.
For the keen readers who have signed up for the MEAP (Manning Early Access Programme) of the book, I can confirm that the editorial team (preparation for eBook and website formatting, checking the edits to address the Technical Editor and Development Editor haven’t introduced any obvious issues) are working on the preparation of Chapter 7 – so that should be available soon. When this chapter is available, the content covering all the foundational aspects of Fluent Bit will be available. The remaining chapters reflect the advanced features.
The exciting news is that Manning have released several more chapters of our Fluent Bit with Kubernetes book into the MEAP (Manning Early Access Program) – which means about two-thirds of the book is now available in MEAP form.
We’ve also been beefing up the supporting and related information on this website – as we can’t get everything into the book – for the static pages, the most relevant are here and here, and the blog post content can be seen here.
The sample configurations are in our GitHub repo here, and additional demos can be found here. We’ve got a pretty cool demo being built, which takes Fluent Bit into the world of ChatOps (and it isn’t just sending notifications) – it will eventually become visible in the repo – but to see it sooner, keep an eye out for our conference presentations.
The hyper scaler cloud vendors all offer Logging and monitoring capabilities. But they tend to focus on supporting their native services. If you’re aware of Oracle’s Cloud (OCI) messaging, then you’ll know that there is a strong recognition of the importance of multi-cloud. This extends not only to connecting apps across clouds but also to be able to observe and manage cloud-spanning solutions. Ultimately, most organizations want to headline observability-related views of their solutions.
Late last year, I presented these ideas, illustrating the ideas with the use of Fluent Bit and OCI’s Observability and Management products to visualize and analyze what is happening. I finally found the time to write how the very basic demo was built from a clean sheet over on the Oracle Devs blog on Medium.
Useful Resources for Fluent Bit and Observability
This also highlights the fact that the Fluent Bit book, while I believe, once completed, will be through, can’t cover everything – and certainly not build end-to-end use cases like the Oracle Observability & Management example. To help address this, the book includes an appendix of helpful additional information, some of which I have included here, along with other content that we encounter – all of which can be found at Fluentd & Fluent Bit Additional stuff.
With the Christmas holidays happening, things slowed down enough to sit and catch up on some reading – which included reading Cloud Observability in Action by Michael Hausenblas from Manning. You could ask – why would I read a book about a domain you’ve written about (Logging In Action with Fluentd) and have an active book in development (Fluent Bit with Kubernetes)? The truth is, it’s good to see what others are saying on the subject, not to mention it is worth confirming I’m not overlapping/duplicating content. So what did I find?
Cloud Observability In Action has been an easygoing and enjoyable read. Tech books can sometimes get a bit heavy going or dry, not the case here. Firstly, Michael went back to first principles, making the difference between Observability and monitoring – something that often gets muddied (and I’ve been guilty of this, as the latter is a subset of the former). Observability doesn’t roll off the tongue as smoothly as monitoring (although I rather like the trend of using O11y). This distinction, while helpful, particularly if you’re still finding your feet in this space, is good. What is more important is stepping back and asking what should we be observing and why we need to observe it. Plus, one of my pet points when presenting on the subject – we all have different observability needs – as a developer, an ops person, security, or auditors.
Next is Michael’s interesting take on how much O11y code is enough. Historically, I’ve taken the perspective – that enough is a factor of code complexity. More complex code – warrants more O11y or logging as this is where bugs are most likely to manifest themselves; secondly, I’ve looked at transaction and service boundaries. The problem is this approach can sometimes generate chatty code. I’ve certainly had to deal with chatty apps, and had to filter out the wheat from the chaff. So Michael’s approach of cost/benefit and measuring this using his B2I ratio (how much code is addressing the business problems over how much is instrumentation) was a really fresh perspective and presented in a very practical manner, with warnings about using such a measure too rigidly. It’s a really good perspective as well if you’re working on hyperscaling solutions where a couple of percentage point improvements can save tens of thousands of dollars. Pretty good going, and we’re only a couple of chapters into the book.
The book gets into the underlying ideas and concepts that inform OpenTelemetry, such as traces and spans, metrics, and how these relate to Observability. Some of the classic mistakes are called out, such as dimensioning metrics with high cardinality and why this will present real headaches for you.
As the data is understood, particularly metrics you can start to think about how to identify what normal is, what is abnormal, or an outlier. That then leads to developing Service Level Objectives (SLOs), such as an acceptable level of latency in the solution or how many errors can be tolerated.
The book isn’t all theory. The ideas are illustrated with small Go applications, which are instrumented, and the generated metrics, traces, and logs. Rather than using a technology such as Fluentd or Fluent Bit, Michael starts by keeping things simple and directly connecting the gathering of the metrics into tools such as Prometheus, Zipkin, Jaeger, and so on. In later chapters, the complexity of agents, aggregators, and collectors is addressed. Then, the choices and considerations for different backend solutions from cloud vendor-provided services such as OpenSearch, ElasticSearch, Splunk, Instana and so on. Then, the front-end visualization of the data is explored with tools such as Grafana, Kibana, cloud-provided tools, and so on.
As the book progresses, the chapters drill down into more detail, such as the differences and approaches for measuring containerized solutions vs. serverless implementations such as Lambda and the kinds of measures you may want. The book isn’t tied to technologies typically associated with modern Cloud Native solutions, but more traditional things like relational databases are taken into account.
The closing chapters address questions such as how to address alerting, incident management, and implementing SLOs. How to use these techniques and tools can help inform the development processes, not just production.
So I would recommend the book, if you’re trying to understand Observability (regardless of a cloud solution or not). If you’re trying to advance from the more traditional logging to a fuller capability, then this book is a great guide, showing what, why, and how to evaluate the value of doing so.
To come back to my opening question. The books have small points of overlap, but this is no bad thing, as it helps show how the different viewpoints intersect. I would actually say that the Observability in Action shows how the wider landscape fits together, the underlying value propositions that can help make the case for implementing a full observability solution. Then, Logging in Action and the new book, Fluent Bit with Kubernetes, give you some of the common context, and we drill into the details of how and what can be done with Fluent Bit and Fluentd. All Manning needs now is content to deep dive into Prometheus, Grafana, Jaeger, and OpenSearch to provide an end-to-end coverage of first principles to the art of the possible in Observability.
I also have to thank Michael for pointing his readers and sections of Logging in Action that directly relate and provide further depth into an area.
It’s been a busy time on the book front. With the book going into the Manning Early Access Program (MEAP). Since then, we have got three more chapters ready, and we’re waiting for the Manning Technical Editor to come back on the content. Once that feedback is addressed, we go into another peer review round, and the chapters will be added to the MEAP edition.
The peer review process is insightful and helps drive improvements to the content, as the reviewers are deeply informed – after all, some of them coded the functionality I’m writing about. So, the comments and questions that come back deserve careful consideration.
It isn’t just more chapters; the appendices have been moving forward, and we should see the first two appendices in the MEAP content soon.
If that isn’t enough for you – as we developed Chapter 8 (probably ready for Manning by the New Year) and worked on the appendices, we identified content that doesn’t neatly fit the book, so we’ve incorporated that content on this blog and in other places (we’re expecting a blog to be published early in the New Year here). We’ve created a page on this website to help collect together content here, and you can see all the blog content here.
But there is plenty of work still to be done, as Fluent Bit supports both YAML and classic configuration formats. We have focussed on the classic format in the book as more people use that at the moment. But, eventually, the YAML format will become more dominant, so we’re providing the YAML equivalent configurations for all the examples. These need to be finalized and checked.
We think Chapter 9 will come quickly, but the final chapter will take time, as we’re expecting the peer review feedback to land as we start Chapter 10. But there is also a lot more effort involved in the final chapter. Then, it’s another peer review round.
We’ve recently had several pieces published on other websites, so I thought we should link them together.
Here is a short piece on Ubuntu security on OCI here.
Another here (DZone) on the use of Solace for multi-cloud messaging.
We’re expecting another article to appear here in the New Year as well. Plus, the book is moving along at a very nice pace – we’ve got a separate post for that.
On the book front – watch out for the Manning and Packt festive promotions.
The new book focuses on Fluent Bit, given its significant advances, reflected by the fact it is now at Version 2 and is deserving of its own title. The new book is a free-standing book but is complimentary to the Logging In Action book. Logging in Action focuses on Fluentd but compliments by addressing more deeply deployment strategies for Fluent Bit and Fluentd. The new book engages a lot more with OpenTelemetry now it has matured, along with technologies such as Prometheus.
It is the fact that we’ve seen increasing focus in the cloud native space on startup speeds and efficiency in footprints that have helped drive Fluent Bit, as it operates with native binaries rather than using a Just-In-Time compilation like Ruby (used for Fluentd). The other significant development is the support for OpenTelemetry.
The book has entered the MEAP (Manning Early Access Program). The first three chapters have been peer-reviewed, and changes have been applied; another three are with the development editor. If you’ve not read a MEAP title before, you’ll find the critical content is in the chapters, but from my experience, as we work through the process, the chapters improve as feedback is received. In addition, as an author, when we have had time away from a chapter and then revisit it – it is easier to spot things that aren’t as clear as they could be. So, it is always worth returning as a reader and looking at chapters. Then, as we move to the production phases, any linguistic or readability issues that still exist are addressed as a copy editor goes through the manuscript.
I’d like to thank those involved with the peer review. Their suggestions and insights have been really helpful. Plus, the team at Calyptia is sponsoring the book (and happens to be employing a number of the Fluent Bit contributors).
We also have a discount code on the book, valid until 20th November – mlwilkins2
One of the benefits of being a Manning author is that we get access to the Manning book catalog, including those currently in the MEAP early access programme (MEAP). The Road to Kubernetes title was bought to my attention. The book has just become available as a MEAP title; this means that the book has just completed its first major review milestone, and about a third of the book has been written. It does mean our review only covers the first 3 chapters at the moment.
What got my attention with this book is that unlike other titles about Kubernetes \9of which there are a number of great titles in the Manning portfolio already) is that it has adopted a different approach.
Most books focus on one technology and deep dive into that technology and dig into the more advanced features of that specific area. For an experienced IT person, that is great. But, when it comes to Kubernetes, if you’re skills are largely just focused on largely coding with languages like Java, Python, and JavaScript – not unusual for a graduate or junior developers it means the amount of reading and learning curve to get to grips with developing and deploying containers to Kubernetes is considerable. Here, Justin has taken the approach of assuming basic development skills and then taking you on a journey of focussing on the basics of containers, deployment automation, and then Kubernetes with just enough to be able to deploy a simple solution using good practices. This makes the learning path to gaining the skills that allow you to work within a team and building containerized solutions a lot easier.
I imagine once the book is complete and you’ve followed it through, you’ll be in a position to focus on learning new, more advanced aspects of containers and Kubernetes in a focused manner to meet the needs of a day-to-day job.
Having coached and mentored junior developers and graduates, this is a book I’d recommend to help them along, and if my experience with the Manning book development process is anything to go by, as Justin goes through the major milestones, this book will go from good to great.
My only word of caution is that this book will take the reader on a journey of building and deploying microservices to Kubernetes. Don’t be fooled into thinking Kubernetes and microservices are easy – there is a lot of technologies that I don’t think the book will go into (but then not all developers need to understand details such the differences in network fabric (Calico, Flannel) or container engines cri-o, Docker Engine, deploying support tooling for things like observability). Without good design and, depending upon your solution, a handle in a variety of more specialized areas, it is still possible to get yourself into a mess, even for the most experienced of teams.
You must be logged in to post a comment.