Those I interact with more regularly will know I have this last week started a new job. You could say I’ve followed a common path for Ace Directors that don’t have instrumental roles within a company and joined Oracle. But some things won’t change; my new role as Cloud Developer Evangelist means I will still be producing blog content. My writing will appear on Oracle’s websites through the Blog, Community, Developer, and Architecture parts of the site, plus potential contributing to other high profile sites. But We’ll continue to add posts here, including referencing contributions in other locations.
I will continue submitting papers/presentations to conferences and presenting. Hopefully, we’ll start to get out and see people whilst presenting as well.
This does mean a change to my Ace Director status, as I will become an Alumni of the community, and we’ll be updating logos etc. But I have developed many friendships and contacts within the ace community. Not to mention, I believe and value what the Ace Community does, so while I may not be officially part of the community anymore, I will endeavour to support my friends and the wider community.
Leaving Capgemini has been uncomfortable, as I have left behind many great people that I’ve enjoyed working with (although some of those have also preceded me into Oracle). But to do more of what I have learnt over the years as the most rewarding (talking and writing about applying technology to solve problems, sharing insights and knowledge), particularly as part of the Ace community, meant a need for a new challenge.
To those who have contributed and influenced my journey – as ever my thanks.
I’ve had some time to catch up on books I’d like to read, including Kubernetes Best Practises in the last few weeks. While I think I have a fair handle on Kubernetes, the development of my understanding has been a bit ad-hoc as I’ve dug into different areas as I’ve needed to know more. This meant reading a Dummies/Introduction to entry style guide would, to an extent, likely prove to be a frustrating read. Given this, I went for the best practises book because if I don’t understand the practises, then there are gaps in my understanding still, and I can look at more foundation resources.
As it goes, this book was perfect. It quickly covered the basics of the different aspects of Kubernetes helping to give context to the more advanced aspects, and the best practices become almost a formulated summary in each section. The depth of coverage and detail is certainly very comprehensive, explaining the background of CNI (Container Network Interface) to network-level security within Kubernetes.
The book touched upon Service Meshes such as Istio and Linkerd2 but didn’t go into great depth, but again this is probably down to the fact that Service Mesh ideas are still maturing, and you have initiatives like SMI (Service Mesh Interface still in the CNCF’s sandbox).
In terms of best practices, that really stood out for me:
Use of Taints and Tolerations for refined control of pod deployment (Allowing affinity to be controlled to optimise resilience, or direct types of pod deployment to nodes with specialist capabilities such as GPU).
There are a lot more differences and options then you might realize in terms of ingress controller capabilities, so take time to identify what you may need from an ingress controller.
Don’t forget pods can be scaled vertically with the VPA (Vertical Pod Autoscaler)as well as horizontally through the HPA.
While using a managed persistence service will make statement storage a lot easier, stateful sets will give you a very portable solution.
As with a lot of technical books I read. As I go through the book I build up a mind map of what I think are the key points. Doing so leaves me with a resource I can use as a quick reference, but creating the mind map helps reinforce the learning. So here is the mind map …
We’ve also scored another success, this time we’ve been invited to speak at WorldFestival in August, this is an online conference organized by the same team behind DeveloperWeek. This is the first time outside of an Oracle linked event where I’ve been amongst the first few named speakers, so proud of that. The conference looks really interesting as it looks beyond just core developer themes with conference tracks on Space & Transportation, Smart Cities, Robotics, Digital Health to name a few of the 12 streams. Worth checking out.
I wrote a blog after completing my first book with Robert van Mölken about what was involved. That post can be seen here. It focussed on the processes with Packt and how Robert and I worked to try to ensure the book felt consistent despite the two of us writing.
Writing Logging In Action was a solo project for a different publisher. It seems like an opportunity to share some fresh insights.
Time and Effort
Surprisingly the time between signing the contract and the manuscript being completed ready to go through the final production process didn’t vary hugely roughly 15-18 months. The final steps of preparing to go to print did take longer, in part down to the number of extra steps taken by Manning to ensure the book was polished. Upon reflection, I think that is at least partly down to the fact you need the chapters to need to flow, particularly when one chapter leads directly to the next. So you do get periods of respite until your co-author has got sufficiently far enough with their writing to enable you to start the next part. Writing solo, as soon as one chapter is completed you’re into the next, so no periods of respite.
When it comes to the amount of actual time involved. That is different, I didn’t keep track exactly. But knowing what I typically did each week and roughly how many quiet periods I had I think it works out to be 300 hours give or take 50 hours. That sounds a lot, but then if you look at it as 1 hour per page it doesn’t seem too significant.
Using Time and Self Discipline
The way time is used has been a little different, when co-authoring you have to allow time for coordinating and supporting each other, peer-reviewing writing pushing each other along in terms of keeping to plan. The Manning development team are pretty good at keeping you moving without it feeling like you’re being chased, and will provide constructive and supportive feedback. But your co-authors will know the subject matter very well and know what your best work is like, so are able to challenge you when peer-reviewing the work. When we wrote the API book, I remember one of my colleagues reviewing a chapter and coming back saying it was a solid chapter, but I know you have explained these ideas more clearly. When I went back over the chapter, I could see what they meant. As a result a better book.
The bottom line is it can be difficult to bring a critical eye to your own work, particularly soon after have written it. But you do need that self-discipline when working on your own. This is where the Manning editorial team really stood out.
The book is published, that’s the end?
It is easy to think that’s the end of things, and certainly in terms of solid writing it is. But after all the invested effort in writing it. You might as well help promote the book and take advantage of the reputation of being a published author. This means presenting on the subject of your book. The book will provide a level of credentials & credibility to the subject you’ve written about. Despite being an introvert (which is why I take pleasure in the writing process) getting through the pre-presentation nerves, feelings of imposter syndrome once on stage and talking about your subject can be a rush, particularly as you finish. The personal payoff from presenting can come after the event, when someone who has seen your presentation says to you afterwards, that really helped me, or they really enjoyed or found the presentation thought-provoking.
If presenting is too much then these days there are other paths available, such as writing articles for journals, participating in podcasts. Having participated in several podcasts if you have a good host, this can be good fun.
What do I get out of writing?
The benefit of royalties certainly won’t replace a typical developer’s salary, unless you’re really lucky. Even with mainstream publications, only a small proportion of authors are successful enough for it to become their day job. But, there are indirect benefits. If you want something to put your CV above many others – then a book will really help. This is often why a lot of freelancers write books – it helps provide credibility over others. There is no doubt that my writing has made a difference to a change in job. I suspect that joining Capgemini and my next move has been a lot easier because of it. Not to mention, I’ve known clients like the idea that within the team they’ve engaged are people with credibility beyond just the supplier. Depending upon your employer, the marketing value for them to employ you (or me) as an author (and by implication an SME) add differentiation as well.
Writing solo again?
I’ve heard technical book authors say, never again once they’ve been published. A few I know have written multiple books. Given the experience, I think co-authoring is easier. But the gratification of completing a solo effort is so much greater.
The technical book landscape is shifting, technology cycles seem to be accelerating (or is it that I’ve reached an age where time seems to go by so much quicker) which is impacting a book’s shelf life. The ability to provide, receive and expect more interactive engagement is evolving – LiveBooks, Katacoda etc. The need to consume smaller pieces across multiple sources is growing as we need to build new skills, but don’t want to start from scratch (as I described here for example).
There is no doubt I’ll get involved in another book project. But a solo writing project will probably be smaller so we can shorten that development cycle.
The book has had a title change as Manning found that links the book was clashing with other solutions using the term ‘Unified Logging’. With the name change it helps bring the book inline with the Manning naming with their In action series. This means the book website is now https://www.manning.com/books/logging-in-action.
With the name change we’ve agreed that there should an additional chapter added. As I’d written the book with a view that everything we cover applies to both modern solutions such as Microservices coming from the CNCF camp but equally relevant to more traditional IT landscapes. Within the book we have explianed how things are positioned and can be used in Kubernetes, but it was agreed with our editorial team that not tackling the configuration of Fluentd with Kubernetes and Docker was to an extent ignoring a key community that will be using Fluentd. So the new chapter will be introduced to address this aspect.
In terms of progress we’re into the 1’s – 1 Chapter to start (the new one), 1 Chapter back from the Technical Editor (Logging Best Practises) – some edits to be done, 1 Chapter now with the editor (How To Create Custom Plugins), 1 Chapter being finished (Logging Frameworks) and finally 1 peer review cycle to go.
Given the lovely review comments that have been quoted on the book’s page. I can only recommend if you have an interest in logging and monitoring then check it out through Manning Early Access Programme (MEAP).
My blogging is way down compared with only a post about OKit – OCI Design (on Windows). It largely comes down to lots of work on our Fluentd book. Chapter 6 is now available in the MEAP. As the promo info says …
Earlier chapters have been tweaked, with some additional improvements which will make the live reading experience better.
Another chapter and an appendix should be finding their way to MEAP very soon as it was handed over by our project editor. That will make it seven chapters available, and all the appendices.
Whilst the peer review is taking place the chapter covering plugin development is progressing. The development work has got the basics of the output plugin with log events being stored in Redis and the input being worked on as well. If you want a peak, keep an eye on my GitHub repository (here).
We recorded a podcast with the excellent guys over at Adventures In DevOps. We don’t have the exact date for the podcast to be released, but I imagine it will sometime during Jan 2021. I’d recommend checking out the podcasts. I’ve been dipping into their back catalogue of recordings and the team ask some really thought provoking questions.
If that wasn’t enough, we’ve been fortunate enough to have some time to talk with leading members of the Fluentd and Fluent Bit projects which was a real pleasure. Hopefully, as we leave this horrendous year behind we’ll get to talk and possibly collaborate some more.
Things have been very hectic, so much so we’ve not really had much time to write a blog. Most of which has been related to my Fluentd book, so what has been happening (and keep an eye on my Twitter account for a promo code 🙂 ) …
First 4 chapters went through a rigorous peer review, as a result a number of improvements have been made,
with Chapter 5 having been reviewed by our technical editor, and little bit of refinement applied it should be reaching MEAP very very soon along with updated appendices,
Chapter 6 has been reviewed by our development editor, so some revisions to apply and then onto the technical editor,
Chapter 7 writing in progress, with about 1/3rd complete including examples of applying scaling configuration that can be run on a desktop
So what is to follow:
We’re on Manning’s Twitch channel to do a session, which will cover Fluentd, some examples, the book and what it will cover,
Once Chapter 7 is done, then we go through a comprehensive review with external input. Depending on the feedback from this, we make another sweep through the existing chapters to make further improvements,
Chapter 8 I suspect will be the hardest to write, as we actually get into creating our own Plugin. So I it maybe a little while before this gets completed. The subsequent chapters will come more easily as we’ve got them part written in a rough draft already,
We have another round of external peer reviewing to come which will cover everything, so I’m sure we’ll be doing some refinements
A podcast recording is scheduled in December.
Talking of Manning on Twitch, this looks like a bit of a hidden Gem – worth looking at not only because of the live stream, but all the previous recordings with other Manning authors are available to watch.
If that isn’t enough with a day job, we have had some major work done on our house. Now we’re moved back in, there are lots of DIY jobs to do, get all the furniture back from storage. Every room apart from the kitchen needs to be painted. But that is my sob story.
I’m hoping to find time to experiment with Oracle’s new cloud native Log Management as this is built on Fluentd foundations and a little bit with API tech – but this is likely to be the Christmas break.
Manning have made section of my book freely available. The excerpt from the first draft of my book Unified Logging with Fluentd illustrates the Fluentd take on Hello World – the extract can be found at http://mng.bz/nzm8. This is from the 1st chapter to help set the scene of how Fluentd can be configured. The following quote comes from one of the peer reviewers:
The extract includes the use of the log simulator tool – https://github.com/mp3monster/LogGenerator which takes some configuration and can either play synthetic data or replay real logs as current log events in what ever format you want to simulate – for example standard Log4J through to Apache Server logs with the relevant time separation between events.
Book Discount …
If this isnt enough temptations, then perhaps saying that on September 22: Deal of the Day book is my book Unified Logging with Fluentd. Use code dotd092220au at https://bit.ly/3mBRLK2
For those who regularly follow my blog, I like Groovy as a way of scripting given its portability, proximity to Java, its use of the JVM means its really easy to work with. With the extra language features it makes it easy to work with without needing to setup Maven builds to manage dependencies for something simple. As a result, the utils I’ve produced to support Oracle API Platform CS (here) for example are written with Groovy.
However, not everyone is such a fan, or sees Groovy as niche and, to be honest in the last few years Python has made huge in roads in the scripting space. So I can appreciate the preference for Python to be used. This was really brought home to me by a peer review comments for my latest book project. This got me to thinking what options do I have to remove Groovy from the equation, without resulting in needing to mess with maven POM files. The options as I saw it are:
Replace any use of Groovy extensions with Java libraries
Switch to using Java 11 and the shebang feature (more in a minute)
Compile code with Groovyc and package the jars manually including dependencies.
From Java 11, the ability for a single Java file to be executed without compiling etc was introduced – known as shebang (more here). The issue here is a lot of applications have been certified against Java 8, and in an Enterprise environment this is important. Which means either installing Java 11 locally without changing environment variables, or you need to add code to switch Java versions as necessary. The upside being that you can run your code as a script, and then if necessary build it into a jar to distribute. Aside from the need to switch between JDK versions, Java’s language gets ever richer, but Groovy has a nice set of extensions that make JSON object navigation really nice, and these aren’t necessarily in core Java.
Groovy provides a tool to compile Groovy to bytecode for the JVM (i.e. create class files). From which we can create a JAR file with using the javac command. But this doesn’t bundle the Groovy dependencies necessary to run the jar, as a result Groovy still needs to be installed. You could do this through the use of a maven, Gradle or other build tools. But we’re back to creating a POM file or equivalent., and to be honest creating a POM file from scratch is slow going unless everything lines up neatly with file structures, which it won’t because when you produce a script you’re not expecting to need to worry about packages and associated folder structures.
In looking to see if someone had simplified the process I came across this excellent tool scriptjar (more here). Script jar takes the location of the class files, creates a jar file and loads it with the Groovy jar dependencies, and assembles the manifest file etc. Net result, a jar file created in 2 lines that has no Groovy deployment dependencies to use, and will work with older Java versions such as Java 8.
I started out referring to the API tools, but the same mechanism can be applied to the LogSimulator tool I’ve created that allows you to generate or replay existing logs in real-time or faster than real-time, making it easy to test log monitoring setups from a monitored source all the way through (today it only handles log file replaying or the use of the Java native logging).