• Home
  • Site Aliases
    • www.cloud-native.info
    • oracle.cloud-native.info
    • Phil-Wilkins.uk
  • About
    • Background
    • Presenting Activities
    • Internet Profile
      • LinkedIn
    • About
  • Books & Publications
    • Logging in Action with Fluentd, Kubernetes and More
      • Logging in Action with Fluentd – Book
      • Fluentd Book Resources
      • Log Generator
    • API & API Platform
      • API Useful Resources
    • Oracle Integration
      • Book Website
      • Useful Reading Sources
    • Publication Contributions
  • Resources
    • GitHub
    • Mindmaps Index
    • Oracle Integration Site
    • Useful Tech Resources …
      • Oracle Tech Resources inc Open Source
      • Useful Tech Resources
      • Python Setup & related stuff
  • Music

Phil (aka MP3Monster)'s Blog

~ from Technology to Music

Phil (aka MP3Monster)'s Blog

Tag Archives: book

Phoenix project

21 Saturday Jan 2023

Posted by mp3monster in General, Technology

≈ 1 Comment

Tags

agility, automation, book, CI/CD, CICD, devops, DevSecOps, Gene Kim, IaC, OWASP Top 10, Phoenix project, Security, software, Work

The Phoenix Project by Gene Kim has been recommended reading for the IT industry for a long time now. It’s been on my to-read list for a good while, but to be candid, it never made the top of my reading list, as I had some reservations. Who wants to read a novel about IT if you already work and live it every day.

Recently IT Revolution, to celebrate its 10th anniversary, offered the book through Amazon for free for a limited period (even now, as a Kindle ebook, it isn’t that expensive anymore). Given I had determined I should read it at some point, I took the opportunity to get a copy. As it happens, through the Christmas break, I got into a run of reading books, and with it being at the top of my ebook list, I bit the bullet.

First of all, the story was engaging and very readable, characters are likable, human, and relatable. It isn’t a huge book either (perhaps I’ve been looking at too many 500+ page novels), making it a fairly quick read. As a pure novel, some of the devices to keep the narrative moving along were perhaps a little obvious, but then given the goal of the story, that isn’t really an issue. It didn’t break the reading flow, which did keep the pages turning with a plausible story; plausible enough to wonder how much of the story was based on a real-life experience of Gene or one of his writing collaborators.

What struck me the most is that most industry writing I have read doesn’t address all the points the book makes. So, DevOps, as is typically presented by a lot (most ?) content, uses some variation of a diagram like the infinite cycle, as shown here:

Image courtesy of Amis.nl

Not only that, it is common to view DevOps as focusing on either :

  • Automation, particularly around CI/CD and IaC (Infrastructure as Code)
  • The development team also owns the operations tasks

But the book portrayed DevOps as both and neither of these. I say this as these approaches can help with the goal, but they should be subservient to the larger objective. Unfortunately, we do get caught up with the mechanics and tools and not the wider goal. The story is about how to deliver business value and needs in a streamlined manner, so we aren’t tying up the investment (time or spend) any longer than necessary. Yes, that does mean IaC and CI/CD style automation but only in service of the goal, which is the business need, not IaC.

The book also highlights the point of continuously working on improvement and paying down against debt, as removing debt is part of the way we remove the blockages to the streamlining (in the story, we actually see the release pipeline being temporarily stopped so that time could be invested in paying down the debt). Yet, this aspect is rarely discussed in a lot of the industry content on the subject. Maybe we are, in part, our own enemy here, as debit work is not greenfield. It is going back over old ground and making it better. We all love breaking new ground and leaving the past behind. Not to mention many organizations measure progress on the number of features rather than how well a feature serves the business goals. I have to admit to that mistake, which in our world of microservices is a bit of a mistake. After all, aren’t microservices about doing one thing and doing it well?

Another interesting view that the book put a lot of emphasis on was a variant of what is sometimes called the Eisenhower matrix. Anyone who has done any leadership training will most likely recognize it (see below).

However, the quadrants of work are as the book describes them are:

  • Quadrant 1 – Project work (i.e., planned activities central to the business)
  • Quadrant II – IT work (work that is planned and needed but doesn’t originate from the business, such as building new infrastructure)
  • Quadrant III – Updates and changes (e.g., system patching)
  • Quadrant IV – Unplanned (e.g., outage recovery work, demands on the team that has bypassed scheduling / divert individuals from the agreed goals, etc.)

The key difference between this representation and that of the book’s work definition is the words on each of the columns and rows. For example, Quadrant IV isn’t ‘Not Important’ and ‘Not Urgent’ – but it would be fair to say ‘Not Wanted’ and ‘Not Productive’. Unplanned work is the killer and roughly aligns with quadrant IV. This comes from the issues of not dealing with technical debt and solution facilities etc.

My last observation is that in the last couple of years, we have seen the rise of DevSecOps, which recognizes the need that security should be as much of the delivery process as Dev and Ops. The book (written 10 years ago) showed that security should be part of the DevOps process. Like other areas, the story seeks to address the point that security focussing on just the development and operational processes while necessary for things like catching OWASP Top 10 also needs to see the bigger picture otherwise, you could easily add additional needs that are already handled by controls elsewhere in the end-to-end business processes. That doesn’t mean to say security controls can5 be in software, but are they part o& improvement and pushing actions left vs. getting out of the starting blocks?

More reading

The book provides references, but for my own personal benefit, a number of particularly interesting and useful references are made, which may interest:

  • The Goal by Eliyahu M. Goldratt (a novel addressing his Theory of Constraints in the same way as The Phoenix Project is a novel around the DevOps handbook)
  • Beyond the Goal by Eliyahu M. Goldratt
  • Value stream mapping
  • Theory of Constraints
  • DevOps Kaizen: Find and Fix What Is Really Behind Your Problems
  • Tech debt spiral
  • Work In Progress and impact of high amounts of WIP

Share this:

  • Twitter
  • Facebook
  • LinkedIn
  • Print
  • Pocket
  • Email
  • Tumblr
  • Reddit
  • Pinterest
  • WhatsApp
  • Skype

Like this:

Like Loading...

Podcast with Anatolii Ulitovskyi of UNmiss

28 Wednesday Dec 2022

Posted by mp3monster in Books, General, Podcasts, Technology

≈ Leave a comment

Tags

API, book, integration, manning, Oracle, podcast

Just before the Christmas break, I got to record an excellent podcast with Anatolii of UNmiss. It was a great conversation about Cloud Integration, APIs, and approaches to Cloud-based integration. While I am not in a consulting role in the conventional sense, a lot of an Evangelist’s task is still to listen, understand, and, when necessary, challenge assumptions and help people understand how technologies can help address problems. This might include sketching out a journey of evolution and improvement. During the podcast, we discussed some of these ideas.

You can listen to the podcast audio or the live video stream of the conversation here.

In addition to some of the practices, we’ve used. The conversation touched upon books. My books are on the sidebar, including links to Manning, who, as a publisher, I’d recommend. I’ve previously blogged some reading recommendations and previously written some book reviews which may be of interest to anyone following up.

Share this:

  • Twitter
  • Facebook
  • LinkedIn
  • Print
  • Pocket
  • Email
  • Tumblr
  • Reddit
  • Pinterest
  • WhatsApp
  • Skype

Like this:

Like Loading...

Published content

22 Thursday Dec 2022

Posted by mp3monster in General, Technology

≈ Leave a comment

Tags

API, book, Cloud, DevRel, Notification, OCI, Oracle, Queue, SEDaily, Software Engineering Daily

We haven’t blogged too much recently as we have been busy helping get and producing content for my employer Oracle, working with Software Engineering Daily, and developing a collaborative book. So, I thought I’d pull together some links to these new resources.

Continue reading →

Share this:

  • Twitter
  • Facebook
  • LinkedIn
  • Print
  • Pocket
  • Email
  • Tumblr
  • Reddit
  • Pinterest
  • WhatsApp
  • Skype

Like this:

Like Loading...

LogSimulator New Feature – Custom Targets with OCI Logging example

14 Friday Oct 2022

Posted by mp3monster in Cloud, development, General, logsimulator, Oracle, Technology

≈ Leave a comment

Tags

book, loggenerator, logging, logsimulator, OCI, Prometheus

Those who have been using my Logging in Action book will know that to help test the configuration of monitoring tools including Fluentd we have built a LogGenerator that can very easily play and replay logging events into a variety of destinations and formats. all written in Groovy to make the utility easy to run as a script and extend without needing to set up a proper Java development environment.

With the number of different destinations built into the script and the logic to load the source log events and format them the utility is getting rather large for a single file. Rather than letting it continue to grow as we add more destinations to pump log events too, I’ve extended the implementation so you can point to a Groovy file that implements the logic to send the log events. It only requires three simple methods to be implemented.

To demonstrate the feature we have created a custom extension and fully documented it. The extension allows you to send log events to the OCI Logging service. This includes an optional crude aggregation mechanism as sending individual log events is a little inefficient over REST. By doing this we can send synthetic or playback logs as if we’re an application in real-life to ensure that any alerting or routing for the logging works properly before we get anywhere production and do not need to run the application and induce error events.

Beyond this, we’re also thinking about creating a plugin to fire log events at Prometheus so we can send events using the Prometheus pushgateway. As a result, we can tune Prometheus’ configuration.

More improvements – refactoring the existing code

We will refactor the existing code to use the same approach which should make the code more maintainable, but the changes won’t stop the utility from working as it always has (so we won’t break out the existing output channels from the core).

We have also started to improve the code commenting – so hopefully it will make the code a bit more navigable.

Share this:

  • Twitter
  • Facebook
  • LinkedIn
  • Print
  • Pocket
  • Email
  • Tumblr
  • Reddit
  • Pinterest
  • WhatsApp
  • Skype

Like this:

Like Loading...

Practical Steps when it comes to writing a technical book

11 Tuesday Oct 2022

Posted by mp3monster in Books, General, manning, Oracle Press, Packt, Technology

≈ 1 Comment

Tags

author, book, tips, writing

Following my article on Software Engineering Daily, here are some practical things that will help you if you’re considering taking on a technical book project.

Identifying a Publisher

While it is easy to self-publish today. The recognition comes from having worked with a traditional publisher as they have processes that ensure a level of quality. Not all publishers are equal, and some publishers are attributed with more prestige than others. In addition to this, some publishers are willing to take a risk on a subject and/or author. Have a look at the titles already published, and whether there are any publishers you can connect to.

When comes to contacting the publishers, most of their websites will have a page for recruiting authors. Some are easier to find than others. Here are a couple:

  • Packt
  • Manning
  • Pearson
  • APress

If, or when you get to talk to a publisher it is worth ensuring you understand how their editorial process works and what is expected from you? Plus what happens if you find yourself in the position of not being able to work to the original schedule. Day-to-day work can get in the way which you hadn’t expected.

Continue reading →

Share this:

  • Twitter
  • Facebook
  • LinkedIn
  • Print
  • Pocket
  • Email
  • Tumblr
  • Reddit
  • Pinterest
  • WhatsApp
  • Skype

Like this:

Like Loading...

Contributing to Software Engineering Daily

10 Monday Oct 2022

Posted by mp3monster in General, Technology

≈ Leave a comment

Tags

blog, book, SE Daily, software engineering, writing

For a long time, I’ve tracked and read articles on Software Engineering Daily. We’ll day represents what is hopefully the first of many articles that we will write for them. The article is about the kind of people that make technical book authors, and the perception we have of authors – so if you’re interested check it out here.

Some more content on the subject of books …

  • What does it take to write a tech book?
  • Creating screenshots of application shells – easing the writing process
  • Practical Steps when it comes to writing a technical book

Share this:

  • Twitter
  • Facebook
  • LinkedIn
  • Print
  • Pocket
  • Email
  • Tumblr
  • Reddit
  • Pinterest
  • WhatsApp
  • Skype

Like this:

Like Loading...

Coding over Cocktails with Fluentd

06 Friday May 2022

Posted by mp3monster in Books, Fluentd, General, manning, Podcasts, Technology

≈ Leave a comment

Tags

book, Coding over Cocktails, Logging in Action, podcast, Toro Cloud

I’ve been fortunate enough to appear on a podcast with the excellent Coding Over Cocktails team from Toro Cloud. we got to talk about some of the ideas discussed in my Logging In Action book. You can check the podcast out via their website which includes all the episode details and links to all the platforms that host the podcast. There have been some great previous guests such as Luis Weir (my old boss), Chris Richardson of Microservices.io, Matthew Reinbold from Postman, Sam Newman to name just a few.

Share this:

  • Twitter
  • Facebook
  • LinkedIn
  • Print
  • Pocket
  • Email
  • Tumblr
  • Reddit
  • Pinterest
  • WhatsApp
  • Skype

Like this:

Like Loading...

Handling multi-line log entries with Fluentd

19 Tuesday Apr 2022

Posted by mp3monster in Books, Fluentd, General, logsimulator, manning, Technology

≈ Leave a comment

Tags

book, Fluentd, multiline, parser, regex

One aspect of logging I didn’t directly address with my Fluentd book was consuming multiline logs, such as those you’ll often see when a stack trace is included in the log output. Implementing the feature with Fluentd isn’t hugely complex as it leverages the use of regular expressions (addressed in the book in more depth) to recognize the 1st line in a multiline log entry and for subsequent lines.

I didn’t address it for a couple of reasons:

  • Using parsers is fairly inefficient, particularly when you’re using a parser to just decide how to then transform a line (this is why I’m not a huge fan of some of the 12 Factor App‘s recommendations when it comes to logging).
  • Incorporating into your Fluentd parser configurations for specific app log setups is arguably increasing the level of coupling.
  • Many logging frameworks can talk directly to Fluentd as we saw in the book. This is can be more efficient, which means that the log event is more likely to be passed over in a structured format (therefore less work to do).
  • Alternatively, frameworks like Log4J2 have the means to strip line feeds, etc at source – https://logging.apache.org/log4j/2.x/manual/layouts.html (see replace)

But let’s also be realistic, many applications will be configured to simply log to a file and aren’t likely to be changed. At which point we do need to process such situations, so is it done? The process remains largely the same as the tail plugin we illustrated. Except we introduce a different parser called multiline. The documentation provided by Fluentd includes several examples of multiline configurations that will work for default log formats (such as Log4J and Rails). If we took our most basic source setup:

<source>
  @type tail
  path ./Chapter3/basic-file.*
  read_lines_limit 5
  tag simpleFile
  <parse>
    @type none
  </parse>
</source>

Then assuming our log Simulator played back multiline logs (the provided configuration doesn’t do that) extended to consume standard Log4J2 logs we would have a configuration as follows:

<source>
  @type tail
  path ./Chapter3/basic-file.*
  read_lines_limit 5
  tag simpleFile
  <parse>
     @type multiline
     format_firstline /\d{4}-\d{1,2}-\d{1,2}/
     format1 /^(?<time>\d{4}-\d{1,2}-\d{1,2} \d{1,2}:\d{1,2}:\d{1,2}) \[(?<thread>.*)\] (?<level>[^\s]+)(?<message>.*)/
  </parse>
</source>

As you can see we’ve set the parser type to multiline. Then there are two regular expressions, format_firstline is used to help recognize the start of a log event. Every line of the log is tested with this expression as we now assume unless this produces a valid result that the line will be part of a multiline event. If you look at the expression you’ll realize it is looking for a DateTime stamp in the form YYYY-MM-DD. This does mean if you generate a log that starts with the date even if it is part of a multiline output then you’ll trip up the parser. You could extend the expression – but the longer it is the slower the processing.

Following format_firstline we have in our example format1 which describes how to process the first line. This can be extended to define how to handle subsequent lines but this could be multiple format definitions. They do need to be presented in numerical order eg. format1, format2, format, and so on.

LogSimulator – Playing back multi-line logs

The Log simulator uses a very similar mechanism to Fluentd to understand how to playback multiple line logs. When it is reading the log lines in for replay it uses a regular expression to recognize the start of a new log entry (FIRSTOFMULTILINEREGEX) defined in the properties file. The simulator will concatenate lines together until it either hits the end of the file or has a new line that complies with the REGEX. It stores the line with an encoded /n (newline character). It will then print the log using the format specified and will allow the /n to create a newline (or not) based on another config parameter (ALLOWNL).

Share this:

  • Twitter
  • Facebook
  • LinkedIn
  • Print
  • Pocket
  • Email
  • Tumblr
  • Reddit
  • Pinterest
  • WhatsApp
  • Skype

Like this:

Like Loading...

Web Assembly – Free O’Reilly Book

13 Wednesday Apr 2022

Posted by mp3monster in Books, General

≈ Leave a comment

Tags

book, free, oreilly, WASM, WebAssembly

Red Panda is currently making the O’Reilly book by Brian Sletten – WebAssembly the Definitive Guide free to download (well, the price of your email address).

So why is worth snagging a download? Well, WebAssembly (Wasm) is the new way to deliver browser-based solutions to a client without having to develop with JavaScript. As a result, you have the chance to reuse more of your existing code. It also has the benefit of having been precompiled to a binary form, offering code security and compact download. But Wasm also has been seeing adoption in back-end server solutions as well, particularly for feature extensions, etc.

Share this:

  • Twitter
  • Facebook
  • LinkedIn
  • Print
  • Pocket
  • Email
  • Tumblr
  • Reddit
  • Pinterest
  • WhatsApp
  • Skype

Like this:

Like Loading...

Istio In Action

25 Friday Mar 2022

Posted by mp3monster in Book Reviews, Books, General, manning

≈ Leave a comment

Tags

book, Istio, K8s, Kubernetes, manning, mesh, mindmap

Christian Posta and Rinor Maloku’s book with Manning, Istio In Action has just been published. I’ve previously said it’s a good book, and that’s not surprising given Christian’s role at solo.io. When the final chapters became available I started to go through it in more detail and built a mind map (As with the recent review of Kubernetes best practices). The map can be seen below.

As you can see the map is very substantial reflecting on the depth and value of the book. For those who look at the maps, may notice there are a couple of chapters not fully mapped. I will update the map to fill those gaps in, but given they focus on monitoring and observability, I was less concerned about those areas given my own writing. The book’s exercises are very much built around using Docker Desktop making it very easy to spin up the examples and exercises. If you want to know about Istio Service Mesh on K8s then I’d recommend it.

Reading through the book, I’ve learned details that I was not entirely aware of, for example the integration of non K8s workloads into the mesh. The tuning of Istio to keep it highly performant with a lot of workloads.

The book can be obtained from:

  • Manning
  • Amazon

And other retailers of course.

Share this:

  • Twitter
  • Facebook
  • LinkedIn
  • Print
  • Pocket
  • Email
  • Tumblr
  • Reddit
  • Pinterest
  • WhatsApp
  • Skype

Like this:

Like Loading...
← Older posts

Aliases

  • phil-wilkins.uk
  • cloud-native.info
  • oracle.cloud-native.info

I work for Oracle, all opinions here are my own & do not necessarily reflect the views of Oracle

Oracle Ace Director Alumni

TOGAF 9

Logging in Action

Oracle Cloud Integration Book

API Platform Book


Oracle Dev Meetup London

Categories

  • App Ideas
  • Books
    • Book Reviews
    • manning
    • Oracle Press
    • Packt
  • Enterprise architecture
  • General
    • economy
    • LinkedIn
    • Website
  • Music
    • Music Resources
    • Music Reviews
  • Photography
  • Podcasts
  • Technology
    • APIs & microservices
    • chatbots
    • Cloud
    • Cloud Native
    • Dev Meetup
    • development
      • languages
        • node.js
    • drone
    • Fluentd
    • logsimulator
    • mindmap
    • OMESA
    • Oracle
      • API Platform CS
        • tools
      • Helidon
      • ITSO & OEAF
      • Java Cloud
      • NodeJS Cloud
      • OIC – ICS
      • Oracle Cloud Native
      • OUG
    • railroad diagrams
    • TOGAF
  • xxRetired

My Other Web Content & Contributions

  • Amazon Author entry
  • API Platform
  • Dev Meetup (co-managed)
  • Fluentd Book
  • ICS Book Website
  • OMESA
  • Ora World
  • Oracle Community Directory
  • Packt Author Bio
  • Phil on Blogs.Oracle.com
  • Sessionize Profile

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 2,540 other subscribers

RSS

RSS Feed RSS - Posts

RSS Feed RSS - Comments

February 2023
M T W T F S S
 12345
6789101112
13141516171819
20212223242526
2728  
« Jan    

Twitter

  • File system replication is now available as a fully managed solution for your enterprise workloads with #OCI File S… twitter.com/i/web/status/1…Next Tweet: 1 day ago
  • Find out how NIBIO's Smart Forest research center utilizes @OracleCloud to store, process, and apply #AI to data to… twitter.com/i/web/status/1…Next Tweet: 2 days ago
  • .@TechArena's latest ebook "Seven Strategies for Maximizing Organizational Return” highlights #OCI. Bev Crair discu… twitter.com/i/web/status/1…Next Tweet: 2 days ago
  • Clever use of SSH tunnelling, Applies to multiple Linux flavours and Windows .... blogs.oracle.com/developers/pos…Next Tweet: 2 days ago
  • 25% done with Let's Do It, by Bob Stanley goodreads.com/user_status/sh…Next Tweet: 2 days ago
Follow @mp3monster

History

Speaker Recognition

Open Source Summit Speaker

Flickr Pics

Pembroke CastleSeven Bridge Crossing
More Photos

    Social

    • View @mp3monster’s profile on Twitter
    • View philwilkins’s profile on LinkedIn
    • View mp3monster’s profile on GitHub
    • View mp3monster’s profile on Flickr
    • View philmp3monster’s profile on Twitch
    Follow Phil (aka MP3Monster)'s Blog on WordPress.com

    Blog at WordPress.com.

    • Follow Following
      • Phil (aka MP3Monster)'s Blog
      • Join 216 other followers
      • Already have a WordPress.com account? Log in now.
      • Phil (aka MP3Monster)'s Blog
      • Customize
      • Follow Following
      • Sign up
      • Log in
      • Report this content
      • View site in Reader
      • Manage subscriptions
      • Collapse this bar
     

    Loading Comments...
     

    You must be logged in to post a comment.

      Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
      To find out more, including how to control cookies, see here: Our Cookie Policy
      %d bloggers like this: