• Home
  • Site Aliases
    • www.cloud-native.info
    • oracle.cloud-native.info
    • Phil-Wilkins.uk
  • About
    • Background
    • Presenting Activities
    • Internet Profile
      • LinkedIn
    • About
  • Books & Publications
    • Logging in Action with Fluentd, Kubernetes and More
      • Logging in Action with Fluentd – Book
      • Fluentd Book Resources
      • Log Generator
    • API & API Platform
      • API Useful Resources
    • Oracle Integration
      • Book Website
      • Useful Reading Sources
    • Publication Contributions
  • Resources
    • GitHub
    • Oracle Integration Site
    • Oracle Resources
    • Mindmaps Index
    • Useful Tech Resources
    • Python Setup & related stuff
  • Music
    • Music Reading

Phil (aka MP3Monster)'s Blog

~ from Technology to Music

Phil (aka MP3Monster)'s Blog

Tag Archives: Ruby

Securing credentials in Fluentd configurations

07 Tuesday Jun 2022

Posted by mp3monster in development, Fluentd, General, manning, Technology

≈ Leave a comment

Tags

Conjur, env vars, environment variables, Fluentd, Hashicorp, open source, Ruby, secrets, Security, slack, token, Vault

When configuring Fluentd we often need to provide credentials to access event sources, targets, and associated services such as notification tools like Slack and PagerDuty. The challenge is that we don’t want the credentials to be in clear text in the Fluentd configuration.

Using Env Vars

In the Logging In Action with Fluentd book, we illustrated how we can take the sensitive values from environment variables so the values don’t show up in the configuration file. But, we’ve seen regularly the question of how secure is this, can’t the environment variable be seen by everyone on that machine?

The answer to this question comes down to having a deeper understanding of how environment variables work. There is a really good explanation here. The long and short of it is that environment variables can only be seen by the process that creates the variable and any child process will receive a copy of the parent’s variables.

This means that if we create the variable in a shell, only that shell and any processes launched by that shell can see the environment variable. So as long as we don’t set variables up as part of a system-level configuration then we already have a level of security. So we could wrap the start of Fluentd with a script that sets the environment variables needed. Then everything launches that script.

An even better way?

Continue reading →

Share this:

  • Twitter
  • Facebook
  • LinkedIn
  • Print
  • Pocket
  • Email
  • Tumblr
  • Reddit
  • Pinterest
  • WhatsApp
  • Skype

Like this:

Like Loading...

Mastering Puppet Review

09 Tuesday Sep 2014

Posted by mp3monster in Books, General, Technology

≈ Leave a comment

Tags

book, EasticSearch, Foreman, Kibana, Logstash, mCollective, Packt, Pulp, Puppet, review, Ruby, Splunk, Thomas Uphill, YAML

Packt’s Mastering Puppet kicks off with substantial first chapter on how to setup Puppet in a manner that can then scale. The core of this is driven by an explanation of the constituent parts of a Puppet solution and where the workload is. In terms of execution this is as much about understanding the configuration of things like Apache, Passenger and Ningx as it is Puppet. As part of the explanation there are indicative numbers in terms of supportable scale which reflects the knowledge of the product.

Looking at configuration distribution for headless deployments with Git is a solid well considered piece and the writing suggests considers all the needs of a solid deployment of a production quality solution such as access control, whilst supporting collaborative working etc. it would be interesting to have seen how that would have stacked against capabilities such as Zookeeper.

As we move through the chapters the books continues with more advanced themes such as using Hiera as a object hierarchical framework for managing configuration and on into leveraging Puppet forge and various Git repositories (and the challenges when linking to git repositories of the latest code vs a release). With the repositories we can draw in additional tooling and how to incorporate these capabilities into a deployment. This includes looking at several modules that practical experience from the author would recommend.

By chapter 6 we’re into writing our own custom modules and facts and deploying them. So you can do things such as create modules to manage your custom solutions.

The next natural step is to look at the reporting aspects of Puppet, orchestration through marionette collective (mCollective). Obviously to report you need to gather the activity information, so the book touches on the out of the box (OOTB) approach and moves onto the idea of using IRC; presentation via Foreman and Puppet Dashboard. Finally then with a reporting view, the next step is to dynamically query the nodes in Puppet environment which uses mcollective to communicate back & forth with the nodes.

So now we have a dynamically configurable set of Nodes, which can report and have dynamic querying against the nodes.  Final chapters cover the use of things like PuppetDB, roles & profiles and developing and debugging your puppet environment.

Reading the book, I get the feeling that a fair grasp of Linux system administration would help (i.e. a bit more than the average developer). There are a few useful touches that I think could have been included, such as external references such as man pages for RPM or site for the Pulp tool mentioned. But, as criticisms go, this as much me being too lazy to Google. The only other refinement would be inclusion of some diagrams to support the words. As they say a picture can tell a 1000 words, even if this was to just show the hierarchy or directory structures involved.

Compared to the recently reviewed Puppet Reporting book, this book isn’t for someone starting out with Puppet (but the Packt site says as much). You atleast need to have got some basic understanding or practical exposure to Puppet,  and exposure to a development environment is an added bonus.  So if you’re setting out with Puppet you might consider starting with the Puppet 3 Beginner’s Guide (Amazon) or Instant Puppet 3 Starter (Amazon).  Having got those under your belt, try this book to to really develop the use of Puppet configuration and deployment.  When it comes to reporting I’d look at this book along with reporting book (reviewed here).  This book feels like more options are on offer, but Puppet Reporting is a lot richer (but you’d expect that given the different book emphasis).

In summary – good solid book, full of practical experience and ideas.  But don’t try to use this as a jumpstart to Puppet.

Below are a few links I thought might be helpful as they aren’t in the book:

  • YAML – human readable serialization format
  • Pulp – software repository management app
  • Ruby – Open Source OO programming language
  • Foreman – tool capable of extending puppet to deliver PXE capabilities along with capabilities such as reporting
  • Splunk – BigData style analytics on log files etc
  • Elasticsearch / Logstash / Kibana (ELK) – set of tools to provide analytics against log files
  • ActiveMQ – Apache implementation of a JMS compliant messaging solution used my mcollective

Mastering Puppet at Amazon.

Share this:

  • Twitter
  • Facebook
  • LinkedIn
  • Print
  • Pocket
  • Email
  • Tumblr
  • Reddit
  • Pinterest
  • WhatsApp
  • Skype

Like this:

Like Loading...

Puppet Reporting & Monitoring Book Review

27 Friday Jun 2014

Posted by mp3monster in Book Reviews, Books, General, Packt, Technology

≈ Leave a comment

Tags

book, Michael Duffy, monitoring, Packt, packtpub, Puppet, reporting, review, Ruby

So the Packt book (Puppet Reporting and Monitoring) focuses down on a couple of aspects of the Puppet Toolset, as a result this is a relatively short book with only a couple hundred pages. As an enterprise architect I am no expert Puppet practitioner, my knowledge of Ruby is high level (part of the reason I reviewed this book is I wanted to better understand the art of the possible in these areas).  But despite this the book does an exceptionally good job defining a context and then explaining and showing what could be done, down to code examples.  In doing so, the author Michael Duffy introduces a number of open source libraries that can be leveraged to provide dashboard views, presentation of report content whilst maximising the leveraging of the Puppet ecosystem such as the Puppet DB (an abstracted database with a REST + JSON API).  The book goes beyond just implementation of monitoring and reporting but also engages with considerations such as deployment.  without ‘boiling the ocean’ the book provides a very good illustrations of the art of the possible and provides plenty of references to source information so working how you want to implement you own solutions.

My only criticism of the book, and it is a minor one at that is a few more diagrams to help illustrate ideas (particularly in the first chapter when discussing deployment considerations) would help get ideas across easily.

On the strength of this  book,  I hope that Michael considers taking on other authoring projects as this has been one of the best written technical books I’ve read in sometime.

Puppet Reporting & Monitoring
Useful Links:

  • Book – http://bit.ly/1qbSxKC
  • Puppet Labs – http://puppetlabs.com/
  • Puppet DB – http://docs.puppetlabs.com/puppetdb/
  • Ruby – https://www.ruby-lang.org/
  • Michael Duffy – http://www.stunthamster.com/, http://uk.linkedin.com/pub/michael-duffy/40/809/17a

Share this:

  • Twitter
  • Facebook
  • LinkedIn
  • Print
  • Pocket
  • Email
  • Tumblr
  • Reddit
  • Pinterest
  • WhatsApp
  • Skype

Like this:

Like Loading...

Aliases

  • phil-wilkins.uk
  • cloud-native.info
  • oracle.cloud-native.info

I work for Oracle, all opinions here are my own & do not necessarily reflect the views of Oracle

Oracle Ace Director Alumni

TOGAF 9

Logging in Action

Oracle Cloud Integration Book

API Platform Book


Oracle Dev Meetup London

Categories

  • App Ideas
  • Books
    • Book Reviews
    • manning
    • Oracle Press
    • Packt
  • Enterprise architecture
  • General
    • economy
    • LinkedIn
    • Website
  • Music
    • Music Resources
    • Music Reviews
  • Photography
  • Podcasts
  • Technology
    • APIs & microservices
    • chatbots
    • Cloud
    • Cloud Native
    • Dev Meetup
    • development
      • languages
        • node.js
    • drone
    • Fluentd
    • logsimulator
    • mindmap
    • OMESA
    • Oracle
      • API Platform CS
        • tools
      • Helidon
      • ITSO & OEAF
      • Java Cloud
      • NodeJS Cloud
      • OIC – ICS
      • Oracle Cloud Native
      • OUG
    • railroad diagrams
    • TOGAF
  • xxRetired

My Other Web Content & Contributions

  • Amazon Author entry
  • API Platform
  • Dev Meetup (co-managed)
  • Fluentd Book
  • ICS Book Website
  • OMESA
  • Ora World
  • Oracle Community Directory
  • Packt Author Bio
  • Phil on Blogs.Oracle.com
  • Sessionize Profile

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 2,573 other subscribers

RSS

RSS Feed RSS - Posts

RSS Feed RSS - Comments

March 2023
M T W T F S S
 12345
6789101112
13141516171819
20212223242526
2728293031  
« Feb    

Twitter

  • Learn how @OracleCloud can help you improve the efficiency of your business operations at the upcoming Level Up eve… twitter.com/i/web/status/1…Next Tweet: 20 hours ago
  • Join Juan Loaiza for the Data Strategies Day keynote at Level Up to learn how to eliminate complexity by leveraging… twitter.com/i/web/status/1…Next Tweet: 1 day ago
  • RT @WunderlichRd: Great post by @mp3monster around how APIs are relevant in so many industries! lnkd.in/eshagCDKNext Tweet: 1 day ago
  • King’s College Hospital London in Dubai announces a strategic collaboration with Oracle Cerner to help accelerate i… twitter.com/i/web/status/1…Next Tweet: 1 day ago
  • Catch the @Oracle and @NVIDIA teams at #GDC23, as they'll be collaborating to bring the full NVIDIA accelerated com… twitter.com/i/web/status/1…Next Tweet: 1 day ago
Follow @mp3monster

History

Speaker Recognition

Open Source Summit Speaker

Flickr Pics

Pembroke CastleSeven Bridge Crossing
More Photos

    Social

    • View @mp3monster’s profile on Twitter
    • View philwilkins’s profile on LinkedIn
    • View mp3monster’s profile on GitHub
    • View mp3monster’s profile on Flickr
    • View philmp3monster’s profile on Twitch
    Follow Phil (aka MP3Monster)'s Blog on WordPress.com

    Blog at WordPress.com.

    • Follow Following
      • Phil (aka MP3Monster)'s Blog
      • Join 218 other followers
      • Already have a WordPress.com account? Log in now.
      • Phil (aka MP3Monster)'s Blog
      • Customize
      • Follow Following
      • Sign up
      • Log in
      • Report this content
      • View site in Reader
      • Manage subscriptions
      • Collapse this bar
     

    Loading Comments...
     

    You must be logged in to post a comment.

      Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
      To find out more, including how to control cookies, see here: Our Cookie Policy
      %d bloggers like this: