• Home
  • Site Aliases
    • www.cloud-native.info
    • oracle.cloud-native.info
    • Phil-Wilkins.uk
  • About
    • Background
    • Presenting Activities
    • Internet Profile
      • LinkedIn
    • About
  • Books & Publications
    • Logging in Action with Fluentd, Kubernetes and More
      • Logging in Action with Fluentd – Book
      • Fluentd Book Resources
      • Log Generator
    • API & API Platform
      • API Useful Resources
    • Oracle Integration
      • Book Website
      • Useful Reading Sources
    • Publication Contributions
  • Resources
    • GitHub
    • Mindmaps Index
    • Oracle Integration Site
    • Useful Tech Resources …
      • Oracle Tech Resources inc Open Source
      • Useful Tech Resources
      • Python Setup & related stuff
  • Music

Phil (aka MP3Monster)'s Blog

~ from Technology to Music

Phil (aka MP3Monster)'s Blog

Tag Archives: Ruby

Securing credentials in Fluentd configurations

07 Tuesday Jun 2022

Posted by mp3monster in development, Fluentd, General, Technology

≈ Leave a comment

Tags

Conjur, env vars, environment variables, Fluentd, Hashicorp, open source, Ruby, secrets, Security, slack, token, Vault

When configuring Fluentd we often need to provide credentials to access event sources, targets, and associated services such as notification tools like Slack and PagerDuty. The challenge is that we don’t want the credentials to be in clear text in the Fluentd configuration.

Using Env Vars

In the Logging In Action with Fluentd book, we illustrated how we can take the sensitive values from environment variables so the values don’t show up in the configuration file. But, we’ve seen regularly the question of how secure is this, can’t the environment variable be seen by everyone on that machine?

The answer to this question comes down to having a deeper understanding of how environment variables work. There is a really good explanation here. The long and short of it is that environment variables can only be seen by the process that creates the variable and any child process will receive a copy of the parent’s variables.

This means that if we create the variable in a shell, only that shell and any processes launched by that shell can see the environment variable. So as long as we don’t set variables up as part of a system-level configuration then we already have a level of security. So we could wrap the start of Fluentd with a script that sets the environment variables needed. Then everything launches that script.

An even better way?

Continue reading →

Share this:

  • Twitter
  • Facebook
  • LinkedIn
  • Print
  • Pocket
  • Email
  • Tumblr
  • Reddit
  • Pinterest
  • WhatsApp
  • Skype

Like this:

Like Loading...

Mastering Puppet Review

09 Tuesday Sep 2014

Posted by mp3monster in Books, General, Technology

≈ Leave a comment

Tags

book, EasticSearch, Foreman, Kibana, Logstash, mCollective, Packt, Pulp, Puppet, review, Ruby, Splunk, Thomas Uphill, YAML

Packt’s Mastering Puppet kicks off with substantial first chapter on how to setup Puppet in a manner that can then scale. The core of this is driven by an explanation of the constituent parts of a Puppet solution and where the workload is. In terms of execution this is as much about understanding the configuration of things like Apache, Passenger and Ningx as it is Puppet. As part of the explanation there are indicative numbers in terms of supportable scale which reflects the knowledge of the product.

Looking at configuration distribution for headless deployments with Git is a solid well considered piece and the writing suggests considers all the needs of a solid deployment of a production quality solution such as access control, whilst supporting collaborative working etc. it would be interesting to have seen how that would have stacked against capabilities such as Zookeeper.

As we move through the chapters the books continues with more advanced themes such as using Hiera as a object hierarchical framework for managing configuration and on into leveraging Puppet forge and various Git repositories (and the challenges when linking to git repositories of the latest code vs a release). With the repositories we can draw in additional tooling and how to incorporate these capabilities into a deployment. This includes looking at several modules that practical experience from the author would recommend.

By chapter 6 we’re into writing our own custom modules and facts and deploying them. So you can do things such as create modules to manage your custom solutions.

The next natural step is to look at the reporting aspects of Puppet, orchestration through marionette collective (mCollective). Obviously to report you need to gather the activity information, so the book touches on the out of the box (OOTB) approach and moves onto the idea of using IRC; presentation via Foreman and Puppet Dashboard. Finally then with a reporting view, the next step is to dynamically query the nodes in Puppet environment which uses mcollective to communicate back & forth with the nodes.

So now we have a dynamically configurable set of Nodes, which can report and have dynamic querying against the nodes.  Final chapters cover the use of things like PuppetDB, roles & profiles and developing and debugging your puppet environment.

Reading the book, I get the feeling that a fair grasp of Linux system administration would help (i.e. a bit more than the average developer). There are a few useful touches that I think could have been included, such as external references such as man pages for RPM or site for the Pulp tool mentioned. But, as criticisms go, this as much me being too lazy to Google. The only other refinement would be inclusion of some diagrams to support the words. As they say a picture can tell a 1000 words, even if this was to just show the hierarchy or directory structures involved.

Compared to the recently reviewed Puppet Reporting book, this book isn’t for someone starting out with Puppet (but the Packt site says as much). You atleast need to have got some basic understanding or practical exposure to Puppet,  and exposure to a development environment is an added bonus.  So if you’re setting out with Puppet you might consider starting with the Puppet 3 Beginner’s Guide (Amazon) or Instant Puppet 3 Starter (Amazon).  Having got those under your belt, try this book to to really develop the use of Puppet configuration and deployment.  When it comes to reporting I’d look at this book along with reporting book (reviewed here).  This book feels like more options are on offer, but Puppet Reporting is a lot richer (but you’d expect that given the different book emphasis).

In summary – good solid book, full of practical experience and ideas.  But don’t try to use this as a jumpstart to Puppet.

Below are a few links I thought might be helpful as they aren’t in the book:

  • YAML – human readable serialization format
  • Pulp – software repository management app
  • Ruby – Open Source OO programming language
  • Foreman – tool capable of extending puppet to deliver PXE capabilities along with capabilities such as reporting
  • Splunk – BigData style analytics on log files etc
  • Elasticsearch / Logstash / Kibana (ELK) – set of tools to provide analytics against log files
  • ActiveMQ – Apache implementation of a JMS compliant messaging solution used my mcollective

Mastering Puppet at Amazon.

Share this:

  • Twitter
  • Facebook
  • LinkedIn
  • Print
  • Pocket
  • Email
  • Tumblr
  • Reddit
  • Pinterest
  • WhatsApp
  • Skype

Like this:

Like Loading...

Puppet Reporting & Monitoring Book Review

27 Friday Jun 2014

Posted by mp3monster in Book Reviews, Books, General, Packt, Technology

≈ Leave a comment

Tags

book, Michael Duffy, monitoring, Packt, packtpub, Puppet, reporting, review, Ruby

So the Packt book (Puppet Reporting and Monitoring) focuses down on a couple of aspects of the Puppet Toolset, as a result this is a relatively short book with only a couple hundred pages. As an enterprise architect I am no expert Puppet practitioner, my knowledge of Ruby is high level (part of the reason I reviewed this book is I wanted to better understand the art of the possible in these areas).  But despite this the book does an exceptionally good job defining a context and then explaining and showing what could be done, down to code examples.  In doing so, the author Michael Duffy introduces a number of open source libraries that can be leveraged to provide dashboard views, presentation of report content whilst maximising the leveraging of the Puppet ecosystem such as the Puppet DB (an abstracted database with a REST + JSON API).  The book goes beyond just implementation of monitoring and reporting but also engages with considerations such as deployment.  without ‘boiling the ocean’ the book provides a very good illustrations of the art of the possible and provides plenty of references to source information so working how you want to implement you own solutions.

My only criticism of the book, and it is a minor one at that is a few more diagrams to help illustrate ideas (particularly in the first chapter when discussing deployment considerations) would help get ideas across easily.

On the strength of this  book,  I hope that Michael considers taking on other authoring projects as this has been one of the best written technical books I’ve read in sometime.

Puppet Reporting & Monitoring
Useful Links:

  • Book – http://bit.ly/1qbSxKC
  • Puppet Labs – http://puppetlabs.com/
  • Puppet DB – http://docs.puppetlabs.com/puppetdb/
  • Ruby – https://www.ruby-lang.org/
  • Michael Duffy – http://www.stunthamster.com/, http://uk.linkedin.com/pub/michael-duffy/40/809/17a

Share this:

  • Twitter
  • Facebook
  • LinkedIn
  • Print
  • Pocket
  • Email
  • Tumblr
  • Reddit
  • Pinterest
  • WhatsApp
  • Skype

Like this:

Like Loading...

Aliases

  • phil-wilkins.uk
  • cloud-native.info
  • oracle.cloud-native.info

I work for Oracle, all opinions here are my own & do not necessarily reflect the views of Oracle

Oracle Ace Director Alumni

TOGAF 9

Logging in Action

Oracle Cloud Integration Book

API Platform Book

Oracle Dev Meetup London

Categories

  • App Ideas
  • Books
    • Book Reviews
    • manning
    • Oracle Press
    • Packt
  • Enterprise architecture
  • General
    • economy
    • LinkedIn
    • Website
  • Music
    • Music Resources
    • Music Reviews
  • Photography
  • Technology
    • APIs & microservices
    • chatbots
    • Cloud
    • Cloud Native
    • Dev Meetup
    • development
      • languages
        • node.js
    • drone
    • Fluentd
    • logsimulator
    • mindmap
    • OMESA
    • Oracle
      • API Platform CS
        • tools
      • Helidon
      • ITSO & OEAF
      • Java Cloud
      • NodeJS Cloud
      • OIC – ICS
      • Oracle Cloud Native
      • OUG
    • TOGAF
  • xxRetired

Twitter

  • RT @RhonddaBryant: Why can’t we do this?Next Tweet: 4 days ago
  • Ready to improve your #CloudNative skills? Download these free ebooks for cloud native #developers. Learn about… twitter.com/i/web/status/1…Next Tweet: 5 days ago
  • Registration for @Oracle #CloudWorld is now open! Save $200 on your pass by registering early. We might be biased,… twitter.com/i/web/status/1…Next Tweet: 5 days ago
  • Apollo GraphQL – some pointers blog.mp3monster.org/2022/06/16/apo…Next Tweet: 1 week ago
  • K8s dashboard capability without needing to deploy K8s dashboard blog.mp3monster.org/2022/06/14/k8s…Next Tweet: 2 weeks ago
Follow @mp3monster

OraWorld

OraWorld

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 614 other followers

Blogs I Follow

  • MiddlewarEArchitecT
  • Site Title
  • All about modern software technologies & architectures
  • Rick's blog
  • A journey in development
  • Phil (aka MP3Monster)'s Blog
  • RedThunder.Blog
  • A millennial's musings
  • Shalindra's Blogs
  • BTplusMore
  • Creativenauts
  • PaaS Community Blog
  • RedStack
  • Musings of an Enterprise Software Technologist
  • The Open Group Blog
  • SutoCom Solutions
  • Rob's Wall Of Music
  • DataCentricSec.com
  • A World of Events

My Other Web Content & Contributions

  • All My Links
  • Amazon Author entry
  • API Platform
  • Dev Meetup (co-managed)
  • Fluentd Book
  • GitHub
  • http://phil-wilkins.uk/
  • ICS Book Website
  • Mindmaps
  • Monster's Photos
  • my Capgemini Profile
  • OMESA
  • Oracle Community Directory
  • Packt Author Bio
  • Phil on Blogs.Oracle.com

RSS

RSS Feed RSS - Posts

RSS Feed RSS - Comments

Calendar

June 2022
M T W T F S S
 12345
6789101112
13141516171819
20212223242526
27282930  
« May    

Other Pages

  • About
    • Internet Profile
    • Presenting Activities
  • Books & Publications
    • API & API Platform
      • API Useful Resources
      • Useful Reading Sources
    • Logging in Action with Fluentd, Kubernetes and More
    • Oracle Integration
  • Mindmaps Index
  • Useful Tech Resources
    • Oracle Tech Resources inc Open Source
    • Python Setup & related tips

Speaker Recognition

Open Source Summit Speaker

Flickr Pics

Sound InstallationSound InstallationSound InstallationSound Installation
More Photos

History

Goodreads

OraNA

Aggregated by OraNA

Blogroll

  • A Journey in Development
  • A Neate Blog
  • Amy Simpson Grange blog
  • Blog by Robert van Mölken (co-author on ICS book)
  • Developer Community
  • Exigency In Specie
  • Flexagon (FlexDepoy)
  • GitHub
  • http://phil-wilkins.uk/
  • Ora World
  • PaaS Community
  • SOA4U

Calendar

June 2022
M T W T F S S
 12345
6789101112
13141516171819
20212223242526
27282930  
« May    

Categories

  • App Ideas
  • Books
    • Book Reviews
    • manning
    • Oracle Press
    • Packt
  • Enterprise architecture
  • General
    • economy
    • LinkedIn
    • Website
  • Music
    • Music Resources
    • Music Reviews
  • Photography
  • Technology
    • APIs & microservices
    • chatbots
    • Cloud
    • Cloud Native
    • Dev Meetup
    • development
      • languages
        • node.js
    • drone
    • Fluentd
    • logsimulator
    • mindmap
    • OMESA
    • Oracle
      • API Platform CS
        • tools
      • Helidon
      • ITSO & OEAF
      • Java Cloud
      • NodeJS Cloud
      • OIC – ICS
      • Oracle Cloud Native
      • OUG
    • TOGAF
  • xxRetired

Twitter

  • RT @RhonddaBryant: Why can’t we do this?Next Tweet: 4 days ago
  • Ready to improve your #CloudNative skills? Download these free ebooks for cloud native #developers. Learn about… twitter.com/i/web/status/1…Next Tweet: 5 days ago
  • Registration for @Oracle #CloudWorld is now open! Save $200 on your pass by registering early. We might be biased,… twitter.com/i/web/status/1…Next Tweet: 5 days ago
  • Apollo GraphQL – some pointers blog.mp3monster.org/2022/06/16/apo…Next Tweet: 1 week ago
  • K8s dashboard capability without needing to deploy K8s dashboard blog.mp3monster.org/2022/06/14/k8s…Next Tweet: 2 weeks ago
Follow @mp3monster

Flickr Pics

Sound InstallationSound InstallationSound InstallationSound Installation
More Photos

My Other Web Content & Contributions

  • All My Links
  • Amazon Author entry
  • API Platform
  • Dev Meetup (co-managed)
  • Fluentd Book
  • GitHub
  • http://phil-wilkins.uk/
  • ICS Book Website
  • Mindmaps
  • Monster's Photos
  • my Capgemini Profile
  • OMESA
  • Oracle Community Directory
  • Packt Author Bio
  • Phil on Blogs.Oracle.com

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 614 other followers

History

Other Pages

  • About
    • Internet Profile
    • Presenting Activities
  • Books & Publications
    • API & API Platform
      • API Useful Resources
      • Useful Reading Sources
    • Logging in Action with Fluentd, Kubernetes and More
    • Oracle Integration
  • Mindmaps Index
  • Useful Tech Resources
    • Oracle Tech Resources inc Open Source
    • Python Setup & related tips

RSS

RSS Feed RSS - Posts

RSS Feed RSS - Comments

Social

  • View @mp3monster’s profile on Twitter
  • View philwilkins’s profile on LinkedIn
  • View mp3monster’s profile on GitHub
  • View mp3monster’s profile on Flickr
  • View philmp3monster’s profile on Twitch
Follow Phil (aka MP3Monster)'s Blog on WordPress.com

Blog at WordPress.com.

MiddlewarEArchitecT

Site Title

All about modern software technologies & architectures

Cloud-native, technology and software architecture

Rick's blog

End-to-End OIC to SAP integration

A journey in development

A blog-post by blog-post journey of a ERP Cloud Solutions Degree Apprentice

Phil (aka MP3Monster)'s Blog

from Technology to Music

RedThunder.Blog

Demystifying cloud technologies...

A millennial's musings

Shalindra's Blogs

Technofunctional Blogs

BTplusMore

Business, Technology and more

Creativenauts

Personal, design, inspiration, interests.

PaaS Community Blog

by Jürgen Kress

RedStack

Oracle Database and Cloud Stuff

Musings of an Enterprise Software Technologist

My thoughts on Enterprise Software Technologies...and more.

The Open Group Blog

Achieving business objectives through technology standards

SutoCom Solutions

Success & Satisfaction with the Cloud

Rob's Wall Of Music

Thoughts of a lifelong music hoarder...

DataCentricSec.com

A World of Events

A Blog for Event and Data Analytics

  • Follow Following
    • Phil (aka MP3Monster)'s Blog
    • Join 614 other followers
    • Already have a WordPress.com account? Log in now.
    • Phil (aka MP3Monster)'s Blog
    • Customize
    • Follow Following
    • Sign up
    • Log in
    • Report this content
    • View site in Reader
    • Manage subscriptions
    • Collapse this bar
 

Loading Comments...
 

You must be logged in to post a comment.

    Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
    To find out more, including how to control cookies, see here: Our Cookie Policy
    %d bloggers like this: