Tag Archives: Ansible

Testing Terraform

26 Tuesday Jul 2022

Posted by in Cloud, General, Technology

≈ Leave a comment

Tags

, , , , , , , , ,

Infrastructure as Code (IaC) should be treated the same way as any other code. That is to say that we should be considering configuration management, testing, regression, code quality, and coverage. We should be addressing these points for the same reasons we address them with our application code. Such as ensuring that we don’t introduce bugs as things evolve and develop, ensuring that the code is maintainable over a prolonged period etc.

The problem is that the only real way to test IaC is to run it. Particularly with the likes of Terraform where it is largely declarative rather than containing a lot of logic. This point is nicely conveyed by Yevgeniy Brikman’s presentation (below)

The presentation goes on to illustrate Terratest which has the look and feel of JUnit or any other xUnit test framework. Terratest is implemented in Golang, But to be honest, given the nature of Terraform ( largely declarative meaning it enables ideas of composition and not sophisticated logic) it means the writing of the tests isn’t going to demand anything clever like how to achieve polymorphic behavior through Go’s type structures.

While Yevgeniy focussed on testing by invoking an application on the infrastructure deployed something we’ve described though our Platform Test logic (more here). You may wish to test things further by interrogating infrastructure components. For example, do I have the right number of nodes in a dynamic group or are container or server logs going into the cloud monitoring services.

Performing such checks is very easy with OCI as it provides a Golang SDK making it very easy to write tests that can call the OCI APIs and interrogate the setup. Better still when considering whether the Terraform configuration will behave correctly to support dynamic/auto-scaling can be done easily without modifying the Terraform configurations as part of the Terratest logic can include Go API calls to temporarily modify scaling triggers or invoking code that can stimulate OCI dynamic features.

Testing App Configuration

There is an interesting question to be considered. There is a point of separation between when to use Terraform (or Pulumi and others for that matter) and tools better suited to application deployment and configuration like Ansible and Chef. Therefore should we separate the testing of these details? Maybe I am too purist but seeing local and remote execs in Terraform as these actions are very opaque and can be used to conceal things or unwittingly depend on the way Terraform handles its dependency graph.

Of course, Ansible has its test framework ansible_test and has the means to measure test coverage. So one possibility is to treat Ansible as a separate module, independently test it, and then integrate its use in the wider picture of deploying infrastructure.

OKit – OCI Design (on Windows)

08 Tuesday Dec 2020

Posted by in Cloud, General, Oracle, Technology

≈ 1 Comment

Tags

, , , , , , , ,

OKit is a tremendous tool for the visual design and development for your Oracle Cloud environment. Visualizing your networks, positioning of service gateways and so on makes it a lot easier than filling in web forms or writing Terraform files as you can see the relationships between the different parts far more easily. For the same reason really that a lot of people use Visio and other tools for this work. The real beauty is that OKit can generate the Terraform and Ansible scripting that can then be used to deliver the implementation.

Okit for visual design of Oracle Cloud

The tool isn’t currently an official Oracle product, but something built by the Oracle A-Team (a small team of gurus who have a role blending developer advocacy, architect supporting customers for the special edge cases and providing thought leadership). But we can hope that someone brings it into the fold and perhaps even incorporates it securely into the cloud dashboard. In the mean time, the code in its entirety is available on GitHub.

Continue reading →

Ansible Book Review Part 4

14 Saturday Mar 2015

Posted by in Book Reviews, Books, Packt, Technology

≈ 1 Comment

Tags

, , , , , , , , , , , , , ,

This the final part of the detailed look at Packt book, Learning Ansible. As the book says in the opening to chapter 6 we’re into the back straight, into the final mile. The first of  two final  chapters look at provisioning of platforms on Amazon AWS, DigitalOcean and the use of the very hip and cool Docker plus updating your inventory of systems given that we have dynamically introduced new ones.  The approach is illustrated by not only instantiating servers but delivering a configured Hadoop cluster. As with everything else we’ve seen in Ansible there isn’t a standardised approach to all IaaS platforms as that restricts you the lowest common denominator which is contrary to Ansible goals described early on. But deploying the Hadoop elements on the two cloud IaaS providers is common.  Although the chapter is pretty short, I did have to read through this more carefully, as the book leverages a lot of demonstrated features from previous chapters (configuration arrays etc) which meant seeing the key element of the interaction with AWS was harder. It does mean if you tried diving into this chapter straight away, although not impossible does require a bit more investment from the reader to see all the value points. That said it is great to see through the use of the various features how easy to setup the provisioning in the cloud is, and the inventory update. Perhaps the win would have been to just so the simple provision and then the clever approach.

Chapter 7 focuses on Deployment. When I read this, I was a little nonplussed, hadn’t we been reading about this in the previous 6 chapters. But when you look at the definition provided:

“To position (troops) in readiness for combat, as along a front or line.”
Excerpt From: “Learning Ansible.” Packt Publishing. 

You can start to see the true target of what we’re really thinking about, which is the process of going from software build to production readiness. So having  gone through the software packaging activities you need to orchestrate the deployment across potentially multiple servers across a server farm. This orchestration piece is really just pulling together everything that has been explained before but also share some Ansible best practise. Then finally an examination of the Ansible approach for the nodes to pull deployments and updates.

The final piece of the book is an Appendix which looks at the work to bring Ansible to the Windows platform, Ansible Galaxy and Ansible Tower.  Ansible Galaxy is a repository of roles build by the Ansible community. Ansible Tower provides a web front end to the Ansible server. The Tower product is the commercial side of the Ansible company – and effectively sales here fund the full time Ansible development effort.

So to summarise …

The Learning Ansible book explains from first principles to the very rich capabilities of building packaging software, instantiating cloud servers or containers through to configuring systems and deploying applications into new environments; and then capturing instantiated system details into the Ansible inventory. How Ansible compares with the more established solutions in this space in the form of Puppet and Chef is discussed, and the pros and cons of the different tools. All the way through, the books has been written in an easy engaging manner. You might even say wonderfully written. The examples are very good with the possible exception of 2 cases (just merely good in my opinion), the examples are supported with very clear explanations that demonstrate the power of the Ansible product. Even if you choose not to use Ansible, this book does an excellent job of showing the value of not resorting to the ‘black art’ of system build and configuration and suggesting good ways to realising automation of this kind of activity, in many place undoubtedly thought provoking

Prior Review Parts:

Learning Ansible Review Part 3

08 Sunday Mar 2015

Posted by in Book Reviews, Books, Packt, Technology

≈ 1 Comment

Tags

, , , , , , , , , ,

Chapters 4 & 5 of Packt’s Learning Ansible continue to build out strategies needed for enterprise class deployment and configuration management, for example error handling, rollback and reporting in chapter 4.  As chapter, the amount of new Ansible capabilities being introduced is not as substantive as prior chapters, and emphasis is more only what could be described as best practise. For example creating Playbooks that have the means to be invoked to re-establish a prior state if the the execution of the current playbook was to throw up an error.  The callback explanation does need a bit more understanding of how Python works as implementing a callback involves a little bit of Python coding, but the points into which you can hook actions is very rich.

From knowing how to trap callbacks it becomes possible to initiate notifications when events occur in playbooks which is where this chapter moves onto with monitoring and alerting. This really focuses on has my playbook executed as expected and reporting back through means such as email, nagios and graphite.  The examples with email and nagios miss a trick, although the text says you can incorporate output from tasks – it isn’t illustrated; yet if something falters you’d want to see the task output.

Chapter 5 goes into how you might write your own custom modules and test them. Ansible will support any language that is available in your target environment, although Python is the recommended language given its general availability and is the language used to write Ansible, and Ansible modules can be leveraged to shorten the effort in creating custom modules. The chapter then walks through examples using Python, Bash shell scripting and Ruby. A lot of the work appears to be centred on extracting the appropriate parameters to allow the module to run with. The final part of the chapter looks at testing with the Python Nose library.

Solid chapters, and perhaps a little shorter than the first few, but importantly continuing to be well written although perhaps a couple of small missed opportunities to be great chapters.

Prior chapter reviews:

Learning Ansible Review Part 2

18 Wednesday Feb 2015

Posted by in Book Reviews, Books, General, Packt, Technology

≈ 2 Comments

Tags

, , , , , , ,

Example Ansible playbook

So having had a significant introduction to Ansible and its capabilities, chapter 2 gets into developing Ansible Playbooks. To do this the book confronts development practises of environment creation and management scripts, or the tendency for ops teams not to apply development like rigour. It is refreshing to read things like ….

… seen these practices at close quarters, we firmly believe that these are more or less similar to voodoo practices that need to be done away with.

Excerpt From: “Learning Ansible.” Packt Publishing.

 

The book does there for dig into the basics of using Git, and illustrating it using the configuration files from an earlier example. The next step would be if you follow development practises to use a Continuous Integration (CI). The book is surprisingly brief on this subject, compared to say Git.

In a development lifecycle the next step is to test, which elegantly introduces Ansible and Vagrant for instantiating virtual machines that the Ansible playbook can be tested against.

This creates the environment by which several additional capabilities provided by Ansible can aide testing activities particularly the ability to tag activities and then run the Ansible script targeting the specifically tagged activities. The additional ability that allows Ansible Playbooks to be executed in a way that allows it only tell what would be changed, rather than perform the change.

The book takes you on from there to introducing how Serverspec could be used. Things for me gain a little too much velocity, perhaps it doesn’t help that I am not familiar with Ruby.

The final part of the chapter and we’re back on solid ground with options on configuring SCM solutions such as Git to support deployment and how Ansible can support the same playbook in different environments such production, preproduction where the playbook is the same but you will be working with different server ids and credentials.

Even if you weren’t to use Ansible there are some thought provoking and good principles for config management and system scripting here.

So by having completed the first two chapters all the principle of Ansible are covered along a range of guiding practises that reflect good development practises in the context of producing environment management. There are limits and the next chapters then go into how to build upon the basics for so it is easy to create  more advanced Playbooks for real world environments with capabilities like playbook iteration, including Playbooks within Playbooks and conditionals for example.

The chapter builds upon the simple examples used on the first chapters showing how the Playbooks can be made to be a lot smarter. For example rather than a task to install each individual application required you can build a table of configuration values and get the task to iterate through the set of RPMs and versions for example. The structures to iterate over can be multi dimensional so you can define some advanced configuration and keep the tasks simple. By building on the earlier examples it helps highlight the benefits of the feature being explained which really helps.

Other features explained here include handlers, so you can trigger one or more activities to be performed once the playbook tasks have completed – so restarting processes can be executed only when all tasks are completed for example.  Other development like features like including other Playbooks are introduced (inline with the idea of DRY – Don’t Repeat Yourself). This leads into using roles, where than identifying your target servers by name or IP you can bring everything together by assigning the server roles e.g. DBTier, AppTier etc. The final two elements are the approach to templating using the Python Jinga2 framework and most crucially given that you will be handling configuration data and passwords Security.

Another couple of well written chapters that embody both insight into Ansible but also hung on the good development ideas so even if you choose not to use Ansible, some of the thinking here could be applied or at-least used to formulate questions as to how the ideas might translate to chef or puppet for example.

Learning Ansible Review Part 1

05 Thursday Feb 2015

Posted by in Book Reviews, Books, Packt, Technology

≈ 2 Comments

Tags

, , , , , , ,

I was going to publish the first part of my review of the Packt book Learning Ansible after every couple of chapters, but the preface and 1st chapter are pretty substantive not mention starts to address some of my questions regarding Ansible compared to Puppet and Chef.

So the first thing that is sticking is that the author’s style is a very easy to read flowing style. It means that your entire focus is on the content, and you don’t feel like you’re being lectured.

The preface is carries a lot of really valuable content setting the context and capabilities into which Ansible is working. It openly identifies the other leading products covering the different aspects in which someone interested in the space would want to know about or may already understand. So the following areas are called out:

  • Configuration Management
  • Provisioning
  • Deployment
  • Orchestration
  • Monitoring and Alerting
  • Logging

Chapter 1 pretty much faces into the question of differentiating Ansible from Chef and Puppet. The key being that Ansible is not a master and distributed client model which Puppet and Chef offer (although both offer a Masterless models which just distribute the client). What is missing from this initial conversation is that consideration of security. In a master and client model security can be stronger because even if you compromise the master, you still have an intermediary in the agent to protect against malicious actions. Where as breaching the Ansible node means you will have obtained SSH access to all the nodes available for management as this is how Ansible interacts with the nodes it is managing.

Despite this, the arguments are simply laid out, particularly the other significant difference is that a lot of the client-agent approaches mean they offer an abstraction of the types of activities you might want for example install an application which are abstracted from the likes of msi, yum, Red Hat Network and so on. The argument for not having this is that to provide abstraction you potentially end up dropping to the lowest common denominator (unless the tool implements capabilities not naturally available on the platform). A very fair and valid argument. It is also likely in an enterprise environment you’re probably using a small set of different types of environment and potentially only type type of environment for different solutions I.e. Your website is unlikely to be hosted by servers running Red Hat, Debian, Ubuntu and Fedora.

The first chapter takes you through a series of simple examples derived from a classic variant of IT’s classic Hello World solution. This does make for a sizeable first chapter (about 1/5th of the book) but does introduce all the core principles, ideas and capabilities that are embodied and provided by Ansible. If you want to know whether Ansible is likely to meet what you want to do then reading just this chapter will probably give you a view on whether you’re likely to be able to do what you want.

Although the book makes references to the support of Windows, but this is still in Alpha phase (Ansible Windows is Coming).This does mean that the examples are Linux only. Additionally it appears from the ansible site (the book doesn’t provide any indication of this) the central Ansible node(s) will still have to be Linux. The book in this case appears to trying to future proof itself. We hope that the author will provide Windows equivalent downloadable demos.

The long and short of it, a great start to a book.

Another book review on the way

27 Tuesday Jan 2015

Posted by in Books, Packt

≈ Leave a comment

Tags

, , , , , ,

Packt Publishing have given me the opportunity to review their book on Learning Ansible – an IT environment/apps/deployment automation framework. Should be interesting to see how Ansible compares to the big names of Chef and Puppet.  Flicking through the book, there appear to be plenty of examples and illustrations using common open source tools such as MySQL and Apache.

We’ll blog as we go, so should have a post every few chapters, so watch this space.

Things are going to be busy as I am also technically reviewing another Camel Book for Packt as well.