Tag Archives: reporting

Analytics and Stats for APIs

05 Friday Oct 2018

Posted by in API Platform CS, General, Oracle, Technology, tools

≈ 3 Comments

Tags

, , , , , , , ,

NOTE:This utility needs revamping to support IDCS for more see Making Scripts Work with IDCS Deployed PaaS

The Oracle API Platform provides the means to examine statistics and slice and dice the numbers by application, gateway, duration and so on resulting in visually appealing graphical representations.  The way the analytics works means you can book mark specific views, so you can return the same report view with the relevant features as often as you like.  However, presently there is no data export option.

The question why would I want to export the information comes down to several possible use cases, all of which relate to cost management.  The API Platform will eventually have all the desired data views, but now something to help address the following:

  • money-tization, we can see which consumer has been using the services by how much and then send the data to a companies accounting systems to invoice the users
  • Ability to examine demand and workload over time to create a projection of the likely infrastructure – to achieve this the API statistics need to be overlaid with infrastructure and performance details so we can extrapolate API growth against server workload.

To address these kinds of requirements, we have taken advantage of the fact the API Platform has drunk its own Champagne as they say and made many of the analytics querying APIs publicly available.  As with the other API Platform tools, the logic has been written in Groovy, and freely available for use – we’ve covered the code through a Create Common license.

Tool includes a range of parameters to allow the data retrieved into a CSV file having filtered in a number of different ways – which logical gateways to examine, which API or Application(s) to report on.  Finally, just to help some basic stats are produced with a count of logical gateways, API calls, APIs defined and Application definitions. The first three factors inform your cloud costs. Together the stats can help Oracle understand your use case. Note that the parameters which impact the CSV generation can also materially impact the reporting numbers.

Parameters:

The 1st three values must always be provided and in the order shown here

  1. user name to access the source management cloud
  2. password for the source management cloud
  3. The server address without any attributes e.g. https://1.2.3.4

All the following values are optional

  • -h or -help – provides this information
  • -g – Logical gateway to retrieve numbers from e.g. production or development. using ALL with this parameter will result in ALL gateways being examined
  • -f – the file to target the CSV data should be written to. If not set then the default of
  • -t – indicates whether the data provided should be taken from an APPS perspective or from an API view by passing either APPS | API
  • -d – will get script to report more information about what is happening
  • -p – reporting period which is defined by a number as follows:
    • 0 – Last 365 days – data is given as per month
    • 1 – Last 30 days – this is the default if no information is provided – data is given as per day
    • 2 – Last 7 days – data is given as per day
    • 3 – Last day – data is given as per hour

NB – still testing the utility at this moment – will remove this comment once happy

Learning Ansible Review Part 3

08 Sunday Mar 2015

Posted by in Book Reviews, Books, Packt, Technology

≈ 1 Comment

Tags

, , , , , , , , , ,

Chapters 4 & 5 of Packt’s Learning Ansible continue to build out strategies needed for enterprise class deployment and configuration management, for example error handling, rollback and reporting in chapter 4.  As chapter, the amount of new Ansible capabilities being introduced is not as substantive as prior chapters, and emphasis is more only what could be described as best practise. For example creating Playbooks that have the means to be invoked to re-establish a prior state if the the execution of the current playbook was to throw up an error.  The callback explanation does need a bit more understanding of how Python works as implementing a callback involves a little bit of Python coding, but the points into which you can hook actions is very rich.

From knowing how to trap callbacks it becomes possible to initiate notifications when events occur in playbooks which is where this chapter moves onto with monitoring and alerting. This really focuses on has my playbook executed as expected and reporting back through means such as email, nagios and graphite.  The examples with email and nagios miss a trick, although the text says you can incorporate output from tasks – it isn’t illustrated; yet if something falters you’d want to see the task output.

Chapter 5 goes into how you might write your own custom modules and test them. Ansible will support any language that is available in your target environment, although Python is the recommended language given its general availability and is the language used to write Ansible, and Ansible modules can be leveraged to shorten the effort in creating custom modules. The chapter then walks through examples using Python, Bash shell scripting and Ruby. A lot of the work appears to be centred on extracting the appropriate parameters to allow the module to run with. The final part of the chapter looks at testing with the Python Nose library.

Solid chapters, and perhaps a little shorter than the first few, but importantly continuing to be well written although perhaps a couple of small missed opportunities to be great chapters.

Prior chapter reviews:

Puppet Reporting & Monitoring Book Review

27 Friday Jun 2014

Posted by in Book Reviews, Books, General, Packt, Technology

≈ Leave a comment

Tags

, , , , , , , ,

So the Packt book (Puppet Reporting and Monitoring) focuses down on a couple of aspects of the Puppet Toolset, as a result this is a relatively short book with only a couple hundred pages. As an enterprise architect I am no expert Puppet practitioner, my knowledge of Ruby is high level (part of the reason I reviewed this book is I wanted to better understand the art of the possible in these areas).  But despite this the book does an exceptionally good job defining a context and then explaining and showing what could be done, down to code examples.  In doing so, the author Michael Duffy introduces a number of open source libraries that can be leveraged to provide dashboard views, presentation of report content whilst maximising the leveraging of the Puppet ecosystem such as the Puppet DB (an abstracted database with a REST + JSON API).  The book goes beyond just implementation of monitoring and reporting but also engages with considerations such as deployment.  without ‘boiling the ocean’ the book provides a very good illustrations of the art of the possible and provides plenty of references to source information so working how you want to implement you own solutions.

My only criticism of the book, and it is a minor one at that is a few more diagrams to help illustrate ideas (particularly in the first chapter when discussing deployment considerations) would help get ideas across easily.

On the strength of this  book,  I hope that Michael considers taking on other authoring projects as this has been one of the best written technical books I’ve read in sometime.

Puppet Reporting & Monitoring
Useful Links: