Tag Archives: API

My next Packt Project has been announced!

17 Saturday Nov 2018

Posted by in APIs & microservices, Packt, Technology

≈ Leave a comment

Tags

, , , , ,

My next Packt project (via O’Reilly) is not a book, but a short online training course about  good API design, API 1st and some tools that can support an API 1st methodologies. Register for the session here.

It includes a closer look at cloud tools such as Oracle’s excellent Apiary (sorry if this sounds like a sales pitch, but it is a good tool and the words of the found of RestLet confirm this) along with SwaggerHub and a few other options.

A good API goes beyond just the payload definition and I’ll walk through the other considerations and explain why these other areas are important.

Defining Boundaries for Logical Gateways on the API Platform a multi cloud / multi region context

31 Wednesday Oct 2018

Posted by in API Platform CS, General, Oracle, Technology

≈ Leave a comment

Tags

, , , ,

The Oracle API Platform takes a different licensing model to many platforms, rather than on CPU it works by the use of Logical Gateways and blocks of 25 million successful API calls per month. This means you can have as many actual gateway nodes as you like within a logical group to ensure resilience as you like, essentially how widely you deploy the gateways is more of a maintenance consideration (i.e. more nodes means more gateways to take through a maintenance process from the OS through to the gateway itself).

In our book (here) we described the use of logical gateways (groups of gateway nodes operating together) based on the classic development model, which provides a solid foundation and can leverage the gateway based routing policy very effectively.

logical partitions

But, things get a little trickier if you move into the cloud and elect to distribute the back end services geographically rather than perhaps have a single global instance for the back-end implementation and leverage technologies such as Content Delivery Networks to cache data at the cloud edge and their rapid routing capabilities to offset performance factors.

map1

Classic Global split of geographies

Some of the typical reasons for geographically distributing solutions are …

  • Low hit rate on data meaning caching solutions like CDNs are unlikely to yield performance benefits wanted and considerable additional work is needed to ‘warm’ the cache,
  • Different regions require different back end implementations ordering of products in one part of the world maybe fulfilled using a partner, but in another it is directly satisfied,
  • Data is subject to residency/sovereignty rules – consider China for example. But Germany and India also have special considerations as well.

So our Global splits start to look like:

map2

Global Split now adding extra divisions for India, China, Russia etc

The challenge that comes, is that the regional routing which maybe resolved on the Internet side of things through Geo Routing such as the facilities provided by AWS Route53 and Oracle’s Dyn DNS as a result finding nearest local gateway. However Geo DNS may not be achievable internally (certainly not for AWS), as a result routing to the nearest local back-end needs to be handled by the gateway. Gateway based routing can solve the problem based on logical gateways – so if we logically group gateways regionally then that works. But, this then conflicts with the use of gateway based routing for separation of Development, Test etc.

Routing Options

So, what are the options? Here are a few …

  • Make you Logical divisions both by environment and by region – this is fine if you’re processing very high volumes i.e. hundreds of millions or more so the cost of additional Logical gateways is relatively small it the total budget.

map3

Taking the geo split and applying the traditional layers as well has increased the number of Logical gateways

This problem can be further exacerbated, if you consider many larger organisations are likely to end up with different cloud vendors in the same part of the world for example AWS and Azure, or Oracle and Google. So continuing the segmentation can become an expensive challenge as the following view helps show:

map4

It is possible contract things slightly by only have development and test cloud services where ever your core development center is based. Note that in the previous and next diagrams we’ve removed the region/country specific gateway drivers.

map5

  • Don’t segment based on environment, but only on region – but then how do you control changes in the API configuration so they don’t propagate immediately into production?
  • Keep the existing model but clone APIs for each region – certainly the tooling we’ve shared (Managing API Policy Versioning in Oracle API Platform) makes this possible, but it’s pretty inelegant and error prone as it be easy to forget to clone a change, and the cloning logic needs to be extended to take into account the bits that must be region specific.
  • Assuming you have a DNS address for the target, you could effectively rewrite the resolution of the address by changing its meaning in each gateway node’s host file. Inelegant, but effective if you have automated deployment and configuration of your gateway servers.
  • Header based routing with the region and environment as header attributes. This does require either the client to set the values (not good as you’re revealing to your API consumer traits of the implementation), or you apply custom policies before the header based routing that insert those attributes based on the gateway’s location etc.
  • Build a new type of gateway based routing which allows both the environment (dev, test etc) and location (region) to inform the routing,

Or, and the point of this blog, use gateway based routing and leverage some intelligent DNS naming and how the API Platform works with a little bit of Groovy or a custom Java policy.

Continue reading →

Analytics and Stats for APIs

05 Friday Oct 2018

Posted by in API Platform CS, General, Oracle, Technology

≈ Leave a comment

Tags

, , , , , , , ,

The Oracle API Platform provides the means to examine statistics and slice and dice the numbers by application, gateway, duration and so on resulting in visually appealing graphical representations.  The way the analytics works means you can book mark specific views, so you can return the same report view with the relevant features as often as you like.  However, presently there is no data export option.

The question why would I want to export the information comes down to several possible use cases, all of which relate to cost management.  The API Platform will eventually have all the desired data views, but now something to help address the following:

  • money-tization, we can see which consumer has been using the services by how much and then send the data to a companies accounting systems to invoice the users
  • Ability to examine demand and workload over time to create a projection of the likely infrastructure – to achieve this the API statistics need to be overlaid with infrastructure and performance details so we can extrapolate API growth against server workload.

To address these kinds of requirements, we have taken advantage of the fact the API Platform has drunk its own Champagne as they say and made many of the analytics querying APIs publicly available.  As with the other API Platform tools, the logic has been written in Groovy, and freely available for use – we’ve covered the code through a Create Common license.

Tool includes a range of parameters to allow the data retrieved into a CSV file having filtered in a number of different ways – which logical gateways to examine, which API or Application(s) to report on.  Finally, just to help some basic stats are produced with a count of logical gateways, API calls, APIs defined and Application definitions. The first three factors inform your cloud costs. Together the stats can help Oracle understand your use case. Note that the parameters which impact the CSV generation can also materially impact the reporting numbers.

Parameters:

The 1st three values must always be provided and in the order shown here

  1. user name to access the source management cloud
  2. password for the source management cloud
  3. The server address without any attributes e.g. https://1.2.3.4

All the following values are optional

  • -h or -help – provides this information
  • -g – Logical gateway to retrieve numbers from e.g. production or development. using ALL with this parameter will result in ALL gateways being examined
  • -f – the file to target the CSV data should be written to. If not set then the default of
  • -t – indicates whether the data provided should be taken from an APPS perspective or from an API view by passing either APPS | API
  • -d – will get script to report more information about what is happening
  • -p – reporting period which is defined by a number as follows:
    • 0 – Last 365 days – data is given as per month
    • 1 – Last 30 days – this is the default if no information is provided – data is given as per day
    • 2 – Last 7 days – data is given as per day
    • 3 – Last day – data is given as per hour

NB – still testing the utility at this moment – will remove this comment once happy

Oracle Developer Meetup London – September 2018

18 Tuesday Sep 2018

Posted by in APIs & microservices, Dev Meetup, General, Technology

≈ Leave a comment

Tags

, , , , , , , , , ,

#OracleDevMeetup in London - GraphQL

Last night we ran the latest of the Oracle Developer Meetups in London. This time Luis Weir presented on GraphQL, which got an very engaged discussion about the strengths and weaknesses of GraphQL, in-depth points about how the error paths should be handled among many other things.

The presentation material Luis used is based upon his Devoxx session earlier this year and can be seen here:

The links to Luis’ examples can be found on his GitHub account – https://github.com/luisw19/graphql-samples

After a insightful and thought provoking presentation on GraphQL the Drones with APIs project had its latest update.  Providing a lot of laughter to the evening’s proceedings. Including demonstration of flying the drone using REST APIs published via a gateway and Go back-end.  This included the DroneDash presenting a visual presentation of the commands being issues via REST, as seen here:

All the code, API definitions and documentation for people to add or extend can be found in the meetup’s GITHub – https://github.com/oracledeveloperslondon/.

A few of the useful links used or mentioned last night are:

 

The next meetup is planned for Monday November 19th.  Topics  will be published soon.

 

#OracleDevMeetup in London - GraphQL

Implementing Oracle API Platform Cloud Service

31 Thursday May 2018

Posted by in APIs & microservices, Books, General, Packt, Technology

≈ Leave a comment

Tags

, , , , , , , , ,

After months of labour, the arrival of new family members for a couple of the authors the Implementing Oracle API Platform Cloud Service book as finally been published. The book has been included into Packt’s Expert series so, earns(?) the privilege of having photos of the authors on the cover.  The book can be purchased directly from Packt (go here) or from book retails such as Amazon (here).

 

B08683_front-Coverx600

It has been an interesting experience. Whilst working as part of a team of four authors lightened the writing load, a lot more energy went into communication so things were lined up. If you want a challenge, why not read the book and try to work out who wrote which chapters!

Continue reading →

Documenting APIs on the Oracle API Platform

21 Monday May 2018

Posted by in API Platform CS, General, Oracle, Technology

≈ 1 Comment

Tags

, , , , ,

The last week or two I have been working on a new API Platform utility to add to my existing tools (see here). This tool addresses the question of generating documentation.  Much as been said about API documentation and the quality of it, check out these articles :

If you look at these articles and others, there are some common themes, which are:

  • Document the URI / payload
  • Describe error handling
  • Describe contracts such as how many API calls
  • How the API is authenticated

Apiary covers the first theme to a first class standard,  and you will see Apiary called out for its ability to document APIs in a lot of articles. Well written API Blueprints will cover the bulk of the second bullet. But the other points tend to fall outside of a Blueprint and fit more the API Policies and their use.

Not everyone is so commited or enjoys writing documentation. The other driver for going beyond the use of Apiary is that some organizations feel the need to have a traditional word style document to capture/define an API’s contract in detail. With the API Platform the management portal enables an API to be published into the developer portal with the Apiary definition and a markdown file for further documentation.

Continue reading →

Tracing Executions in an API Environment

07 Wednesday Feb 2018

Posted by in API Platform CS, APIs & microservices, General, Technology

≈ 1 Comment

Tags

, , , , , , ,

As APIs become more pervasive within our solutions we see the arrival of not just design and cataloging tools such as Apiary, Apigee and others but also the arrival of gateways. The gateways provide execution of operations including validation, accounting (moneytization), routing, and other controls such as throttled checks that would often not occur until the first contact with a service bus. For example initial routing based on the API call, fine grained authentication and authorisation (differing from your firewalls who will just perhaps authorise).

In the more traditional integration middleware of Oracle Service Bus and SOA (regardless of cloud, on-premises) you can trace the execution through the middleware end to end. This tracing can be achieved because the platform creates and assigns the a UUID (aka eCID) and ensures that it is carried through the middleware. It is this very behaviour that allows Oracle to provide Business Activity Monitoring without any need to be invasive. Burt not only that in a highly distyributed environment you can track the processing of a transaction from end to end.

The challenge now is that the first point of middleware style behaviour can be at the gateway. So we actually need to move forward the UUID or equivalent forward the the first point of contact. Not only that we need to think about the fact we will see non Oracle integration middleware involved.  Within Spring, Kabana and other established frameworks and tools which are getting signficiant traction with the rise of Microservices, the Ids being used are not the same as the UUIDs used by Oracle. For example Spring Cloud Slueth uses the same HTTP header Ids that Zipkin and Kabana support:

  • X-B3-TraceId
  • X-B3-SpanId
  • X-B3-ParentSpanId

More information can be found here and here.

For the new Oracle API Platform Cloud Service we can check for the existance of the header attributes as a policy and apply actions such as:

  • Apply a header with a TraceId or spanId,
  • If a SpanId exists then we may wish to nest our call as a child span by moving the SpandId to the ParentSpanId and creating a new SpanId.

Ultimately it would be more attractive to apply the logic using the API Platform’s SDK but to get things rolling applying the IDs with the API Groovy policy is sufficient (more here).

Next question that begs, is where to put the ID and want to call it? Put the value in the body, and you’re invading the business aspect of an API with execution specific details, not to mention potentially  changing the API definition. Whilst stuffing HTTP(s) headers with custom attributes is often discouraged as the values aren’t to immediately visible. In my opinion at least the answer largely has been set by president for us. If you weren’t using HTTPS but JMS then you would use the header, but also a number of frameworks already exist that do make use of this strategy such as those mentioned above.

Using the header defined by Kabana etc means that the very process will mean that a number of Support tools out of the box will understand and be able to help visualise your logs with no additional effort.

The following is a Groovy script that could ber used for the purposes of applying an Id appropriately into the HTTP header in the API Platform:

if (!context.getApiRequest().getHeaders().containsKey ("baggage-UUID "))
{
  // use current time to seed random generator
  def now = java.time.Instant.now()
  def random = new java.util.Random(now.getEpochSecond())

  // create an array of 16 bytes to hold the random value
  byte[] uid = new byte[16]
  random.nextBytes(uid)

  // convert the random string from Hex to an ASCII string
  Writable uidInHex= uid.encodeHex()
  String uidStr= uidInHex.toString() 

  // set the outbound header
  context.getServiceRequest().setHeader("baggage-UUID", uidStr)
}

API Design

05 Monday Feb 2018

Posted by in API Platform CS, APIs & microservices, Books, General, mindmap

≈ Leave a comment

Tags

, , , ,

When it comes to ensuring I keep up good practises, I try to look at books  in areas I think I have a good handle on such as APIs.  Why?  well it confirms and validates I’m upto date; sometimes another view point can spark ideas on how to make something better, improve an approach or simply understand another way of explaining an idea.  The later is important as the key benefit of knowing something is the opportunity to help someone else. Not everyone communicates or understands ideas in the same so this is always helpful.

Designing Great Web APIsSo recently I ran through James Higginbotham’s Designing Great Web API’s book(let).  Often when goping through a book I mindmap it so that I can share it, and refer to it as a lit of prompts reminders if necessary.  Whilst’s James’ book doesnt  reveal anything new or relevatory for anyone working with APIs it does provide a good succinct explination to  basic practises. So here is the mindmap:

great APIs

You can also access my MapWise view here. James’ book can be obtained freely from O’Reilly here.

The book doesn’t go into the depth of details for practises that Apiary (Pro Edition) offers with style guidelines which will describe morec detailed recommended practises (more here).