Latest on book and APIs

15 Tuesday Dec 2020

Posted by mp3monster in Books, Fluentd, General, manning, Technology

Tags

AdeventuresInDevOps, book, development, Fluentd, manning, MEAP, podcast, Redis, unified logging

My blogging is way down compared with only a post about OKit – OCI Design (on Windows). It largely comes down to lots of work on our Fluentd book. Chapter 6 is now available in the MEAP. As the promo info says …

What’s new?
Chapter 6, “Filtering and Extrapolation”
Gain control and insight!
Last chapter, we touched on the use of the Filter directive. But that was just the tip of the iceberg! In Chapter 6, we’ll plunge below the surface, exploring the when, why, and how of applying filters to give us more insight and precise control over events.
Promo Email from Manning

Earlier chapters have been tweaked, with some additional improvements which will make the live reading experience better.

Another chapter and an appendix should be finding their way to MEAP very soon as it was handed over by our project editor. That will make it seven chapters available, and all the appendices.

Whilst the peer review is taking place the chapter covering plugin development is progressing. The development work has got the basics of the output plugin with log events being stored in Redis and the input being worked on as well. If you want a peak, keep an eye on my GitHub repository (here).

But is isn’t all writing…

I presented on Twitch – you can catch that at https://m.twitch.tv/videos/809295979 I’ve been offered the opportunity to present again, so keep an eye out for something next year.

We recorded a podcast with the excellent guys over at Adventures In DevOps. We don’t have the exact date for the podcast to be released, but I imagine it will sometime during Jan 2021. I’d recommend checking out the podcasts. I’ve been dipping into their back catalogue of recordings and the team ask some really thought provoking questions.

If that wasn’t enough, we’ve been fortunate enough to have some time to talk with leading members of the Fluentd and Fluent Bit projects which was a real pleasure. Hopefully, as we leave this horrendous year behind we’ll get to talk and possibly collaborate some more.

API Caching with the Oracle API Platform

01 Friday Feb 2019

Posted by mp3monster in API Platform CS, APIs & microservices, General, Oracle, Technology

≈ 4 Comments

Tags

API, API-CS, cache-control, caching, Coherence, ehcache, forceResponse, Helidon, IETF, Oracle, proxy, Redis, RGC

We have been developing some advanced custom API policies for a client and in the process picked up on a few insights that didn’t even make into the API book. One of these policies is to provide an optimization around caching of API calls. The rest of this blog will talk about the tricks we have specifically applied to link an API Gateway to a caching mechanism and why.

Before I go into the details, I’d like to thank the Oracle product management team and particularly Glenn Mi at Oracle for their support getting through the deeper undocumented elements of the capabilities of the API Platform SDK.

Caching Options

Caching comes in may forms, and is motivated by varying reasons and not always the wanting the same behaviours. When getting into the subject of caching it is surprising how polarised people’s view points can be about which cache strategies are correct. The following diagram illustrates the diversity of caches that could appear in an end to solution solution.

Bringing together a caching technology in the Reverse Proxy model and an API Gateway makes a lot of sense. Data being provided to API consumers needs to be protected whether it comes from a cache or an active back-end system. At the same time you also want to exploit an API Gateway to provide analytics on API traffic, so any caching needs to be behind the gateway. But, if In front of an application layer then we can reduce the application workload.

When it comes to caching technology to partner with the gateway, there are a number of options available from Coherence to ehCache, memcache and Redis. We have avoided Coherence, whilst the gateway currently runs on a WebLogic server, we don’t want to need to unduly distort the performance profile and configuration of the Gateway by forcing a cache onto that server. In addition to which as Coherence is a licensed addition to WebLogic it raises difficult questions about licensing when deploying gateways (with gateways licensed based on logical groupings and API volumes but Coherence is licensed by OCPU). We also know that Oracle is moving towards having a micro-gateway which may mean we see the gateway engine moved onto something like Helidon (but this last point is my speculation).

We have elected to use Redis for several reasons –

Available as a PaaS service with several cloud providers (AWS & Azure) so no setup or management effort but can also be deployed on-premises,
Has an out of the box deployment that means cached entities can have a time to live (TTL) rather than needing to implement separate processes to expire cached values,
The ability to make it scale through clustering,
Cost

This caching model also allows us to optionally allow application development teams to push into the cache directly results. So rather than waiting on TTL the cache can be refreshed directly or even primed, rather than having to create fake requests to prime a cache.