• Home
  • Site Aliases
    • www.cloud-native.info
  • About
    • Background
    • Presenting Activities
    • Internet Profile
      • LinkedIn
    • About
  • Books & Publications
    • Log Generator
    • Logs and Telemetry using Fluent Bit
      • Fluent Bit book
      • Book Resources in GitHub
      • Fluent Bit Classic to YAML Format configurations
    • Logging in Action with Fluentd, Kubernetes and More
      • Logging in Action with Fluentd – Book
      • Fluentd Book Resources
      • Fluentd & Fluent Bit Additional stuff
    • API & API Platform
      • API Useful Resources
    • Oracle Integration
      • Book Website
      • Useful Reading Sources
    • Publication Contributions
  • Resources
    • GitHub
    • Oracle Integration Site
    • Oracle Resources
    • Mindmaps Index
    • Useful Tech Resources
      • Fluentd & Fluent Bit Additional stuff
      • Recommended Tech Podcasts
      • Official Sources for Product Logos
      • Java and Graal Useful Links
      • Python Setup & related stuff
  • Music
    • Monster On Music
    • Music Listening
    • Music Reading

Phil (aka MP3Monster)'s Blog

~ from Technology to Music

Phil (aka MP3Monster)'s Blog

Category Archives: Oracle

All things related to Oracle

More Free Oracle Cloud than you might know

04 Monday Nov 2019

Posted by mp3monster in Cloud, General, Oracle, Technology

≈ Leave a comment

Tags

CloudEvents, CNCF, events, free, Functions, Notifications, Oracle

free-750x536The news about Oracle offering some free cloud services ‘for life’ is making an impact.  But, the free services don’t end there. The pricing of some other native cloud services includes some free bands. So it’s worth keeping an eye on the fine print. I wouldn’t be surprised if we see limited capacity access in other areas.

Oracle Functions – whilst the core of this service is built on the open-source Fn Project (also largely driven by Oracle) the managed service has a free tier allowing up to 2 million invocations that can consume 400, 000 gigabytes of memory per second use (details can be seen here). Plenty enough to experiment with the concepts behind Serverless aka FaaS capabilities.

Oracle Notifications whilst focussed on the technical side of gathering key event data from OCI and its services, as the document states “sending notifications to numerous interested parties, or even synchronizing the moving parts of a distributed application” – this obviously means a service with characteristics a bit like AWS’ SNS. Like SNS it can be hooked up to email and other HTTPS services using Oracle Events which also has free use. Events is particularly interesting as it is bases the event structure on the CNCF CloudEvents spec. There is an excellent illustration of such a use case in the Oracle blogs here.

It will be interesting to see if we a similar trend with other Oracle cloud-native services. A new take on the now-defunct Application Container Cloud Service (ACCS) would be an ideal vehicle – whether there is sufficient demand for such a capability is not clear (it would in effect be an always live service like a Kubernetes solution, but the simpler, smaller footprint more like Functions in a multi-tenant environment. At the same time, it doesn’t have potential latency of a Function being activated).

Share this:

  • Share on Facebook (Opens in new window) Facebook
  • Share on X (Opens in new window) X
  • Share on Pocket (Opens in new window) Pocket
  • Share on Reddit (Opens in new window) Reddit
  • Email a link to a friend (Opens in new window) Email
  • Share on WhatsApp (Opens in new window) WhatsApp
  • Print (Opens in new window) Print
  • Share on Tumblr (Opens in new window) Tumblr
  • Share on Mastodon (Opens in new window) Mastodon
  • Share on Pinterest (Opens in new window) Pinterest
  • More
  • Share on Bluesky (Opens in new window) Bluesky
  • Share on LinkedIn (Opens in new window) LinkedIn
Like Loading...

OGB Appreciation Day : Support of Hybrid

11 Friday Oct 2019

Posted by mp3monster in General, Oracle, Technology

≈ Leave a comment

Tags

blockchain, FaaS, Functions, Hybrid, Kubernetes, OGB, OIC, Oracle

This is my blog post as part of the Oracle Ground Breakers Appreciation Day (more about this with oracle-base) isn’t about a specific product or feature but an approach or possibly two approaches that exist with many of the PaaS services available from Oracle.

One of the key things that many of Oracle’s products such as Integration Cloud, API Platform and the foundation of Functions (Fn) and Containers is the recognition that many organisations are not so fortunate to be cloud-born, or even working with a cloud-native model for IT. For those organisations who would rather have across location unifying approach, Oracle cloud is not a closed capability like AWS, whilst products like Integration Cloud are at their best on Oracle Cloud Infrastructure, they can be executed in your data centre, or even another cloud.

Whilst the teams I work with experiment and build our service offerings ‘on Oracle’, when we engage with customers to help them with their specific problem spaces, we are more often than not operating in a multi-cloud or on-premises hybrid model.

This hybrid story is helped with a renewed vigour for open source both contributing to but also leading the development of open source. In addition to providing free tiers to some of their stack such as Functions, IaaS and Database (here). Many do forget the Oracle JVM is free as long as you keep up to date, you have got a small footprint Oracle database for free (XE), MySQL is part of the Oracle family. Then many of the modern development technologies are true to the core open-source, Blockchain, Container Engine meaning that the solutions on these layers are portable, can be run on-prem. Yes, Oracle adds value by wrapping these cores with tooling and features that make easier rather than diverging with proprietary Ingress controllers for example.

The irony is that organisations that tend to be associated with a low cost or being faithful to open source goals actually can end up locking you in and appear to be moving away from the original open-source ideals. Consider RedHat, the champion for a lot of open source-based enablement have removed Kubernetes from the official RedHat downloads for their Linux in-favour of a single node license of OpenShift, to get Kubernetes of RHEL you have to go outside of the normal binary source channels (other challenges are documented here).

Share this:

  • Share on Facebook (Opens in new window) Facebook
  • Share on X (Opens in new window) X
  • Share on Pocket (Opens in new window) Pocket
  • Share on Reddit (Opens in new window) Reddit
  • Email a link to a friend (Opens in new window) Email
  • Share on WhatsApp (Opens in new window) WhatsApp
  • Print (Opens in new window) Print
  • Share on Tumblr (Opens in new window) Tumblr
  • Share on Mastodon (Opens in new window) Mastodon
  • Share on Pinterest (Opens in new window) Pinterest
  • More
  • Share on Bluesky (Opens in new window) Bluesky
  • Share on LinkedIn (Opens in new window) LinkedIn
Like Loading...

Millennials in the Workforce – PTK

03 Thursday Oct 2019

Posted by mp3monster in General, OUG

≈ Leave a comment

Tags

article, Capgemini, interview, millenials, OUG, PTK, workforce

Those who know me will be aware that I try to support the UK Oracle User Group’s journal (#PTK) in a number of ways from submitting articles through to being part of the review panel.  I’ve mentioned in the past some of the changes that the journal has undergone (here for example).  But another change is that the editorial team are including more diverse content. For example in the latest issue just out. It includes an article about Millennials in the workforce and how things are changing. A theme that is confronting not only businesses as employers, but as the new generation of influencers and decision-makers and that will be making our enterprise buying decisions, and dare I say it,  members of a user group.

As part of the team who also informs the User Group’s event planning, I happened to throw in some thoughts about supporting and engaging the newer generation. That led to an invitation to participate in an interview which has contributed an interesting article on millennials in the workforce.

Putting the company man hat on for a moment, it was good to highlight the efforts that Capgemini make to support new talent into the organisation.

The article is here, and links to the Tech and App parts of #PTK journal are here.

PTK_ISSUE71_TECH_COVER_FINAL

Share this:

  • Share on Facebook (Opens in new window) Facebook
  • Share on X (Opens in new window) X
  • Share on Pocket (Opens in new window) Pocket
  • Share on Reddit (Opens in new window) Reddit
  • Email a link to a friend (Opens in new window) Email
  • Share on WhatsApp (Opens in new window) WhatsApp
  • Print (Opens in new window) Print
  • Share on Tumblr (Opens in new window) Tumblr
  • Share on Mastodon (Opens in new window) Mastodon
  • Share on Pinterest (Opens in new window) Pinterest
  • More
  • Share on Bluesky (Opens in new window) Bluesky
  • Share on LinkedIn (Opens in new window) LinkedIn
Like Loading...

Handling Socket connectivity with API Gateway

30 Friday Aug 2019

Posted by mp3monster in API Platform CS, General, Oracle, Technology

≈ Leave a comment

Tags

API, event streams, gateway, OAuth, Oracle, Security, socket

At the time of writing the Oracle API Platform doesn’t support the use of Socket connections for handling API data flows. Whilst the API Platform does provide an SDK as we’ve described in other blogs and our book it doesn’t allow the extension of how connectivity is managed.

The use of API Gateways and socket-based connectivity is something that can engender a fair bit of debate – on the one hand, when a client is handling a large volume of data, or expects data updates, but doesn’t want to poll or utilize webhooks then a socket strategy will make sense. Think of an app wanting to listen to a Kafka topic. Conversely, API gateways are meant to be relatively lightweight components and not intended to deal with a single call to result in massive latency as the back-end produces or waits to forward on data as this is very resource-intensive and inefficient. However, a socket-based data transmission should be subject to the same kinds of security controls, and home brewing security solutions from scratch are generally not the best idea as you become responsible for the continual re-verification of the code being secure and handling dependency patching and mitigating vulnerabilities in other areas.

So how can we solve this?

As a general rule of thumb, web sockets are our least preferred way of driving connectivity, aside from the resource demand, it is a fairly fragile approach as connections are subject to the vagaries of network connections, which can drop etc. It can be difficult to manage state (i.e. knowing what data has or hasn’t reached the socket consumer). But sometimes, it just is the right answer. Therefore we have developed the following pattern as the following diagram illustrates.

API Protected Sockets

How it works …

The client initiates things by contacting the gateway to request a socket, with the details of the data wanted to flow through the socket. This can then be validated as both a legitimate request or (API Tokens, OAuth etc) and that the requester can have the data wanted via analyzing the request metadata.

The gateway works in conjunction with a service component and will if approved acquire a URI from the socket manager component. This component will provide a URL for the client to use for the socket request. The URL is a randomly generated string. This means that port scans of the exposed web service are going to be difficult. These URLs are handled in a cache, which ideally has a TTL (Time To Live). By using Something like Redis with its native TTL capabilities means that we can expire the URL if not used.

With the provided URL we could further harden the security by associating with it a second token.

Having received the response by the client, it can then establish the socket-based connection which gets routed around the API Gateway to the Socket component. This then takes the randomly-generated part of the URL and looks up the value in the cache, if it exists in the cache and the secondary token matches then the request for the socket is legitimate. With the socket connection having been accepted the logic that will feed the socket can commence execution.

If the request is some form of malicious intent such as a scan, probe or brute force attempt to call the URL then the attempt should fail because …

  • If the socket URL has never existed in or has been expired from the Cache and the request is rejected.
  • If a genuine URL is obtained, then the secondary key must correctly verify. If incorrect again the request is rejected.
  • Ironically, any malicious attack seeking to overload components is most likely to affect the cache and if this fails, then a brute access tempt gets harder as the persistence of all keys will be lost i.e. nothing to try brute force locate.

You could of course craft in more security checks such as IP whitelisting etc, but every-time this is done the socket service gets ever more complex, and we take on more of the capabilities expected from the API Gateway and aside from deploying a cache, we’ve not built much more than a simple service that creates some random strings and caches them, combined with a cache query and a comparison. All the hard security work is delegated to the gateway during the handshake request.

Thanks to James Neate and Adrian Lowe for kicking around the requirement and arriving at this approach with us.

 

Share this:

  • Share on Facebook (Opens in new window) Facebook
  • Share on X (Opens in new window) X
  • Share on Pocket (Opens in new window) Pocket
  • Share on Reddit (Opens in new window) Reddit
  • Email a link to a friend (Opens in new window) Email
  • Share on WhatsApp (Opens in new window) WhatsApp
  • Print (Opens in new window) Print
  • Share on Tumblr (Opens in new window) Tumblr
  • Share on Mastodon (Opens in new window) Mastodon
  • Share on Pinterest (Opens in new window) Pinterest
  • More
  • Share on Bluesky (Opens in new window) Bluesky
  • Share on LinkedIn (Opens in new window) LinkedIn
Like Loading...

Costs in Multi-Cloud

28 Wednesday Aug 2019

Posted by mp3monster in General, Oracle, Technology

≈ Leave a comment

Tags

AWS, Azure, Cloud, costs, data, ExpressRoute, FastComnect, Oracle, OracleCloud

Over the last couple of years, we have seen growing references to multi-cloud. That is to say, people are recognizing that organisations, particularly larger ones are ending up with cloud services for many different vendors. This at least in part has come from where departments within an organization can buy meaningful resources within their local budgets.

Whilst there is a competitive benefit of the recent partnership agreement between Microsoft and Oracle given the market margin AWS has in comparison to everyone else. Irrespective of the positioning with AWS, this agreement has arisen because of the adoption of multi-cloud. It also provides a solution to the problem of running highly resilient Oracle database setups using RAC, DataGuard etc can be made available to Azure without risk to security and the all-important network performances that are essentially to DB operation. Likewise, Oracle’s SaaS offerings are sector leaders if not best in place, something Microsoft can’t compete with. But at the other end, regardless of Oracle’s offerings, often organisations will prefer Microsoft development ecosystem because of the alignment to office tooling, the ease of building solutions quickly.

Multi-cloud even with the agreements like the Microsoft and Oracle one (See here), doesn’t mean there won’t be higher costs in crossing clouds. Let’s see where the costs reside …

  • Data egress (and in some cases ingress as well) from clouds costs. Whilst the ingress costs have been eliminated because it can be seen as a barrier to selling services, particularly big data. Data egress can, however, be an issue. Oracle has made this cost very low to be almost negligible, but not necessarily others as the following comparison shows …

  • Establishing the high-performance connections That the agreement supports needed between Azure and Oracle cloud is the same tech for the cloud to the ground do incur a cost. In Oracle’s case, there is a fee for the connection (not a large cost, but one that exists none the less) plus any traffic fees the provider of the network connection spanning the data centre locations. This is because you’re leasing capacity on someone’s dedicated fibre or MPLS services. Now, this should prove to be small as part of the enabler of this offering is that both Oracle and Microsoft cloud DCs are often actually physically provided by the same provider or at-least the centres a physically pretty close, as a result of both companies gravitating to locations close together because of the optimal highly available infrastructure (power, telecommunications) legal and commercial factors along with the specialist skills needed.

If data egress is the key challenge to costs, what drives the data egress beyond the obvious content for user interfaces? …

  • Obviously, you have the business data flows, some of these flows will be understood by the business community. But not all, this is down to the way data from the cloud can be exposed to another. For example inefficient services with APIs that require frequent polling and not using expressing the request efficiently, rather than perhaps express the request using HTTP header attributes and other efficiencies or even utilize frameworks such as webhooks so data can be pushed.
  • High-speed data access, often drives data replication having databases in multiple clouds with mirror image data in each location even if the majority of the data is not necessarily needed. This can happen with technologies such as Kafka which for non compacted topics will mean every event can be replicated even if that event has a short lifetime.
  • One of the hidden costs is the operational tasks of gathering logs to a combined view so end to end insights can be obtained. A detailed log can actually yield more ‘data’ by volume than the business flows themselves because it is semi-structured, and intended to be very readable and at the most granular level there to help debug and test.

In addition to the data flows, you need to consider how other linkages in addition to the Oracle-Azure connection are involved. In the detailed documentation, it is not possible to get your on-premises location connected to one of the clouds (e.g. Oracle FastConnect, and then assume your traffic can hop to Azure via the bridge using FastConnect and Azure’s ExpressRoute.  To add performance to your solution parts in both Azure and Oracle Cloud, you still need to have FastConnect and ExpressRoute configured to your on-premises location. This, of course, may impact how bulk data for lift and shift app use cases such as EBS may be applied. For example, if you choose to regularly bulk data transfer between on-premise and EBS via the app/middleware tier rather than via direct DB, and that mid-tier is running in Azure – you will need both routes established.

Conclusion

There is no doubt that the Oracle-Azure cloud to cloud linkage is a step forward, but ‘the devil is in the details‘ as the saying goes. To optimize the benefits and savings we’d suggest that you;

  • you’ll need to think through your use cases – understand data flow and volume (someone bulk syncing application data with a data warehouse?),
  • define a cloud data strategy – to layout principles, approaches and identify compliance needs, this is particularly helpful for custom solution development, so the right level of log data is consolidated with the important details, data retention addresses compliance requirements and doesn’t ratchet up unnecessary costs (there is a tendency to horde data just in case – if this is really wanted, think about how its stored),
  • based on business common usage models define a simple forecasting formula – being able to quantify data costs will always make it easier to challenge back data hoarding tendency,
  • confirm the inter-cloud network vendor charges when working with multi-cloud.

Share this:

  • Share on Facebook (Opens in new window) Facebook
  • Share on X (Opens in new window) X
  • Share on Pocket (Opens in new window) Pocket
  • Share on Reddit (Opens in new window) Reddit
  • Email a link to a friend (Opens in new window) Email
  • Share on WhatsApp (Opens in new window) WhatsApp
  • Print (Opens in new window) Print
  • Share on Tumblr (Opens in new window) Tumblr
  • Share on Mastodon (Opens in new window) Mastodon
  • Share on Pinterest (Opens in new window) Pinterest
  • More
  • Share on Bluesky (Opens in new window) Bluesky
  • Share on LinkedIn (Opens in new window) LinkedIn
Like Loading...

Why do I have an Ace logo on my website?

19 Monday Aug 2019

Posted by mp3monster in General, Oracle

≈ Leave a comment

Tags

Ace, ACED, Ground Breaker, MVP, Oracle

For the observant, you’ll have noticed that I have a logo on the left side of my site saying Oracle Ace. Periodically I get asked what is it, what does it mean, and for those who are less involved in the Oracle community probably don’t know what it means.

What

Most developers will probably have encountered the idea of Java Champions or perhaps Microsoft MVPs (Most Valued Professional). All of these badges and other large vendors such as SAP have comparable ones are a recognition of individuals outside of the organisation (in this case Oracle) who do a lot to support the community and wider technology ecosystem.

These contributions vary but typically take the form activities such as writing blogs/articles/books, answering questions on StackOverflow and other community sites where questions are raised and answered by experts. Organising and/or presenting at conferences.

This is the content helps bridge the gap between the standard guidance, documentation, white papers that the vendors will produce and real-world practical experience.

Whilst you, in theory, you don’t have to be an expert to be part of these advocacy programmes, the reality is to communicate the meaningful value you need to have a level of experience and understanding that is more than the majority. I know a number of people in the Ace community who would deny being experts, and the only thing that differentiates them from everyone else is being willing to stand up and share what they have learnt. I would say that inevitably they are experts, as the processes and resource (at least for Aces inevitably enable that development of expertise as I will try to illustrate shortly).

But before we progress, let me quickly summarise the advocacy communities that Oracle support …

  • 9928f94d6fb7bc0024781fa68e0bc571_400x400Java Champions – these are people working in the pure Java ecosystem
  • Oracle Ace’s – within this community we have three tiers of Ace and which tier reflects the amount of time actively involved in the Ace programme and how much you contribute. So you start out as …
    • Ace Associate progressing to
    • ace-logoAce then a at the top are a smaller community of
    • Ace Directors

Ace’s generally focus on Oracle’s mainstream products from the database to Middleware like WebLogic and Apps

  • obga_badgeThe final group are Groundbreaker Ambassadors – this group are comparable to Ace Directors and actually progress through the Associate and Ace accreditation. But rather than focus on more traditional Oracle offerings this group tend to work with what could be described as modern app dev tech from Microservices and APIs to Blockchain.

Why?

Why get involved in such a community? Whilst I can only speak for myself, I suspect some of my motivators hold true for others, for me, it’s about…

  • The is a strong sense of community amongst the Aces and obviously an inbuilt common interest. Given we often encounter each other at conferences etc, it makes it a lot easier socially when attending conferences. Stuck for a coffee conversation? Go say hello to someone you already know.
  • The value in knowledge and experience is in the sharing of that information, and you can’t beat the sense of validation when someone says – thank you, that really helped me.
  • Talking to other Aces means you may pick up useful insights. Certainly, the Ace community are encouraged to develop relationships with Oracle product management (to be nominated for Ace Director / Groundbreaker Ambassador you need the sponsorship of a product manager).
  • These insights will further your knowledge which makes the day job easier. It becomes easier to influence Oracle when it comes to having features or priorities set that are of interest.
  • Some employers and customers put a value on the Ace recognition as
    • there is the implicit expertise
    • gives indirect channels to product management
    • track record of sharing and enabling others
  • …so it creates some extra career opportunities or a foot up. If you look at eProseed’s website you’ll see that they are very proud to employ a lot of Ace Directors.

Expertise

Coming back to the point of expertise, as you develop within the community, the chances of learning from others increases, but developing relationships with product management means getting to hear about what’s next etc as well as getting to hear the product managers and their thinking. In fact, Ace Directors and Groundbreakers have dedicated briefing sessions and additional access that provides further insight into the product, strategy and direction. These relationships can start to create a virtuous circle of knowledge accumulation.

Biased?

Carrying the badge of a vendor, and obviously contributing to a vendor’s community carries the risk of being perceived as not being independent/impartial or perhaps understanding the wider landscape. But having been part of the community, this is deeply inaccurate. The community members I know take pride in being professional which usually means being clearly impartial and appreciating the wider IT landscape in which they specialize. Being an Ace doesn’t mean you only know Oracle products, many Ace’s in the integration and development space are often also certified on the Azure or AWS platforms for example. What you won’t find is an Ace publicly calling Oracle out, but then with the access afforded/acquired into the organisation means where there are concerns/challenges/issues they are communicated through the relationships developed and this input appears to be taken very seriously.

Benefits

When it comes to benefits, there are some, but I wouldn’t want people to think that it will ‘pay’ for the level of effort put in. The benefits are very much in the realm of acknowledgement for the contributions made. So yes, we get a few goodies – nice polo shirt with the community logo and the alike. Engraved glassware acknowledging your progress to Ace. The real reward for me is the community and having opportunities to share insights and a bit of acknowledgement of the effort invested, everything else is a bonus.

Share this:

  • Share on Facebook (Opens in new window) Facebook
  • Share on X (Opens in new window) X
  • Share on Pocket (Opens in new window) Pocket
  • Share on Reddit (Opens in new window) Reddit
  • Email a link to a friend (Opens in new window) Email
  • Share on WhatsApp (Opens in new window) WhatsApp
  • Print (Opens in new window) Print
  • Share on Tumblr (Opens in new window) Tumblr
  • Share on Mastodon (Opens in new window) Mastodon
  • Share on Pinterest (Opens in new window) Pinterest
  • More
  • Share on Bluesky (Opens in new window) Bluesky
  • Share on LinkedIn (Opens in new window) LinkedIn
Like Loading...

London Oracle Developer Meetup – OIC Patterns and more

09 Tuesday Jul 2019

Posted by mp3monster in Dev Meetup, drone, General, Oracle, Technology

≈ 2 Comments

Tags

developer, drone, meetup, OIC, Oracle

This Meetup was put together quickly as it presented an opportunity to align with other events happening in the Oracle offices. Despite the relatively short notice we a turn out that really made great use of our speaker – Sid Joshi who walked through the Enterprise Level patterns supported by Oracle’s Integration Cloud (OIC) including a demo showing how PaaS4SaaS worked using Service Cloud and OIC making use of VBCS and integration (formerly ICS) parts of the API Platform.

As with all the meet-ups we allow the discussions to flow freely. So, the conversation probed different aspects of OIC. So with the follow up on Several Capgemini use cases of OIC that have won the team awards.

Photo 08-07-2019, 19 15 44
Photo 08-07-2019, 19 15 09
Photo 08-07-2019, 19 19 00

You can see these use cases here. Sid’s presentation is available AppIntegrationPatterns_MeetUp. Additional resources can also be obtained from https://oracle-integration.cloud

As the conversation has focused on OIC and the use cases rather than our ongoing Drones with APIs stories, I have had an interesting follow on discussion about the application of drones.  The drone story has many threads.  The initial driver for the work on the drone has been about bringing something interesting and distinctive to the meetup.  The drone is very tangible, and the source of amusement which makes the meetups a lot more fun.

Continue reading →

Share this:

  • Share on Facebook (Opens in new window) Facebook
  • Share on X (Opens in new window) X
  • Share on Pocket (Opens in new window) Pocket
  • Share on Reddit (Opens in new window) Reddit
  • Email a link to a friend (Opens in new window) Email
  • Share on WhatsApp (Opens in new window) WhatsApp
  • Print (Opens in new window) Print
  • Share on Tumblr (Opens in new window) Tumblr
  • Share on Mastodon (Opens in new window) Mastodon
  • Share on Pinterest (Opens in new window) Pinterest
  • More
  • Share on Bluesky (Opens in new window) Bluesky
  • Share on LinkedIn (Opens in new window) LinkedIn
Like Loading...

Managing API Gateway Costs with Oracle API Platform

03 Wednesday Jul 2019

Posted by mp3monster in API Platform CS, APIs & microservices, General, Oracle, Technology

≈ 2 Comments

Tags

API, cost, gateway, Groovy, IDCS, Licensing, Oracle, scripting

The Oracle API Platform adopted an intelligent pricing model by basing costs on API call volumes and Logical gateway node groupings per hour. In our book about the API Platform (more here). We suggested that a good logical grouping would be to reflect the development, test, pre-production and production model. This makes it nice and easy to use gateway based routing to different environments without needing to change the API policy configuration as you promote your solution through environments.

We have also leveraged naming and Role/Group Based Access Controls to make it easy to operate the API Platform as a shared service, rather than each team having its own complete instance. In doing so the number of logical gateways needed is limited (I.e. not logical gateway divisions on per team models needed). Group management is very easy through the leveraging of Oracle’s Identity Cloud Service – which is free for managing users on the Oracle solutions, and also happens to a respected product in its own right.

Most organisations are not conducting development and testing 365 days a year, for 24 hours a day (yes in an ideal world prolonged soak and load tests would be run to help tease out cumulative issues such as memory leaks, but even then it isn’t perpetual). As a result, it would be ideal to not be using logical gateways for part of the day such as outside the typical development day, and weekends.

Whilst out of the out of hours traffic may drop to zero calls and we may even shutdown the gateway nodes, this alone doesn’t effectively reduce the number of logical gateways as the logical gateway aspect of the platform counts as soon as you create the logical group in the management portal. This in itself isn’t a problem as the API Platform drinks it’s own Champagne as the saying goes, and everything in the UI is actually available as a published REST endpoint. Something covered in the book, and in previous blog posts (for example Making Scripts Work with IDCS Deployed PaaS and Analytics and Stats for APIs). Rather than providing all the code, you can see pretty much all the calls necessary in the other utilities published.

Before defining the steps, there are a couple of things to consider. Firstly, the version of the API deployed to a specific logical gateway may not necessarily be the latest version (iteration) and when to delete the logical gateway this information is lost, so before deleting the logical gateway we should record this information to allow us to reinstate the logical gateway later.

As deleting logical gateways will remove the gateway from the system, when recreating the gateway we can use the same name, but the gateway is not guaranteed to get the same Id as before, as a result, we should when rebuilding always discover the Id from the name to be safe.

A logical gateway can not be deleted until all the physical nodes are reallocated, so we need to iterate through the nodes removing them. When it comes to reconnecting the nodes, this is a little more tricky as reconnecting the gateway appears to only be achievable with information known to the gateway node. Therefore the simplest thing is when bringing the node back online we take the information from the gateway-props.json file and run a script that determines whether the management tier knows about the node. If not then just re-run the create, start, join cycle., otherwise just run the start command.

As with the logical gateway, re-running the create, deploy, start cycle will result in the node having a new Id. This does mean that whilst the logical gateway name and even the node names will remain the same, the analytics data is likely to become unavailable, so you may wish to extract the analytics data. But then, for development and test, this data is unlikely to provide much long term value.

So based on this, our sequence for releasing the logical gateway needs to be…

  1. Capture the deployed APIs and the iteration numbers,
  2. Ideally shutdown the gateway node process itself,
  3. Delete all the gateway nodes from the logical gateway,
  4. Delete the logical gateway,

Recover would then be …

  1. Construct the logical gateway,
  2. Redeploy the APIs with the correct iteration numbers to the logical gateway using the recorded information- if no nodes are connected at this stage, the UI will provide a warning
  3. As gateway nodes, come back online, determine if it is necessary to execute the create, start, join or just start

Of course, these processes can be all linked to scheduling such as a cron job and/or server startup and shutdown processes.

Share this:

  • Share on Facebook (Opens in new window) Facebook
  • Share on X (Opens in new window) X
  • Share on Pocket (Opens in new window) Pocket
  • Share on Reddit (Opens in new window) Reddit
  • Email a link to a friend (Opens in new window) Email
  • Share on WhatsApp (Opens in new window) WhatsApp
  • Print (Opens in new window) Print
  • Share on Tumblr (Opens in new window) Tumblr
  • Share on Mastodon (Opens in new window) Mastodon
  • Share on Pinterest (Opens in new window) Pinterest
  • More
  • Share on Bluesky (Opens in new window) Bluesky
  • Share on LinkedIn (Opens in new window) LinkedIn
Like Loading...

Popping Up on the Net and more

28 Friday Jun 2019

Posted by mp3monster in APIs & microservices, Books, Dev Meetup, development, General, Oracle, Packt, Technology

≈ Leave a comment

Tags

API, blog, books, developer, interview, meetup, Oracle

It’s been a quiet month for this blog, but I’ve been pretty busy with a raft of other activities…

  • a recent article on our sister site – oracle-integration.cloud on  RPA.
  • I also appear in an interview with K21 Academy here.
  • 4186btedcpl._sx403_bo1204203200_Reviewing a new book on Enterprise API Management for Packt which we would very highly recommend if you want to understand the more Enterprise perspectives of adopting APIs, particularly if you’re considering APIs as a potential new revenue stream.
  • UK Oracle User Group committees for TechFest (having been reviewing the paper submissions it looks like it’s going to be an excellent conference in December) and Southern Summit (next week).
  • Just launched a number of sessions for the Oracle London Developer Meetup, with another to be announced soon (Blockchain) and potentially two more before the end of the year (we’re working on the speakers now).
ch-1
meetup-monitoring

 

Share this:

  • Share on Facebook (Opens in new window) Facebook
  • Share on X (Opens in new window) X
  • Share on Pocket (Opens in new window) Pocket
  • Share on Reddit (Opens in new window) Reddit
  • Email a link to a friend (Opens in new window) Email
  • Share on WhatsApp (Opens in new window) WhatsApp
  • Print (Opens in new window) Print
  • Share on Tumblr (Opens in new window) Tumblr
  • Share on Mastodon (Opens in new window) Mastodon
  • Share on Pinterest (Opens in new window) Pinterest
  • More
  • Share on Bluesky (Opens in new window) Bluesky
  • Share on LinkedIn (Opens in new window) LinkedIn
Like Loading...

API Security

01 Saturday Jun 2019

Posted by mp3monster in API Platform CS, APIs & microservices, General, Oracle, Technology

≈ Leave a comment

Tags

API, API Platform, api Security.io, Owasp, pen testing, SANS, Security

I’ve started to subscribe to the APISecurity.io newsletter. The newsletter includes the analysis of recent API based security breaches along with other useful API related news. Some of the details of the breaches make for interesting reading and provide some good examples of what not to do. It is rather surprising how regularly the lack of the application of good practises is, including:

  • Checking the payload is valid to the definition,
  • Checking the payload size to ensure it is in the expected bounds,
  • Use strong typing on the content received it will help validate the content and limit the chances of poisonous content like injected SQL,
  • owaspEnsuring the API has mitigation’s against the classic OWASP Top 10 – SQL Injection, poor authentication implementation.

More broadly, we see that people will recognise the need for applying penetration testing, and look to external organisations to perform the testing, when such work is commissioned the understanding of what the pen tester does is not understood by those logocommissioning the tests (SANS paper of security scoping), therefore know whether all the risks are checked. When you add to that, the temptation to keep such costs down resulting in the service provider not necessarily probing your APIs to the fullest extent. Not all penetration test services are equal, so simply working to a budget isn’t wise, yes there is a need for pragmatism, but only when you understand the cost/risk trade-off.

But also remember application logic and API definitions and the security controls in place change over time as do the discovery of new vulnerabilities on the stack you’re using, along with evolving compliance requirements. All meaning that a penetration test at the initial go-live is not enough and should be an inherent part of an APIs lifecycle.

cloudgs_apimgrWhen it comes to payload checks etc, products like Oracle’s API Platform make it easy to realise or provide out of the box checks for factors such as size limits, implementing payload checks, so better to use them.

If you ever need to be reminded that of why best practises are needed and should be implemented; a mindset of when not if a breach will happen will ensure you’re prepared and the teams are motivated to put the good practises in.

Share this:

  • Share on Facebook (Opens in new window) Facebook
  • Share on X (Opens in new window) X
  • Share on Pocket (Opens in new window) Pocket
  • Share on Reddit (Opens in new window) Reddit
  • Email a link to a friend (Opens in new window) Email
  • Share on WhatsApp (Opens in new window) WhatsApp
  • Print (Opens in new window) Print
  • Share on Tumblr (Opens in new window) Tumblr
  • Share on Mastodon (Opens in new window) Mastodon
  • Share on Pinterest (Opens in new window) Pinterest
  • More
  • Share on Bluesky (Opens in new window) Bluesky
  • Share on LinkedIn (Opens in new window) LinkedIn
Like Loading...
← Older posts
Newer posts →

    I work for Oracle, all opinions here are my own & do not necessarily reflect the views of Oracle

    • About
      • Internet Profile
      • Music Buying
      • Presenting Activities
    • Books & Publications
      • Logging in Action with Fluentd, Kubernetes and More
      • Logs and Telemetry using Fluent Bit
      • Oracle Integration
      • API & API Platform
        • API Useful Resources
        • Useful Reading Sources
    • Mindmaps Index
    • Monster On Music
      • Music Listening
      • Music Reading
    • Oracle Resources
    • Useful Tech Resources
      • Fluentd & Fluent Bit Additional stuff
        • Logging Frameworks and Fluent Bit and Fluentd connectivity
        • REGEX for BIC and IBAN processing
      • Java and Graal Useful Links
      • Official Sources for Product Logos
      • Python Setup & related tips
      • Recommended Tech Podcasts

    Oracle Ace Director Alumni

    TOGAF 9

    Logs and Telemetry using Fluent Bit


    Logging in Action — Fluentd

    Logging in Action with Fluentd


    Oracle Cloud Integration Book


    API Platform Book


    Oracle Dev Meetup London

    Blog Categories

    • App Ideas
    • Books
      • Book Reviews
      • manning
      • Oracle Press
      • Packt
    • Enterprise architecture
    • General
      • economy
      • ExternalWebPublications
      • LinkedIn
      • Website
    • Music
      • Music Resources
      • Music Reviews
    • Photography
    • Podcasts
    • Technology
      • AI
      • APIs & microservices
      • chatbots
      • Cloud
      • Cloud Native
      • Dev Meetup
      • development
        • languages
          • java
          • node.js
          • python
      • drone
      • Fluentbit
      • Fluentd
      • logsimulator
      • mindmap
      • OMESA
      • Oracle
        • API Platform CS
          • tools
        • Helidon
        • ITSO & OEAF
        • Java Cloud
        • NodeJS Cloud
        • OIC – ICS
        • Oracle Cloud Native
        • OUG
      • railroad diagrams
      • TOGAF
    • xxRetired
    • AI
    • API Platform CS
    • APIs & microservices
    • App Ideas
    • Book Reviews
    • Books
    • chatbots
    • Cloud
    • Cloud Native
    • Dev Meetup
    • development
    • drone
    • economy
    • Enterprise architecture
    • ExternalWebPublications
    • Fluentbit
    • Fluentd
    • General
    • Helidon
    • ITSO & OEAF
    • java
    • Java Cloud
    • languages
    • LinkedIn
    • logsimulator
    • manning
    • mindmap
    • Music
    • Music Resources
    • Music Reviews
    • node.js
    • NodeJS Cloud
    • OIC – ICS
    • OMESA
    • Oracle
    • Oracle Cloud Native
    • Oracle Press
    • OUG
    • Packt
    • Photography
    • Podcasts
    • python
    • railroad diagrams
    • Technology
    • TOGAF
    • tools
    • Website
    • xxRetired

    Enter your email address to subscribe to this blog and receive notifications of new posts by email.

    Join 2,555 other subscribers

    RSS

    RSS Feed RSS - Posts

    RSS Feed RSS - Comments

    January 2026
    M T W T F S S
     1234
    567891011
    12131415161718
    19202122232425
    262728293031  
    « Nov    

    Twitter

    Tweets by mp3monster

    History

    Speaker Recognition

    Open Source Summit Speaker

    Flickr Pics

    Gogo Penguin at the BarbicanGogo Penguin at the BarbicanGogo Penguin at the BarbicanGogo Penguin at the Barbican
    More Photos

    Social

    • View @mp3monster’s profile on Twitter
    • View philwilkins’s profile on LinkedIn
    • View mp3monster’s profile on GitHub
    • View mp3monster’s profile on Flickr
    • View mp3muncher’s profile on WordPress.org
    • View philmp3monster’s profile on Twitch
    Follow Phil (aka MP3Monster)'s Blog on WordPress.com

    Blog at WordPress.com.

    • Subscribe Subscribed
      • Phil (aka MP3Monster)'s Blog
      • Join 233 other subscribers
      • Already have a WordPress.com account? Log in now.
      • Phil (aka MP3Monster)'s Blog
      • Subscribe Subscribed
      • Sign up
      • Log in
      • Report this content
      • View site in Reader
      • Manage subscriptions
      • Collapse this bar
     

    Loading Comments...
     

    You must be logged in to post a comment.

      Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
      To find out more, including how to control cookies, see here: Our Cookie Policy
      %d