Node (npm) package licensing

05 Tuesday Jul 2022

Posted by mp3monster in development, General, node.js, Technology

Tags

code, developer, development, Licensing, node.js, package, Technology

When building Node solutions, even if you’re not going to publish the code to a public repository you’re likely to be using package.json to declare the dependencies for your app. Doing this makes it easier to build and deploy a utility. But if you’re conversant with several languages there is a tendency to just adapt your existing skills to work with others. The downside of this is small tooling nuances can catch you off guard and consume time while figuring them out. The workings of packages with NPM (as shown below) is one possible case.

{
  "name": "graph-svr",
  "version": "1.0.0",
  "description": "packages needed for this service",
  "main": "index.js",
  "type": "module",
  "scripts": {
    "start": "node index.js"
  },
  "dependencies": {
    "@graphql-tools/graphql-file-loader": "^7.3.11",
    "@graphql-tools/load-files": "^6.5.4",
    "@graphql-tools/schema": "^8.3.10",
    "@graphql-yoga/node": "^2.4.1",
    "apollo-datasource-rest": "^3.5.2",
    "apollo-server": "^3.6.7",
    "graphql": "^16.4.0",
    "graphql-tools": "^8.2.8"
  },
  "author": "Phil Wilkins",
  "license": "MIT"
}

If you create the package.json using npm init to create the initial version of the file, it is fairly common to set values to default. In the case of the license, this is an ISC license. This is easily forgotten. The problem here is twofold:

Does the license set reflect the constraints of the dependencies and their licenses
Does the default license reflect the position you want?

Looking at the latter point first, This is important as organizations have matured (and tooling greatly improved) when it comes to understanding how open source licensing can impact. This is particularly important for any organizations leveraging open source as part of their revenue generating activities either ‘as a service’ but also selling software solutions. If you put the wrong license here the license checking tools often protecting code repositories may reject your code, even in internal only use cases (yes this tripped me up).

To help overcome this issue you can install a tool that will analyze the dependencies and optionally their dependencies and report back on your license exposure. This tool is called license-report. Once installed (npm install -g license-report) we just need to point the tool to the package.json file. e.g. license-report package.json. We can make the results a lot more consumable by outputting the content in a number of formats. For example a simple text value:

From this, you could set your license declaration in package.json or validate that your preferred license won’t conflict,

Managing API Gateway Costs with Oracle API Platform

03 Wednesday Jul 2019

Posted by mp3monster in API Platform CS, APIs & microservices, General, Oracle, Technology

≈ 2 Comments

Tags

API, cost, gateway, Groovy, IDCS, Licensing, Oracle, scripting

The Oracle API Platform adopted an intelligent pricing model by basing costs on API call volumes and Logical gateway node groupings per hour. In our book about the API Platform (more here). We suggested that a good logical grouping would be to reflect the development, test, pre-production and production model. This makes it nice and easy to use gateway based routing to different environments without needing to change the API policy configuration as you promote your solution through environments.

We have also leveraged naming and Role/Group Based Access Controls to make it easy to operate the API Platform as a shared service, rather than each team having its own complete instance. In doing so the number of logical gateways needed is limited (I.e. not logical gateway divisions on per team models needed). Group management is very easy through the leveraging of Oracle’s Identity Cloud Service – which is free for managing users on the Oracle solutions, and also happens to a respected product in its own right.

Most organisations are not conducting development and testing 365 days a year, for 24 hours a day (yes in an ideal world prolonged soak and load tests would be run to help tease out cumulative issues such as memory leaks, but even then it isn’t perpetual). As a result, it would be ideal to not be using logical gateways for part of the day such as outside the typical development day, and weekends.

Whilst out of the out of hours traffic may drop to zero calls and we may even shutdown the gateway nodes, this alone doesn’t effectively reduce the number of logical gateways as the logical gateway aspect of the platform counts as soon as you create the logical group in the management portal. This in itself isn’t a problem as the API Platform drinks it’s own Champagne as the saying goes, and everything in the UI is actually available as a published REST endpoint. Something covered in the book, and in previous blog posts (for example Making Scripts Work with IDCS Deployed PaaS and Analytics and Stats for APIs). Rather than providing all the code, you can see pretty much all the calls necessary in the other utilities published.

Before defining the steps, there are a couple of things to consider. Firstly, the version of the API deployed to a specific logical gateway may not necessarily be the latest version (iteration) and when to delete the logical gateway this information is lost, so before deleting the logical gateway we should record this information to allow us to reinstate the logical gateway later.

As deleting logical gateways will remove the gateway from the system, when recreating the gateway we can use the same name, but the gateway is not guaranteed to get the same Id as before, as a result, we should when rebuilding always discover the Id from the name to be safe.

A logical gateway can not be deleted until all the physical nodes are reallocated, so we need to iterate through the nodes removing them. When it comes to reconnecting the nodes, this is a little more tricky as reconnecting the gateway appears to only be achievable with information known to the gateway node. Therefore the simplest thing is when bringing the node back online we take the information from the gateway-props.json file and run a script that determines whether the management tier knows about the node. If not then just re-run the create, start, join cycle., otherwise just run the start command.

As with the logical gateway, re-running the create, deploy, start cycle will result in the node having a new Id. This does mean that whilst the logical gateway name and even the node names will remain the same, the analytics data is likely to become unavailable, so you may wish to extract the analytics data. But then, for development and test, this data is unlikely to provide much long term value.

So based on this, our sequence for releasing the logical gateway needs to be…

Capture the deployed APIs and the iteration numbers,
Ideally shutdown the gateway node process itself,
Delete all the gateway nodes from the logical gateway,
Delete the logical gateway,

Recover would then be …

Construct the logical gateway,
Redeploy the APIs with the correct iteration numbers to the logical gateway using the recorded information- if no nodes are connected at this stage, the UI will provide a warning
As gateway nodes, come back online, determine if it is necessary to execute the create, start, join or just start

Of course, these processes can be all linked to scheduling such as a cron job and/or server startup and shutdown processes.

Oracle PaaS – the Good and opportunities to get better

19 Saturday Mar 2016

Posted by mp3monster in General, Java Cloud, Oracle, Technology

≈ Leave a comment

Tags

Cloud, deployment, java, Licensing, Oracle, PaaS, SOA

Although Oracle have been late to the cloud party they are certainly making up for it, by bringing products to the cloud at an amazing pace, and using their core products to build out new offerings at a rate that will mean they will at least catch all the competition across the breadth of PaaS very quickly.

When it comes to taking on Oracle PaaS it does have some quirks, some relate to Oracle’s normal licensing approach, and others I’m told relate to the way US accounting has to work when it comes to realised revenue. A couple of other characteristics I suspect are linked to the fact that the infrastructure for Oracle’s cloud is still being rolled out and grown for capacity.

So firstly the carry over – well outside of a trial account you need to agree and sign a general agreement which provides an overarching legal framework defining terms, conditions and liabilities. This makes dealing with each subsequent purchase a little simpler. Rather than purchasing services as you go, you then purchase credits from Oracle which have a maximum life of 1 year. This does mean you’re not got a pure OPEX spend model – although you do stand a chance of negotiating a better deal as the numbers are naturally bigger. As part of the agreement you’ll get a rate card, so different services cost different amounts – for example a standard edition database will cost x and an enterprise high performance version will cost a bit more. The credits are for specific product families such as SaaS products, products in the PaaS domain for example document cloud, Java cloud, SOA and so on. But make sure the products you might want are in the families you get credits for, there is the odd surprise for example MBaaS isn’t in the same family as the integration products.

In addition your negotiation you need to consider whether services are in metered or unmetered models. Unmetered means you agree a level of capacity for the year. This will obviously work out cheaper than a metered model where you can use up your credits as you choose, with different metering rates – for example hourly and monthly. When this was first explained it looked really good for dealing with the situation of having a baseline demand which could be unmetered and then purchasing metered services to capacity burst. Sadly this isn’t possible out of the box. I suspect because of the way Oracle cloud allocates workload to different work domains. So bursting workload would have to be done as if you’re bursting in 2 different clouds. So if you have a dynamic load you either go unmetered to your maximum demand or metered for everything. Either way you’re not getting the best in terms of cost management. I have to admit I don’t know whether the likes of AWS and Azure when you enter into long term agreements have the same challenges.

One the positive side, with the credits you can then purchase a broad range of configurations of products from just ADB schema all the way a full size Exadata setup. So performing PoCs is pretty easy and figuring out scaling just means burning your credits quickly and instantiating more capacity.

Before getting into instantiating your cloud instances you’d best setup access controls to allow people access controls to creating instances. Then you can start creating instances of the products you want. Make sure you protect your credentials as the way things are setup anyone else recovering them will be a problem.

With services such as SOA and Java you do need to go through the process of creating the different layers, storage, then the database and so on. But unlike building on premise each step only requires a couple of clicks and your done. To put it into context the first time I built a small footprint 11g environment took me a couple of days to work my way through on my own create a DB, deploy RCU,Weblogic, SOA and AIA foundation (no load balancing or security etc) and was no way near secure as a cloud instance. Oracle PaaS in three hours we:

Meet our Customer Success Manager (more on this shortly)
Got the utilities such as putty installed on my laptop
Walked through putty’s key generation quirks and how to avoid the gotchyas
being walked through the process, of setting up management rights to our credits and instance creation
Instaniated storage, debated on the DB option to use, created the SOA CS instance with OSB, a load balancer and configured SSH security and web access routes to our cloud. Plus setup my developers
Had a couple cups of coffee and ordered lunch

With SOA CS and atleast some of the other cloud offerings you also get SSH access to the OS so you can tinker and tune your SOA container and Weblogic etc. Some would argue that totally undermines the ideal of PaaS and that exploiting such a capability means you can end up customising your deployment to the point it will break the moment an update or patch comes along. So it is very double edged. In my mind (but I’m a techie at heart so seeing the engine running is always interesting) it’s good, but must be handled with great care. As they say – with great freedom comes great responsibility.

One of the real wins is that Oracle allocate customers a Cloud Success Manager who are tasked with enabling you to use the Oracle cloud – any problems, guidance needed can be addressed through these people. A cynic might say they exist to help you spend money which becomes released revenue. But our experience is the CSMs are genuinely enthusiastic and helpful – answering questions at 6pm on a Friday (despite my school boy error).

So in our experience so far I’d suggest Oracle could do two things to really make a big advancement – commercially atleast:

Allow payments to be made on a reoccurring model as an alternative to the credits model, perhaps this approach restricts you to metered only services
Allow metered and unmetered services to be utilised together – perhaps as a stretched cluster mentality.

This was first made available at https://community.oracle.com/groups/united-kingdom-user-group/blog/2016/03/25/starting-with-oracles-soa-cs

Oracle SOA Components

20 Saturday Jun 2015

Posted by mp3monster in General, Oracle, Technology

≈ Leave a comment

Tags

Licensing, Oracle, products, SOA, SOA Suite

Understanding all the different elements available to you with Oracle SOA Suite can be a little tricky to say the least as the often the building blocks available are shown in diagrams like:

As shown in http://docs.oracle.com/cd/E28280_01/doc.1111/e10223/01_components.htm

They say a picture says a thousand words. But, those thousand words can omit some details. Take this diagram for example, it only reflects the common elements in the main SOA server. But within the SOA license you also have Oracle Service Bus (OSB) and the Complex Event Processing (CEP) engine which run as separate servers. Which means you’d see the following:

as shown in http://docs.oracle.com/cd/E28280_01/doc.1111/e10223/04_osb.htm

as shown in http://docs.oracle.com/cd/E28280_01/doc.1111/e10223/14_cep.htm

Oh, but hang on; we still haven’t got a representation of JMS, and Coherence. JMS like CEP and OSB actually instantiates as a separate server as well. So got a handle on everything now? Well there is a little confusion still needing to be added to the mix. The following diagram is commonly used showing interoperability with other products with separate licenses:

shown in http://docs.oracle.com/cd/E28280_01/doc.1111/e10223/507_eda.htm

Taking the above diagram at face value, you could interpret things as actually you don’t have the CEP capability as it is the key part of the Oracle Event Driven Architecture Suite.

I have merged together the diagrams to show the bulk of all the SOA stack that comes into the main SOA license. As you can see ….

Designing in a license constrainted World – Reprise

16 Monday Jun 2014

Posted by mp3monster in General, Technology

≈ Leave a comment

Tags

constrained, Design, license, Licensing

A while back I blogged (here) about designing in a license constrained world. Well I’ve taken that blog further and developed a supporting slide deck, to help elaborate on the thinking.

Architecting within a License constrained world

20 Thursday Feb 2014

Posted by mp3monster in Oracle, Technology

≈ Leave a comment

Tags

architecture, Design, Licensing, Oracle, Patterns

In an ideal world software design shouldn’t be driven by software license costs if constraints. But when you can be paying tens or hundreds of thousands of dollars per server for an application or middleware it isn’t an aspect you can ignore. The challenge is when licensing rules are so complex like those for Oracle you either end up with licensing experts reviewing design artefact or you need to find an alternate approach (and the hope of using agile strategies with such a review framework necessary have gone).

For those less aware of Oracle’s licensing you have be licensed by CPU, by users, by profitability and probably will be impacted by atleast 2 of these models. Then each license can also be constrained by usage (unlimited or limited) which says that you can use some products with some things and not others, or use your licenses for only particular activities. Finally you have product dependencies, so the licensing of 1 product and indirectly impact how you can use another. For example I may have unlimited use for Weblogic (on 20 CPUs) but SOA Suite, the components that together allow you to run Process Integration Packs (PIPs) which as a Fusion Middleware offering provide a collection of middleware components to achieve common tasks – for example keep your customer information synchronised between a CRM solution and your accounting solution, which maybe limited to only work with Oracle applications – so extending a PIP to also send one of your own application an event wouldn’t be allowed (unless you’ve built an extension on an approved Oracle application). Then for fun you have what are called Unlimited License Agreements (ULAs) – although they’re not really unlimited.

Just when you think you’ve got a grip of the licensing story, there is one more mix of the pot. When you’re negotiating licensing you’re likely to be working through a purchasing team who aren’t technical Oracle product experts, and licensing discussions are likely to be done whilst costing a programme where unless you’re an enterprise mature organisation or operationally very well instrumented to measure this information it isn’t going to be easy to get volumetrics and an ability to determine likely throughput (i.e. how complex and demanding will your custom logic be). So by the time you get to from your conceptual to-be perspective which told your which products you need to when you’re actually working on the realisation you may well hit challenges.

With all of this in mind, we’ve arrived with the idea of usage scenarios. We’ve tried to differentiate usage scenarios from design patterns, as their goals also differ; a pattern is typically to provide a means to describe and provide good design approaches to technical problems, think of things facades and factory’s from the Gang of Four (GoF) or composite patterns such as VETO and here we seeking a means to communicate what can or can’t be done. These aren’t use cases either, if for no other reason to avoid the UML notation association.

So how does it work, so we have identified common or likely approaches to using our Oracle technology stack, need them so there is a short hand reference (as you have with design patterns) and then determined of the scenario is permissible by licensing rules. The idea is that an application architect or developer can design a solution and then verify the solution against the scenarios. To start with go for the obvious scenarios, as things go forward when a situation crops up where there isn’t a scenario you can add the the catalogue and get confirmation as to compliance. This should mean after a short period of development you’ll reach a point where you’re not consulting licensing experts all the time. The secret is not to try ‘boil the ocean’ on day 1 as you’ll invest a lot of time, potentially creating representations of things you’ll never do and produce a very bulky artefact for your developers to try and work with. Oracle’s AIA Developer Guide

With the scenario we document references to the various license and contract documents showing which clauses drove the decision so you don’t have to rework out how you determined the legitimacy of the scenario. I’ve created a fake representation of a usage scenario below.

There is a further bonus, you can drive into the guidance when there is a need for additional governance attention.

Of course this mechanism doesn’t tackle the question of is there sufficient licensed capacity. As capacity management has its own set of challenges (such as balancing the capacity requirement forecasts for multiple current development programmes that are likely to be taking place vs actual consumption and forecast consumption for business growth).

The following diagram is a mock up of the sort of diagrams produced. Mocked up as I don’t want (and shouldn’t) disclose any information about what specific technologies and approaches we’ve adopted internally.