A Fast (and Dirty) Way to Publish API specs

03 Monday Jun 2024

Posted by mp3monster in APIs & microservices, General, Technology

Tags

API, AstyncAPI, AsyncAPI, Backstage, CLI, CNCF, Git, GitHub, npm, OAS, Open API, Simple Web Server

The API specs created using Open API Specification (OAS) and ASyncAPI specification aren’t just for public API consumption. In today’s world of modular component services that make up a business solution, we’re more than likely to have APIs of one sort or another. These need documenting, perhaps not as robustly as those public-facing ones, but the material needs to be easily accessible.

Spotify’s contribution to the CNCF—Backstage is a great tool for sharing development content, particularly when your document and code repository is at least git-based if not GitHub (you move away from this or don’t easily have permissions to configure application authentication, you can still work with Backstage, but your workload will grow a lot). There is no doubt that Backstage is a very powerful, information-rich product. But that does come at the cost of needing lots of configuration, the generation of metadata descriptors additional to the APIs to be cataloged, etc. All of these can be a little heavy if you’re using Backstage as a low-cost API documentation portal that might fill the gaps that your corporate wiki/doc management (Confluence/SharePoint) solution can’t support (it is one of the very, very few open-source options that can support both OAS and AsyncAPI reader friendly API rendering tools).

We could, of course, adopt the approach of there are free VS Code plugins that can render the friendly views of APIs, so just perform a git pull (or copy the API specs from a central location) to give the nice visualization. This is fine, but the obligation is now on the developer to ensure they have the latest version of the API spec and that they are using VSCode – while it is very dominant as an IDE – not everyone uses it, particularly if you’re working with low code tooling.

There is a fast and inelegant solution to this if you’re not in need of nice features such as attribute-based search and sorting, etc. Both the Open API Specification and the Async API communities have built command line-based renderers that will read your API specification (even if the schema is spread across multiple files) and generate HTML (an index.html file), CCS, and JavaScript renderings that you see in many tools (hyperlinked, folding, with code and payload examples of the API).

So, we need to grab the YAML/JSON specifications and run them through the tool to get the presentation formatting. You do need to get the specs, but we can easily script that with a bit of shell script that retrieves/finds the relevant files from a repository and then runs the CLI utility on the files.

We want to bring the static content to life across the network for developers. So, on a little server, we can host this logic, plus an instance of Apache, IIS, or Nginx if you’re comfortable with one of the industrial superpower web servers. Or use a spin-off project from the Chrome Server called the Simple Web Server. This tool is incredibly simple and provides you with a UI that allows you to configure quickly and easily and then start a web server that can dish up static content. I would hesitate to suggest such an approach for production use cases, but it’s not to be sniffed at for internal solutions, safely behind firewalls, network security, etc.

Steps in summary:

Install NPM
Install a Simpler Web Server – Apache, Nginx, or even Simple Web Server
Install the CLI tools for OpenAPI and AsyncAPI
Script to identify API documents and use the CLIs

Steps …

As all the functionality is dependent on Node, we need both Niode.js and NPM (Node Package Manager). Installing the Node Version Manager (NVM) is the easiest way to do that for Linux, and Mac with the command:

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash

Windows has a separately produced binary called NVM for Windows (which will eventually be superseded by Runtime), which has an installer that can be downloaded from the GitHub releases part of the repo.

Once nvm is installed (and ideally in the OS’ PATH environment variable) we can complete the process with the command:

nvm install lts

Which will see the latest Long Term Support (LTS) version installed.

Open API

To install Open API using NPM:

npm install @openapitools/openapi-generator-cli -g

The command that we will need to wrap in a script is:

npx @openapitools/openapi-generator-cli generate -i <your-open-api-spec.yaml> -g html2 -o <your-output-folder-for-this-api>

As the output generated is index.html with subfolders for the stylesheet and Javascript needed, we recommend using the name of the API Spec file (without the postfix, e.g., .yaml) as the folder name.

AsyncAPI

Just Like the Open API command line, we need to install the Async version using the command line:

npm install -g @asyncapi/cli

The equivalent command to generate the HTML is pretty similar, but, note over time, the template-referenced version will evolve (i.e. @2.3.5 to be a newer version)

asyncapi generate fromTemplate <your-async-api-spec.yaml> @asyncapi/html-template@2.3.5 -o ./<your-output-folder-for-this-api> --force-write

Scripting the Process

As you can see, we need to tease out the API files from the source folder, which may contain other resources, even if such resources are schemas that get included in the API (as our APIs grow in scope, we’ll want to break the definitions up to keep things manageable. but also re-use common schema definitions.

The easiest way to do this is to have a text file providing the path and name of the API definition. Each type of API has its own file – removing the need to first work out which type of API needs to be run.

This also means we can read all the API list files to determine then if any API spec pages need to be removed.

Final Thoughts

One of the things we saw when adopting this approach is that the generating process did highlight an issue in the API YAML that the VS Code plugin for Open API didn’t flag, which was the accidental duplication of the operationId when defining an API (an error when creating related API definitions using a bit of cut, paste, and edit).

A static documentation generator is also available for GraphQL (https://2fd.github.io/graphdoc/); although we have not tested it, the available examples, while making the schema navigable, it isn’t as elegant in presenting the details as our Async and Open APIs,

Extracting Dependencies and Versions for a Node Solution

15 Monday Feb 2021

Posted by mp3monster in development, NodeJS Cloud, Technology

≈ Leave a comment

Tags

jason, JQ, node.js, npm, parser

We have had a requirement from a customer to be able define every package including dependencies within a Node solution (as it happens Apollo GraphQL), not only the complete download path but the version numbering as well. There are many ways to solve this problem. But here is an elegant(?) and portable answer. To ensure that we don’t get pollution from a global node space we created a project package in an empty folder using:

 npm init --yes

This defaults all the package,json settings which for our requirements is fine. Then in the same location its npm install <product from the npm registry to pull> e.g. for Apollo GraphQL:

npm install apollo-server graphql

This will bring down to your npm project all the dependencies putting them in the node_modules child folder. We’re now in a position to retrieve all the details of the packages, their dependencies and version information. This can be done by using the command:

npm list --json

Continue reading →

Push Notifications with a bit of Node.js

16 Thursday Apr 2015

Posted by mp3monster in General, NodeJS Cloud, Oracle, Technology

≈ 1 Comment

Tags

API library, HelloWorld, javascript, JDeveloper, Node, node.js, npm, Oracle, prowl, push notifications

So I have written a couple of blogs about Push Notifications with a bit of Java (see here as the post that pulls all of this together). But this time we’re going to do something similar with Node.js. This blog entry is going to position us so we can then take a simple solution and push up to the cloud – as I use Oracle a lot then we’ll be looking at the Oracle cloud as a final step.

To start we need a local instance of node.js. Given the fact it is a small footprint we can pretty much install anywhere. So you’ll need to download Node.JS from the official site, and install it. I’m not going to walk through the installation guidance as it is well documented elsewhere (http://blog.teamtreehouse.com/install-node-js-npm-windows for example). You do want ensure you include the NPM capability (node.js packaging & deployment tool). Make sure that Node is on your path so we can reference the binary without a lots of file paths. You also want to ensure that node.js is up and running.

Next up is to the the Prowl API library that makes interacting with prowl simple and helps illustrate the deployment framework (NPM) used by node.js. So following the link from the Prowl website or go directly here and download the library. If you download the zip file as I did, you’ll find it has a folder called node-prowl-master. You need to unpack this and rename to node-prowl. and run the command

npm install node-prowl

When I first tried to deploy the Prowl API library then I did see an error. This isn’t the API but actually the Node.js installation (atleast on my Windows platform) as you can see:

I found googling using node.js and ENOENT showed up plenty of help to solve errors. In this situation the solution was purely to create the folder. Then re-running the action without problem.

When the npm command works you’ll see something like:

So hopefully in addition to the prerequisites described in this earlier post we should have everything ready to progress. So I’ve continued to use JDeveloper 12c, but using the general profile and set up a web solution project. This does create a large directory structure given we’re producing some simple Javascript. But the structure is right for a proper development effort, and lazy habits form poor practises – so lets work with it.

With the project setup, we need craft a little JavaScript. To we’re good to go – lets just try hello world, with a tiny twist, we’ll get the hostname using a Node library with this code:

// our very first node program

// get info about the OS
var os = require(‘os’);

// say hello world and include the hostname
console.log(“Hello world, we’re running on ” + os.hostname());

Before we do anything else, lets be a bit clever, to allow us to run our Node script within JDeveloper. This can be done by adding a new Tool through the Tools –> External Tools … menu. Which will display the following screen:

Asa you can see in this image I have already selected New… and walked through the configuration screens, you’ll probably want to use a configuration similar to what I have in the following steps:

With this setup in JDeveloper with the Editor focus on our JavaScript, goto the Tools menu and you’ll see your Node entry. Just click on it. We’ll then see the results in the message window, as you can see here:

Alternatively in a command window you just need to run the command from the folder with the JavaScript (or include the path):

node helloworld.js

So lets take things up a notch and send our mobile device a message. So using the following code, we can use the prowl-api and initiate a message:

var Prowl = require(‘node-prowl’); // pull in the prowl API we deployed with NPM earlier

var prowl = new Prowl(‘your-prowl-key-here‘); //setup your API key

var now = new Date();

// ready to send the message, passing a function reference to handle the response

var message = ‘hello mobile device, the time is ‘+ now.toUTCString();
prowl.push(message, ‘NodeJS App’, prowlReplyHandler);

//function to handle the response from the prowl API lib
function prowlReplyHandler ( err, remaining )
{

if( err )
{
var errorStr = err.message;
console.log( ‘I have an error ‘ + errorStr);
}
else
{
console.log( ‘I said:’ + message+ ‘; I have ‘ + remaining + ‘ calls available’ );
}

}

Note you’ll need to replace your-prowl-key-here in the above code with you genuine API key registered with the Prowl web app. Then we can run the application, and should see:

Our mobile device will show:

Next steps, in the next post – run through through a cloud hosting of node.js and extend the capability to be a simple service, which will mean packaging ourselves up and other exciting things.