Simplifying the escaping of JSON strings

08 Thursday Jun 2023

Posted by mp3monster in development, General, Technology

Tags

CLI, development, JQ, JSON, software, Technology, utility

when you’re testing apps, it is pretty common to want to send JSON via CURL to a local endpoint. The problem is that this usually means that the string you provide curl needs to have characters escaped, such as quote marks. By hand, this can be irritating to sort out, particularly if you’re using an IDE to make sure the JSON is correct. I’d concluded this is hardly a new problem; someone must have produced a nice little multiple-platform command line utility that can do it for you. The result was a bit more surprising.

There are plenty of online utils that solve it, but if you’re working with data, you don’t want to publicly share (or the fiddling around with copy-pasting to your browser). Nothing wrong with these tools, but you can’t script them without resorting to RPA (Robotic Process Automation) either. Here are a couple of services I found that are straightforward, and when I’ve tried them, not plagued by annoying ads.

But finding command line tools, well, finding an answer, has proven a bit more challenging. For removing escaped characters, you could use jq, but we actually want to go the other way to use curl with JSON that has been escaped. I have come across conversations covering the use of bash (making use of awk and sed. Plus, details about how the manipulation could be done in various languages (so you could code your own solution if so inclined. Coding is unlikely to take much effort, but testing permutations is going to demand effort).

The one solution I have found that meant I could escape (or reverse) JSON locally is a plugin for VS Code called appropriately JSON-escaper, which does what is needed in a nice and clean manner. All credit to Joshua Poehls for the tool.

The solution JSON-escaper built on top of a more generic JavaScript utility which addresses escaping special characters which can be found here.

Bucharest Tech Week Conference – Monoliths in a Microservices World

29 Monday May 2023

Posted by mp3monster in General, Technology

≈ Leave a comment

Tags

anti-corruption, Apache, API, architecture, Bucharest, Celix, conference, Felix, Istio, Linkerd, micro-kernel, Microservices, monoliths, OSGi, presenting, Tech Week, Verrazzano

Last week I was fortunate enough to have the opportunity to present at the Software Architecture Summit as part of the Bucharest Tech Week conference. My presentation, Monoliths in a Microservice World, was all new content that, by chance, worked well, bringing together a number of points made by other speakers. The presentation aimed at the challenges of adopting Microservices and whether Monoliths had a place in modern IT, and for those of us not fortunate enough to be working for one of the poster children for microservices like Netflix, Amazon, etc, how we can get our existing monoliths playing nicely with microservices.

The conference may not have the size of Devoxx (yet), but it certainly had quality with presenters from globally recognized organizations such as Google (Abdelkfettah Sghiouar), Thoughtworks (Arne Lapõnin), Vodafone (IT Services business unit – _VOIS – Stefan Ciobanu), Bosch, as well as subsidiaries of companies like DXC (Luxsoft) and rapid growth SaaS vendor LucaNet.

As a presenter, you’re always wanting to walk the tightrope of being at the biggest conferences to maximize reach for your message while at the same time wanting the experience to be friendly and personable, which often means slightly smaller conferences. The Software Architecture Summit balanced that really well; rather than lots of smaller breakout sessions, the conference focussed on a single auditorium for a large number of attendees, with presentation slots varying in length depending upon the subject matter. If a session didn’t interest you, then there were plenty of exhibitors to talk with – although, from what I saw, the auditorium was full during the sessions, reflecting the interest in the content.

“Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live.” – John F. Woods
Quote of the conference – as cited by @DevPaco (Paco van Beckhoven)

The conference organizers (Universum) certainly put in the effort to ensure the presenters were looked after. It is the little touches that really make the difference, such as taking care of logistics which can be as simple as organizing airport transfers. A letter of thanks will be waiting for you at the hotel after the event, organizing a meal for the presenters at a local restaurant and so on.

Continue reading →

SSH Key File Permissions

22 Monday May 2023

Posted by mp3monster in development, General, Technology

≈ Leave a comment

Tags

keys, permissions, Security, SSH, Windows

Permissions on ssh key files on Windows can be rather annoying. If you try to use ssh it will protest about the permissions and will stop the secure connection. On Linux, it is easy to modify the permissions with a chmod command (chmod 700 *.key).

Update

Since originally writing this blog post, we came across a cmd (.bat) script that can alter the file permissions for Windows 10 and later (the basis of the script can be found here). With this script’s directory in the PATH variable, we can call it anywhere with the command protect-key.bat my-key-file.key, and it will correct the permissions accordingly.

View Code

To overcome the permissions issues, we need to make several changes to the file’s security properties to apply the following changes:

Switch off inheritance using the Disable inheritance button (images 1 and 2 below)
Remove grants to user groups other than Administrators (image 3)
Remove users who do not needing access is recommended.

The following image shows the ideal end state:

Free Information Security Book

11 Thursday May 2023

Posted by mp3monster in Books

≈ Leave a comment

Tags

APress, ebook, free, infosec, Security

Apress has made one of their InfoSec books freely available in EBook format here. It isn’t the most up-to-date text, but it does deal with a lot of the ideas, principals45, and issues rather than low-down detailed specifics, meaning it still holds a lot of relevance today (e.g. Social Engineering4), Cryptography, Malware, etc). So if you want an easy starter read into this space that’s free you can’t go wrong with this.

Article in DevOps Magazine – Stop Polling Let’s Go Streaming

02 Tuesday May 2023

Posted by mp3monster in APIs & microservices, Books, General, Technology

≈ Leave a comment

Tags

API, API Conference, blog, demio, devops, JAXLondon, magazine, polling, streaming

DevOps Magazine 2-2023 which includes my article

A while back, I was invited to contribute to Devmio (the knowledge portal driven by the publishers involved with the JAX London and other events). After a little bit of delay from my end, I offered an article that they decided was sufficient to be incorporated into DevOps magazine.

You can check out the content at:

Update (July 23) – no longer behind a paid wall

The article has been used in a blog post now and a mailout from the API Conference – read for free, check it out.

IAM and IDCS do more than support AuthZ

01 Monday May 2023

Posted by mp3monster in development, General, Oracle, Technology

≈ Leave a comment

Tags

data, development, OCI, Oracle, SCIM, Security, software

We could solve this with custom integrations, or we can exploit an IETF standard called SCIM (System for Cross-domain Identity Management). The beauty of SCIM is that it brings a level of standardization to the mechanics of sharing personal identity information, addressing the fact that this data goes through a life cycle.

While Oracle’s IDCS and IAM support identity management for authentication and authorization for OCI and SaaS such as HCM, SCM, and so on. Most software ecosystems need more than that. If you have personalized custom applications or COTS or non-Oracle SaaS that need more than just authentication and need some of your people’s data needs to be replicated.

The lifecycle would include:

Creation of users.
Users move in and out of groups as their roles and responsibilities change.
User details change, reflecting life events such as changing names.
Users leave as they’re no longer employees, deleted their account for the service, or exercise their right to be forgotten.

It means any SCIM-compliant application can be connected to IDCS or IAM, and they’ll receive the relevant changes. Not only does it standardize the process of integrating it helps handle compliance needs such as ensuring data is correct in other applications, that data is not retained any longer than is needed (removal in IDCS can trigger the removal elsewhere through the SCIM interface). In effect we have the opportunity to achieve master data management around PII.

SCIM works through the use of standardized RESTful APIs. The payloads have a standardized set of definitions which allows for customized extension as well. The customization is a lot like how LDAP can accommodate additional data.

The value of SCIM is such that there are independent service providers who support and aid the configuration and management of SCIM to enable other applications.

Securing such data flows

As this is flowing data that is by its nature very sensitive, we need to maximize security. Risks that we should consider:

Malicious intent that results in the introduction of a fake SCIM client to egress data
Use of the SCIM interface to ingress the poisoning of data (use of SCIM means that poisoned data could then propagate to all the identity-connected systems).
Identity hijacking – manipulating an identity to gain further access.

There are several things that can be done to help secure the SCIM interfaces. This can include the use of an API Gateway to validate details such as the identity of the client and where the request originated from. We can look at the payload and validate it against the SCIM schema using an OCI Function.

We can block the use of operations by preventing the use of certain HTTP verbs and/or URLs for particular or all origins.

New Article for SE Daily…

27 Thursday Apr 2023

Posted by mp3monster in ExternalWebPublications, Fluentd, General, Technology

≈ Leave a comment

Tags

Cloud, external publications, Hybrid, monitoring, multicloud, observability, SE Daily, Software Engineering Daily

We’ve just had a new article published for Software Engineering Daily which looks at monitoring in multi-cloud and hybrid use cases and highlights some strategies that can help support the single pane of glass by exploiting features in tools such as Fluentd and Fluentbit that perhaps aren’t fully appreciated. Check it out …

Challenges of Multi-Cloud and Hybrid Monitoring

Recommended Listening

26 Wednesday Apr 2023

Posted by mp3monster in Music

≈ Leave a comment

I’ve always had a fascination for pirate radio. A chance to hear non-mainstream playlisted music – that Peelsque subversiveness. My inner DJ may not be John Peel, probably closer to Lauren Laverne and Jo Whiley with the geekiness of Paul Gambaccini, but here are some suggested playlists …

OpenLens or Lens app

17 Monday Apr 2023

Posted by mp3monster in Cloud Native, development, General, Technology

≈ Leave a comment

Tags

K8s, Kubernetes, lens, OpenLens, openlens-node-pod-menu, Plugin

I wrote about how much I like the lens app K8s dashboard capability without needing to deploy K8s dashboard. Sadly recently, there has been some divergence from K8sLens being a pure open source to a licensed tool with an upstream open-source version called Open Lens (article here). It has fallen to individual contributors to maintain the open-lens binary (here) and made it available via Chocolatey and Brew. The downside is that one of the nice features of K8sLens has been removed – the ability to look at container logs. If you read the Git repo issue on this matter – you’ll see that a lot of people are not very happy about this.

If you read through all the commentary on the ticket, you’ll eventually find the following part of the post that describes how the feature can be reintroduced.

In short, if you use the extensions feature and provide the URL of the extension as @alebcay/openlens-node-pod-menu then the option will be reintroduced. The access to the extension is here:

The details …

The extension identified is detailed here.

I’m not sure why, but I did find the installation a little unstable, and needed to reinstall the plugin, restart OpenLens and reenable the plugin. But once we got past that, as you can see below the plugin delivered on its promise.

The problem with the licensing is that it doesn’t distinguish between me as an individual and using Lens for my own personal use vs. using Lens for commercial activities. The condition sets out:

ELIGIBILITY:You or your company have less than $10M in annual revenue or funding.
https://app.k8slens.dev/subscribe

Given this wording, I can’t use the licensed version, even if I was working on an open-source project and in a personal capacity, as the company I’m employed by has more than $10 million in revenue. For me, the issue is $200 per year is a lot for something I only need to use intermittently. While I get k8slens includes additional features such as Lens Security which performs vulnerability management, and Lens Teamwork, along with support, are features and services that are oriented to commercial use – these are features I don’t actually want or need. Lens Kubernetes sounds like an interesting proposition (a built-in distribution of K8s), but when many others already provide this freely – from Docker Desktop to Kind it seems rather limited in value.

We did try installing Komodor, given its claims for an always free edition. But on my Windows 11 Pro (developer early access) installation, it failed to install, as you can see:

Content not here…

15 Saturday Apr 2023

Posted by mp3monster in ExternalWebPublications, General, Technology

≈ Leave a comment

Tags

articles, external publications

Working on global blog domination …

We’ve been busy with blogs on Oracle – and getting to dominate the headlines, as you can see above. With the following blogs:

The API blog is part of a larger piece that explores the use of APIs across some of the industries that Oracle works in and how APIs can enable future innovations. The piece on OCI Queue is almost the opposite of the APIs material. Very detailed and implementation specific, covering the technical details of Terraform and the Stomp messaging protocol.

DZone

We also have an article over on DZone now, for regular readers of my blog, this is familiar stuff, as it looks at the LogGenerator utility I’ve built and working on custom extensions so it can be used with OCI to generate Notifications, messages on OCI Queue and more.