• Home
  • Site Aliases
    • www.cloud-native.info
    • oracle.cloud-native.info
    • Phil-Wilkins.uk
  • About
    • Background
    • Presenting Activities
    • Internet Profile
      • LinkedIn
    • About
  • Books & Publications
    • Logging in Action with Fluentd, Kubernetes and More
      • Logging in Action with Fluentd – Book
      • Fluentd Book Resources
      • Log Generator
    • API & API Platform
      • API Useful Resources
    • Oracle Integration
      • Book Website
      • Useful Reading Sources
    • Publication Contributions
  • Resources
    • GitHub
    • Oracle Integration Site
    • Oracle Resources
    • Mindmaps Index
    • Useful Tech Resources
    • Python Setup & related stuff
  • Music
    • Music Reading

Phil (aka MP3Monster)'s Blog

~ from Technology to Music

Phil (aka MP3Monster)'s Blog

Monthly Archives: April 2022

Visual Studio Code – Oracle Plugins

26 Tuesday Apr 2022

Posted by mp3monster in development, General, Technology

≈ Leave a comment

Tags

code, development, Oracle, plugins, tools, Visual code

Oracle’s product portfolio is significant, from databases (obviously) to GraalVM to a cloud platform capable of competing with GCP, AWS, and Azure. This means locating the Oracle-provided plugins, or community ones can get messy. Depending on your perspective Oracle Developer Plugins could relate to Java and GraalVM or Oracle Database.

As broad as the portfolio, is the Oracle details regarding the plugins. So the following two tables represent what we’ve identified as Oracle-provided tooling, and the second table of plugins we’ve used when working on Oracle-based solutions from the community.

Name / Plugin SearchDescription / Additional DetailsRelated resource links
search:Oracle Labs
This will return all the Oracle plugins related to GraalVM
There are several different extensions covering GrallVM
– GraalVM Tools for Java
– GraalVM Extension Pack for Java (the name here is a little miss leading as there is support for JavaScript, Python, Ruby, R)
– GraalVM Tools for Micronaut
– PointTo-SourceLine
The extension packs also helps bring of SOA journey,
GraalVM.org
micronaut
Oracle Developer Tools
(PL/SQL & Oracle Database)
Provider is Oracle Corporation
This extension enables editing and execution of SQL and PL/SQL for Oracle Database and Oracle Autonomous Databasehttps://www.oracle.com/
database/technologies
/appdev/dotnet/

odtvscodequickstart.html
Oracle JET corelibrary of UI elements that form part of a web UI.Udemy training
Oracle NetSuiteSuiteCloud Extension for Visual Studio Code is part of the SuiteCloud Software Development Kit (SuiteCloud SDK), a set of tools to customize your NetSuite accounts.Netsuite dev blog
Oracle Provided Plugins

Name / Plugin SearchDescription / Additional DetailsRelated resource links
OCI Policy LanguageThis extension is a language highlighter for OCI Policies.
CPQ DevKitCPQKit™ is a website built to augment Oracle CPQ Cloud system’s functionalityhttps://www.cpqkit.com/
ApexIntelliSenseIntellisense for APEX
Docker
YAMLProvides comprehensive YAML Language support to Visual Studio Code, via the yaml-language-server, with built-in Kubernetes syntax support.
KubernetesThe extension for developers building applications to run in Kubernetes clusters
Language Support for JavaSupports Java 11 onwards
GitHub ActionsGithub actions is a means by which actions like commits to github trigger external infrastructure to perform actions such as creating application binaries.
Regexp Explainhelp to evaluate and develop regular expressions

Share this:

  • Twitter
  • Facebook
  • LinkedIn
  • Print
  • Pocket
  • Email
  • Tumblr
  • Reddit
  • Pinterest
  • WhatsApp
  • Skype

Like this:

Like Loading...

Helping ‘mini me’ learn and practice Python coding

21 Thursday Apr 2022

Posted by mp3monster in development, General, Technology

≈ Leave a comment

Tags

children, coding, education, Flask, learning, Music, OrangeCD Catalog, programming, python, STEM

My eldest son is studying computer science at school. As part of that course, like most educational settings they’re using Python to teach programming skills. Having sat with my son to help him get to grips with the coding the different ways of looping, implementing conditions, variable scopes and so on it has been interesting to see that school student level programming books take developers through very very simple steps with the rewards of success immediate but of limited value.

The challenge and concern, with this, is the step from this to delivering something meaningful, that continues to provide a sense of reward, while also something that is closer to real-world work.

This is where a small project I had comes into play. I have for many years been using an app called Orange CD catalog to record all the details of my music collection. The only problem is that it is a thick desktop app and the web solution called Racks and Tags hasn’t seen much development from what I can see for a long time and by current user experience looks archaic. As you may have guessed our pet project is to provide a web interface for the exported data from the Orange CD catalog. You can see the code in my GitHub repo here). I’ve carefully structured the code so the logic of processing the data (i.e. loops and conditions using data structures – using Python’s XMLTree library, which behaves a lot like lists – helping with basic learning). The web serving tier is implemented with Flask using simple annotations. This provides a nice opportunity to talk about the use of HTTP. The Jinja templates are also kept separate, so we don’t get bogged down with HTML, which can be a bit messy (IMHO).

Orange CD Catalogue

We’ll deploy the solution to our little Raspberry Pi, so it can be used anywhere in our network. Ultimately we could use it to stream the albums I’ve ripped to MP3. If my son wants/needs to learn a little bit about databases – then migrating the data into something like SQLLite becomes a simple development opportunity.

This does mean some prep work and including part of the solution, so something works very quickly. For me that’s fine, my Python web skills could benefit from some work and Jinja2 is new to me. But I could take the project embed suitable security and deploy it to my free Oracle cloud instance as a container into K8s. Those bits I won’t be sharing (minimize the risk of someone wanting to test how well I patch code etc).

Feel free to take the code and use it for yourself, extend, etc. But you don’t have to use my idea. The important thing is the idea is going to interest your child. But keep it simple. for example, if your child is a gamer, then perhaps you generate a simple data extract from the gaming servers to capture player performance, then your app does simple things like searching the content, sorting it, and providing a vanilla UI. The secret I think is each bit of the project is simple steps and not over well with heaps of code. Provide results quickly for example just search for an album by its title, and get the results into a webpage quickly.

Share this:

  • Twitter
  • Facebook
  • LinkedIn
  • Print
  • Pocket
  • Email
  • Tumblr
  • Reddit
  • Pinterest
  • WhatsApp
  • Skype

Like this:

Like Loading...

Handling multi-line log entries with Fluentd

19 Tuesday Apr 2022

Posted by mp3monster in Books, Fluentd, General, logsimulator, manning, Technology

≈ Leave a comment

Tags

book, Fluentd, multiline, parser, regex

One aspect of logging I didn’t directly address with my Fluentd book was consuming multiline logs, such as those you’ll often see when a stack trace is included in the log output. Implementing the feature with Fluentd isn’t hugely complex as it leverages the use of regular expressions (addressed in the book in more depth) to recognize the 1st line in a multiline log entry and for subsequent lines.

I didn’t address it for a couple of reasons:

  • Using parsers is fairly inefficient, particularly when you’re using a parser to just decide how to then transform a line (this is why I’m not a huge fan of some of the 12 Factor App‘s recommendations when it comes to logging).
  • Incorporating into your Fluentd parser configurations for specific app log setups is arguably increasing the level of coupling.
  • Many logging frameworks can talk directly to Fluentd as we saw in the book. This is can be more efficient, which means that the log event is more likely to be passed over in a structured format (therefore less work to do).
  • Alternatively, frameworks like Log4J2 have the means to strip line feeds, etc at source – https://logging.apache.org/log4j/2.x/manual/layouts.html (see replace)

But let’s also be realistic, many applications will be configured to simply log to a file and aren’t likely to be changed. At which point we do need to process such situations, so is it done? The process remains largely the same as the tail plugin we illustrated. Except we introduce a different parser called multiline. The documentation provided by Fluentd includes several examples of multiline configurations that will work for default log formats (such as Log4J and Rails). If we took our most basic source setup:

<source>
  @type tail
  path ./Chapter3/basic-file.*
  read_lines_limit 5
  tag simpleFile
  <parse>
    @type none
  </parse>
</source>

Then assuming our log Simulator played back multiline logs (the provided configuration doesn’t do that) extended to consume standard Log4J2 logs we would have a configuration as follows:

<source>
  @type tail
  path ./Chapter3/basic-file.*
  read_lines_limit 5
  tag simpleFile
  <parse>
     @type multiline
     format_firstline /\d{4}-\d{1,2}-\d{1,2}/
     format1 /^(?<time>\d{4}-\d{1,2}-\d{1,2} \d{1,2}:\d{1,2}:\d{1,2}) \[(?<thread>.*)\] (?<level>[^\s]+)(?<message>.*)/
  </parse>
</source>

As you can see we’ve set the parser type to multiline. Then there are two regular expressions, format_firstline is used to help recognize the start of a log event. Every line of the log is tested with this expression as we now assume unless this produces a valid result that the line will be part of a multiline event. If you look at the expression you’ll realize it is looking for a DateTime stamp in the form YYYY-MM-DD. This does mean if you generate a log that starts with the date even if it is part of a multiline output then you’ll trip up the parser. You could extend the expression – but the longer it is the slower the processing.

Following format_firstline we have in our example format1 which describes how to process the first line. This can be extended to define how to handle subsequent lines but this could be multiple format definitions. They do need to be presented in numerical order eg. format1, format2, format, and so on.

LogSimulator – Playing back multi-line logs

The Log simulator uses a very similar mechanism to Fluentd to understand how to playback multiple line logs. When it is reading the log lines in for replay it uses a regular expression to recognize the start of a new log entry (FIRSTOFMULTILINEREGEX) defined in the properties file. The simulator will concatenate lines together until it either hits the end of the file or has a new line that complies with the REGEX. It stores the line with an encoded /n (newline character). It will then print the log using the format specified and will allow the /n to create a newline (or not) based on another config parameter (ALLOWNL).

Share this:

  • Twitter
  • Facebook
  • LinkedIn
  • Print
  • Pocket
  • Email
  • Tumblr
  • Reddit
  • Pinterest
  • WhatsApp
  • Skype

Like this:

Like Loading...

Avoid creating an event storm when using social outputs with Fluentd

14 Thursday Apr 2022

Posted by mp3monster in Fluentd, General, Technology

≈ Leave a comment

Tags

Fluentd, logging, mail storm, monitoring, Plugin, social, supress

Fluentd has an incredible catalogue of plugins including notification and collaboration channels from good old-fashioned email through to slack, teams, and others.

The thing to remember if you use these channels is that if you’re sending errors, from application logs it isn’t unusual for there to be multiple error events as a root event can cause a cascade of related issues. For example, if your code is writing transactions to a database and the database goes down with no failover mechanism, then your code will most likely experience an error, roll back the transaction perhaps to some sort of queue, and then try to process the next event. Which will again fail. This is the classic situation where multiple errors will get reported for the same issue. This problem is often referred to as a mail storm given that there was a time when we didn’t have social collaboration tools and everyone used email.

There are several ways to overcome this problem. But the most simple and elegant of these is using the suppress plugin in its filter mode.

<filter **>
  @type suppress
  interval 60       # period in seconds when the condition to supress is triggered
  num 2             # number of occurences of a value before suppressing
  attr_keys source  # the element of the event to consider.
</filter>

In this example if we encounter an event with an attribute called source containing the same value twice then the suppression will kick in for 60 seconds. If you want the key to the valuebeing checked to be the tag then simply omit the attr_keys parameter.

Of course, we don’t want the suppression to kick in if the same value in the attribute keys occured once every few hours. To address this the occurence count is applied over not a time period, but a number of events received by the configuration of max_slot_num which defaults to 10k, but resets

In the filter mode, this plugin is best positionbed immediately before the match block. This means we don’t accidentally suppress messages before they are routed anywhere else.

For the purposes of a demo this is less of an issue. But for a realworld use case would probably benefit from some tuning. All the documentation for this plugin is at https://github.com/fujiwara/fluent-plugin-suppress

Share this:

  • Twitter
  • Facebook
  • LinkedIn
  • Print
  • Pocket
  • Email
  • Tumblr
  • Reddit
  • Pinterest
  • WhatsApp
  • Skype

Like this:

Like Loading...

Web Assembly – Free O’Reilly Book

13 Wednesday Apr 2022

Posted by mp3monster in Books, General

≈ Leave a comment

Tags

book, free, oreilly, WASM, WebAssembly

Red Panda is currently making the O’Reilly book by Brian Sletten – WebAssembly the Definitive Guide free to download (well, the price of your email address).

So why is worth snagging a download? Well, WebAssembly (Wasm) is the new way to deliver browser-based solutions to a client without having to develop with JavaScript. As a result, you have the chance to reuse more of your existing code. It also has the benefit of having been precompiled to a binary form, offering code security and compact download. But Wasm also has been seeing adoption in back-end server solutions as well, particularly for feature extensions, etc.

Share this:

  • Twitter
  • Facebook
  • LinkedIn
  • Print
  • Pocket
  • Email
  • Tumblr
  • Reddit
  • Pinterest
  • WhatsApp
  • Skype

Like this:

Like Loading...

Knowledge Resources from Oracle

05 Tuesday Apr 2022

Posted by mp3monster in Cloud, General, Oracle, Technology

≈ Leave a comment

Tags

documentation, knowledge, Oracle, resources

Before joining Oracle I used to typically refer to a couple of key resources from Oracle – docs.oracle.com, and occasionally developer.oracle.com and ateam-oracle.com. We’d obviously use cloud.oracle.com and the main oracle.com to be able to reference published stats, success references etc. Now I’m part of the company and working in the OCI product team with an outbound side of things, I needed to gem up on all the assets that exist. So that we can help contribute, and ensure that they are up to date etc. In doing so, the number of resources available is so much more than I’d realized.

Upon reflection, this may have been from the fact we didn’t drill down deeply enough, also in part that Capgemini has its own approaches and strategies as well.

This in part is linked to the organizational structures e.g. OCI Product Management’s outbound work overlaps with the Marketing Developer Relations, for example, something that is inevitable in an organization that provides such a diverse portfolio of products.

For my own benefit, and for others to exploit, the following table summarises the different areas of information. The nature of the content and – where content overlaps or is presented in different ways.

We’ve moved this content so it can be easily revised to here (and accessible from the site menu). But also available here …

oracle.com/goto/ …Domain AddressDescription
oracle.comThis is the commercial side of the Oracle content. But includes a landing page for a lot of Oracle knowledge/documentation.
docs.oracle.comThe home of the core Oracle documentation for all products.
/blogblogs.oracle.comThe members of this team are the ‘gurus’ of product applications. These cover a range of domains – structured in a similar way to blogs.oracle.com with different posts. These posts represent patterns and solutions to problems encountered by the team. How to, or not to implement things.
This can overlap with some blogs in so far as both product blogs and A-Team blogs may address how to leverage product features.
A-TeamDeveloper relations lead, which covers not only Oracle products but also the application of open source. By its very name, there is a strong emphasis on coding (rather than low-code) covering not just Java but .Net languages such as C#, Node, JavaScript, and so on.
There is some content overlap here with the Architectural Center, where Architecture Centre provides reference solutions.
/devdeveloper.oracle.comDeveloper relations lead, which covers not only Oracle products but also the application of open source. By its very name, there is a strong emphasis on coding (rather than low-code) covering not just Java, but .Net languages such as C#, Node, JavaScript, and so on.
There is some content overlap here with the Architectural Center, where Architecture Centre provides reference solutions.
/ref-archs
/playbooks
/deployed
docs.oracle.com/solutions
(AKA Architecture Center)
This is the Architecture Center which provides reference solutions. But these aren’t exclusive to the SaaS products (which would be easy to interpret). A lot of examples cover deploying and running open-source solutions on IaaS, for example, Drupal, WordPress, and Magento to name just a couple. A lot of these are backed up with scripts, Terraform, and code to achieve the deployment and configuration.
In addition to this, there are use cases of what customers have deployed into production (known as built and deployed).
docs.oracle.com/learn/This contains a lot of free tutorials and labs that can be taken a run to implement different things, from deploying a Python with Flask solution on Kubernetes to Creating USB Installation Media for Oracle Linux with Fedora Media Writer. As you can see from these examples, the tutorials cover both Oracle products and open source.
These resources interlink with the Architecture Centre and can overlap with developer.oracle.com.
/gh-devrelgithub.com/oracle-devrelThis contains the code artifacts developed by Developer Relations and the Architecture Center team. So covers Reference Architectures, tutorials, and Live Labs all freely available to use.
/gh-samplesgithub.com/oracle-samplesThis repository contains example solutions and Terraform logic developed by groups such as the A-Team.
/openopensource.oracle.comThis provides a catalog of links to the various open-source repositories available. This includes oracle-sample and devrel but also the many other projects including, but not limited to Helidon, Fn, Verrrazano, GraalVM, Apiary
/communitycommunity.oracle.com/hubThis is where a lot of the Q&A for Oracle Partners and users takes place. Community participation is gamified to encourage participation.
academy.oracle.comThis is really for the educational community (Universities, Colleges & Schools) and provides resources to take you from zero to certified skills for Java and Oracle Database.
education.oracle.comPrimarily for the formal Oracle certification resources, but there are some free training resources here as well.
/apexapex.oracle.comThis provides a catalog of links to the various open-source repositories available. This includes oracle-sample and devrel but also many other projects including, but not limited to Helidon, Fn, Verrrazano, GraalVM, Apiary
/gh-oraclegithub.com/oracleThis is the home for the majority of all Oracle free code, including the Open source projects such as Helidon, JET, Kubernetes Operators, and so on.
Helidon, Fn, and other source projectsAPEX has its own area of dedicated resources. This includes solutions, templates, other development resources, and community.
/youtubewww.youtube.com/oracleYouTube training videos. With multiple channels based on different technologies.
Java (OpenJDK and Oracle JDK)A number of Oracle open-source projects have their own independent web resources as well. Helidon includes additional technical resources.
The ones we know more about are :
Helidon, Fn, Verrrazano, GraalVM, Apiary
But it includes references to Java core language etc.
/paasPaaS CommunityManaged by Jurgen Kress (Prod Mgr for Oracle PaaS). It acts as an aggregator for contributions from the community and shares news about what is happening within Oracle to support customers and partners in the PaaS space.
Developer CommunityPartner site to the PaaS Community with a focus on more development activities around Weblogic, Helidon, etc.
Tutorials etc., relating to the use of Linux and related Cloud Native things such as Verrazano, OCNE (Kubernetes), etc.LunaOracle propel not just their main JDK /JRE, which has license constraints, but also Open JDK. With this comes a dedicated Developer zone as dev.java.
Oracle PodcastsPerhaps not access usable as documentation, how-to, etc. but Podcasts can yield a lot of broad picture insights. Oracle has a range of podcasts covering a diverse range of subjects. Not all podcasts are active at any one time. But the site provides a catalog and episode list.

Share this:

  • Twitter
  • Facebook
  • LinkedIn
  • Print
  • Pocket
  • Email
  • Tumblr
  • Reddit
  • Pinterest
  • WhatsApp
  • Skype

Like this:

Like Loading...

blogs.oracle.com – last time for Capgemini

02 Saturday Apr 2022

Posted by mp3monster in General

≈ Leave a comment

A final blog that I co-wrote with Andy Bell whilst I was at Capgemini has gone live on blogs.oracle.com (here). Future posts will be as an Oracle employee.

This blog post follows up on a webinar that Capgemini did with Oracle and Flexagon reflecting on the application of FlexDeploy within Capgemini’s AIP offering.

Share this:

  • Twitter
  • Facebook
  • LinkedIn
  • Print
  • Pocket
  • Email
  • Tumblr
  • Reddit
  • Pinterest
  • WhatsApp
  • Skype

Like this:

Like Loading...

Aliases

  • phil-wilkins.uk
  • cloud-native.info
  • oracle.cloud-native.info

I work for Oracle, all opinions here are my own & do not necessarily reflect the views of Oracle

Oracle Ace Director Alumni

TOGAF 9

Logging in Action

Oracle Cloud Integration Book

API Platform Book


Oracle Dev Meetup London

Categories

  • App Ideas
  • Books
    • Book Reviews
    • manning
    • Oracle Press
    • Packt
  • Enterprise architecture
  • General
    • economy
    • LinkedIn
    • Website
  • Music
    • Music Resources
    • Music Reviews
  • Photography
  • Podcasts
  • Technology
    • APIs & microservices
    • chatbots
    • Cloud
    • Cloud Native
    • Dev Meetup
    • development
      • languages
        • node.js
    • drone
    • Fluentd
    • logsimulator
    • mindmap
    • OMESA
    • Oracle
      • API Platform CS
        • tools
      • Helidon
      • ITSO & OEAF
      • Java Cloud
      • NodeJS Cloud
      • OIC – ICS
      • Oracle Cloud Native
      • OUG
    • railroad diagrams
    • TOGAF
  • xxRetired

My Other Web Content & Contributions

  • Amazon Author entry
  • API Platform
  • Dev Meetup (co-managed)
  • Fluentd Book
  • ICS Book Website
  • OMESA
  • Ora World
  • Oracle Community Directory
  • Packt Author Bio
  • Phil on Blogs.Oracle.com
  • Sessionize Profile

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 2,573 other subscribers

RSS

RSS Feed RSS - Posts

RSS Feed RSS - Comments

April 2022
M T W T F S S
 123
45678910
11121314151617
18192021222324
252627282930  
« Mar   May »

Twitter

  • .@Oracle #CloudWorld Tour #London is just around the corner! ⏰ 🇬🇧 Don’t forget to register for the one-day #cloud… twitter.com/i/web/status/1…Next Tweet: 4 hours ago
  • Learn how @OracleCloud can help you improve the efficiency of your business operations at the upcoming Level Up eve… twitter.com/i/web/status/1…Next Tweet: 2 days ago
  • Join Juan Loaiza for the Data Strategies Day keynote at Level Up to learn how to eliminate complexity by leveraging… twitter.com/i/web/status/1…Next Tweet: 3 days ago
  • RT @WunderlichRd: Great post by @mp3monster around how APIs are relevant in so many industries! lnkd.in/eshagCDKNext Tweet: 3 days ago
  • King’s College Hospital London in Dubai announces a strategic collaboration with Oracle Cerner to help accelerate i… twitter.com/i/web/status/1…Next Tweet: 3 days ago
Follow @mp3monster

History

Speaker Recognition

Open Source Summit Speaker

Flickr Pics

Pembroke CastleSeven Bridge Crossing
More Photos

    Social

    • View @mp3monster’s profile on Twitter
    • View philwilkins’s profile on LinkedIn
    • View mp3monster’s profile on GitHub
    • View mp3monster’s profile on Flickr
    • View philmp3monster’s profile on Twitch
    Follow Phil (aka MP3Monster)'s Blog on WordPress.com

    Blog at WordPress.com.

    • Follow Following
      • Phil (aka MP3Monster)'s Blog
      • Join 218 other followers
      • Already have a WordPress.com account? Log in now.
      • Phil (aka MP3Monster)'s Blog
      • Customize
      • Follow Following
      • Sign up
      • Log in
      • Report this content
      • View site in Reader
      • Manage subscriptions
      • Collapse this bar
     

    Loading Comments...
     

    You must be logged in to post a comment.

      Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
      To find out more, including how to control cookies, see here: Our Cookie Policy
      %d bloggers like this: