• Home
  • Site Aliases
    • www.cloud-native.info
    • oracle.cloud-native.info
    • Phil-Wilkins.uk
  • About
    • Background
    • Presenting Activities
    • Internet Profile
      • LinkedIn
    • About
  • Books & Publications
    • Logging in Action with Fluentd, Kubernetes and More
      • Logging in Action with Fluentd – Book
      • Fluentd Book Resources
      • Log Generator
    • API & API Platform
      • API Useful Resources
    • Oracle Integration
      • Book Website
      • Useful Reading Sources
    • Publication Contributions
  • Resources
    • GitHub
    • Oracle Integration Site
    • Oracle Resources
    • Mindmaps Index
    • Useful Tech Resources
    • Python Setup & related stuff
  • Music
    • Music Reading

Phil (aka MP3Monster)'s Blog

~ from Technology to Music

Phil (aka MP3Monster)'s Blog

Tag Archives: OKE

Container Registry – pushing and storing containers

12 Thursday May 2022

Posted by mp3monster in Cloud Native, General, Oracle, Oracle Cloud Native, Technology

≈ 2 Comments

Tags

container, development, Docker, Kubernetes, OCIR, OKE, Oracle, registry, Technology

A container registry is as essential as a Kubernetes service as you want to manage the deployable resources. That registry could be the public Docker repository or something else. In most people’s cases, the registry needs to be private as you don’t want to expose your product assets to potential external tampering. As a result, we need a service such as Oracle’s container registry OCIR.

The re of this blog is going to walk through how to push a container you’ve built into OCIR and a gotcha that can trip up users if you make assumptions about how the registry works.

Build container

Let’s assume you’re building your microservices locally or retrieving vetting 3rd party services for use. In both cases, you want to manually push your assets into OCIR manually rather than have an automated build pipeline do it for you.

To make it easier to see what is happening, we can exploit some code from Oracle’s Github repo (such as this piece being developed) or you could use the classic hello world container (https://github.com/whotutorials/docker-busybox-hello-world/blob/master/Dockerfile). For the rest of the post, we’ll assume it is the code developed for the Oracle Architecture Center-provided code.

docker build -t event-data-svc .

This creates a container locally, and we can see the container listed using the command:

docker images

Setup of OCIR

We need an OCIR to target so the easiest thing is to manually create an OCIR instance in one of the regions, for the sake of this illustration we’ll use Ashburn (short code is IAD). To help with the visibility we can put the registry in a separate compartment as a child of the root. Let’s assume we’re going to call the registry GraphQL. So before creating your OCIR set up the compartment as necessary.

fragment of the compartment hierarchy

In the screenshot, you can see I’ve created a registry, which is very quick and easy in the UI (in the menu it’s in the Developer Services section).

The Oracle meu to navigate to the OCIR service
the UI to create a OCIR

Finally, we click on the button to create the specific OCIR.

Deployment…

Having created the image, and with a repo ready we can start the steps of pushing the container to OCIR.

The next step is to tag the created image. This has to be done carefully as the tag needs to reflect where the image is going using the formula <region name>/<tenancy name/<registry name>:<version>. All the registries will be addressed by <region short code>.ocir.io In our case, it would be iad.ocir.io.

docker tag graph-svr:latest iad.ocir.io/ociobenablement/graphql-svr:v0.1-dev

As you may have realized the tag being applied effectively tells OCI which instance of OCIR to place the container in. Getting this wrong can be the core of the gotcha previously mentioned and we’ll elaborate upon it shortly.

To sign in you’ll need an auth token as that is passed as the password. For simplicity, I’ve passed the token in the docker command, which Docker will warn you of as being insecure, and suggest it is passed in as part of a prompt. Note my token will have been changed by the time this is published. The username is built on the structure of <cloud tenancy name>/identitycloudservice/<username>. The identitycloudservice piece only needs to be included for your authentication is managed through IDCS, as is the case here. The final bit is the URI for the appropriate regional OCIR address, as we’ve used previously.

docker login -u ociobenablement/identitycloudservice/philip.wilkins@oracle.com -p XXXXXXXXXXX  iad.ocir.io

With hopefully a successful authentication response we can push the container. It is worth noting that the Docker authenticated connection will timeout which is why we’ve put everything in place before connecting. The push command is very simple, it is the tag name assigned to the artifact including the version number.

docker push iad.ocir.io/ociobenablement/graphql/graph-svr:v0.1-dev
OCIR with several versions of a container

Avoiding the gotcha

When we deal with repositories from Git to SVN or Apache Archiva to Nexus we work with a repository that holds multiple different assets with multiple versions of those assets. as a result, when we identify an asset uniquely we would expect to name things based on server/location, repository, asset name, and version. However, here each repository is designed for one type of asset but multiple versions. In reality, a Docker repository works in the same manner (but the extended path impact is different).

This means it becomes easy to accidentally define a tag with an extra element. Depending upon your OCI tenancy privileges if you get the path wrong, OCI creates a new root compartment container repository with a name that is a composite of the name elements after the tenancy and puts your artifact in that repository, not the one you expected.

We can address this in several ways, first and probably the best option is to automate the process of loading assets into OCIR, once the process is correct, it will remain correct. Another is to adopt a principle of never holding repositories at the root of a tenancy, which means you can then explicitly remove the permissions to create repositories in that compartment (you’ll need to explicitly grant the permissions elsewhere in the compartment hierarchy because of policy inheritance. This will result in the process of pushing a container to fail because of privileges if the tag is wrong.

Visual representation of structure differences

Repository Structure
Registry Structure

Condensed to a simple script

These steps can be condensed to a simple platform neutral script as follows:

docker build -t event-data-svc .
docker tag event-data-svc:latest iad.ocir.io/ociobenablement/event-data-svc:latest

docker login -u ociobenablement/identitycloudservice/philip.wilkins@oracle.com -p XXXXX  iad.ocir.io
docker push iad.ocir.io/ociobenablement/event-data-svc:latest

This script would need modifying for each container being built, but you could easily make it parameterized or configuration drive.

A Note on Registry Standards

Oracle’s Container Registry has adopted the Open Registries standard for OCIR. Open Registries come under the Linux Foundation‘s governance. This standard has been adopted by all the major hyperscalers (Google, AWS, Azure, etc). All the technical spec information for the standard is published through GitHub rather than the main website.

References

  • Push an Image to Oracle Cloud Infrastructure Registry
  • Notes about Repository Creation
  • Creating a Container Registry
  • Open Registries
  • Policies to Control Repository Access

Share this:

  • Twitter
  • Facebook
  • LinkedIn
  • Print
  • Pocket
  • Email
  • Tumblr
  • Reddit
  • Pinterest
  • WhatsApp
  • Skype

Like this:

Like Loading...

Mastering FluentD configuration syntax

19 Thursday Sep 2019

Posted by mp3monster in Cloud, Fluentd, General, Technology

≈ Leave a comment

Tags

configuration, Fluentbit, Fluentd, google, GPC, monitoring, observability, OKE, slack

Getting to grips with FluentD configuration which describes how to handle logging event(s) it has to process can be a little odd (at least in my opinion) until you appreciate a couple of foundation points, at which point things start to click, and then you’ll find it pretty easy to understand.

It would be hugely helpful if the online documentation provided some of the points I’ll highlight upfront rather than throwing you into a simple example, which tells you about the configuration but doesn’t elaborate as deeply as may be worthwhile. Of course, that viewpoint may be born from the fact I have reviewed so many books I’ve come to expect things a certain way.

But before I highlight what I think are the key points of understanding, let me make the case getting to grips with FluentD.

Why master FluentD?

FluentD’s purpose is to allow you to take log events from many resources and filter, transform and route logging events to the necessary endpoints. Whilst is forms part of a standard Kubernetes deployment (such as that provided by Oracle and Azure for example) it can also support monolithic environments just as easily with connections working with common log formats and frameworks. You could view it as effectively a lightweight (particularly if you use FluentBit variant which is effectively a pared-back implementation) middleware for logging.

If this isn’t sufficient to convince you, if Google searches are a reflection of adoption, then my previous post reflecting upon Observability -London Oracle Developer Meetup shows a plot reflecting the steady growth.  This is before taking into account that a number of cloud vendors have wrapped Fluentd/fluentbit into their wider capabilities such as Google (see here).

Not only can you see it as middleware for logging it can also have custom processes and adapters built through the use of Ruby Gems, making it very extensible.

FluentD

Remember these points

and mastering the config should be a lot easier…

Continue reading →

Share this:

  • Twitter
  • Facebook
  • LinkedIn
  • Print
  • Pocket
  • Email
  • Tumblr
  • Reddit
  • Pinterest
  • WhatsApp
  • Skype

Like this:

Like Loading...

Observability -London Oracle Developer Meetup

10 Tuesday Sep 2019

Posted by mp3monster in Dev Meetup, Fluentd, General, Technology

≈ 1 Comment

Tags

Fluentd, Istio, Jaeger, Kiali, logging, meetup, monitoring, observability, OKE, OpenTracing, Oracle, Tracing

meetup-monitoringLast night was the London Oracle Developer Meetup’s sessions around observability.  Andrei Cioaca with a focus on the use of OpenTracing as provided by Jaeger, in a standard Kubernetes deployment with Istio – realized with Oracle Kubernetes Engine (OKE).  This was followed by my session on another pillar using logging via FluentD. Also incorporated into standard Kubernetes, but also able to support traditional monolithic use cases.

@andreicioaca starts talking about Oracle #Kubernetes #OKE #istio and #OpenTracing at the #OracleDeveloperMeetup #London @PaaSCommunity pic.twitter.com/HISzpmxjaN

— Phil Wilkins (@PhilConsultant) September 9, 2019

Andrei provided a great overview of the 3 pillars and the strengths and weaknesses of the different pillars. With the basics covered Andrei then dove into the configuration and execution of Istio combined with Jaeger and the corresponding insights available.  including a look at the kinds of visual insights that Jaeger and Kiali provide.  Some probing conversations followed about the relationship to Spring Cloud Sleuth, Open Zipkin and the OpenTracing as a concept more generally.

Andrei’s presentation material can be found in his GitHub repository here.

search-trend-fluentd

Google Analytics on Search Terms

My session followed a pizza break, as there was a delay in its arrival. With everybody having chatted over pizza about OpenTracing, we picked up on FluentD and the Logging aspect to Observability. FluentD, as an open-source project has been growing steadily, and actually baked into several Log Analytics products and services – as the above analytics from Google shows.

The presentation looked at the growing challenges of modern software in terms of making sense of logging.  We explored the capabilities of FluentD before drilling into real-world use cases and potential deployment models.

As you’ll see from the slides we ran a couple of demos. The configuration for the demos can be found at https://github.com/mp3monster/fluentd-demos along with an example payload.

The next meetup we have organized is around Blockchain, all the details can be found at https://www.meetup.com/Oracle-Developer-Meetup-London/events/264661742/.

Other related info …

  • Mastering Distributed Tracing – book review
  • Article direct to LinkedIn – OpenTracing and API Gateways

Share this:

  • Twitter
  • Facebook
  • LinkedIn
  • Print
  • Pocket
  • Email
  • Tumblr
  • Reddit
  • Pinterest
  • WhatsApp
  • Skype

Like this:

Like Loading...

Aliases

  • phil-wilkins.uk
  • cloud-native.info
  • oracle.cloud-native.info

I work for Oracle, all opinions here are my own & do not necessarily reflect the views of Oracle

Oracle Ace Director Alumni

TOGAF 9

Logging in Action

Oracle Cloud Integration Book

API Platform Book


Oracle Dev Meetup London

Categories

  • App Ideas
  • Books
    • Book Reviews
    • manning
    • Oracle Press
    • Packt
  • Enterprise architecture
  • General
    • economy
    • LinkedIn
    • Website
  • Music
    • Music Resources
    • Music Reviews
  • Photography
  • Podcasts
  • Technology
    • APIs & microservices
    • chatbots
    • Cloud
    • Cloud Native
    • Dev Meetup
    • development
      • languages
        • node.js
    • drone
    • Fluentd
    • logsimulator
    • mindmap
    • OMESA
    • Oracle
      • API Platform CS
        • tools
      • Helidon
      • ITSO & OEAF
      • Java Cloud
      • NodeJS Cloud
      • OIC – ICS
      • Oracle Cloud Native
      • OUG
    • railroad diagrams
    • TOGAF
  • xxRetired

My Other Web Content & Contributions

  • Amazon Author entry
  • API Platform
  • Dev Meetup (co-managed)
  • Fluentd Book
  • ICS Book Website
  • OMESA
  • Ora World
  • Oracle Community Directory
  • Packt Author Bio
  • Phil on Blogs.Oracle.com
  • Sessionize Profile

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 2,574 other subscribers

RSS

RSS Feed RSS - Posts

RSS Feed RSS - Comments

March 2023
M T W T F S S
 12345
6789101112
13141516171819
20212223242526
2728293031  
« Feb    

Twitter

  • Get all the details about the new enhancements to @Oracle Container Engine for Kubernetes, including Serverless… twitter.com/i/web/status/1…Next Tweet: 6 hours ago
  • RT @TechWeekRO: With over 25 years of experience in the software industry, Phil Wilkins, Cloud Developer Evangelist at @Oracle, is coming t…Next Tweet: 8 hours ago
  • SSH Key File Permissions blog.mp3monster.org/2023/03/28/ssh…Next Tweet: 23 hours ago
  • Oracle's Assurance Service gives customers the proactive guidance they need to move their organization forward whil… twitter.com/i/web/status/1…Next Tweet: 1 day ago
  • Fraud affects many businesses and can be costly. But there’s a way to fight it. Scalable Machine Learning algorithm… twitter.com/i/web/status/1…Next Tweet: 1 day ago
Follow @mp3monster

History

Speaker Recognition

Open Source Summit Speaker

Flickr Pics

Pembroke CastleSeven Bridge Crossing
More Photos

    Social

    • View @mp3monster’s profile on Twitter
    • View philwilkins’s profile on LinkedIn
    • View mp3monster’s profile on GitHub
    • View mp3monster’s profile on Flickr
    • View philmp3monster’s profile on Twitch
    Follow Phil (aka MP3Monster)'s Blog on WordPress.com

    Blog at WordPress.com.

    • Follow Following
      • Phil (aka MP3Monster)'s Blog
      • Join 218 other followers
      • Already have a WordPress.com account? Log in now.
      • Phil (aka MP3Monster)'s Blog
      • Customize
      • Follow Following
      • Sign up
      • Log in
      • Report this content
      • View site in Reader
      • Manage subscriptions
      • Collapse this bar
     

    Loading Comments...
     

    You must be logged in to post a comment.

      Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
      To find out more, including how to control cookies, see here: Our Cookie Policy
      %d bloggers like this: