Oracle IT Strategies and logging

20 Saturday Jun 2015

Posted by mp3monster in General, ITSO & OEAF, Oracle, Technology

Tags

accreditation, arcchitecture, autonomics, BCS, Chaos Monkey, ITSO, Kubernetes, monitoring, Netflix, Open Group, semantics, TOGAF, TRM

So I have an objective to get myself certified as an Oracle Technical Architect. Although the training is only open to Oracle and Partners, the exam is open to all. As you may have guessed from my blog posts I use a lot of Oracle technology. However the Technical Architect examination is based largely on Oracle’s IT Strategies library, and usually referred to as ITSO. Before non-Oracle users switch off, the ITSO is actually built around presenting solid good solution agnostic practises, and only once that is laid out does the material overlay Oracle products. So at least 75% percent of the material applies regardless of the vendor (yes cynics will say the practises will naturally lead you to products – but hey someone has to be bad guy). This actually makes it a worthwhile accreditation – as far as any accreditation can go (no I’ve not done a detailed comparison against Open Group’s Certified Architect – very expensive or the BCS accreditation – bound to BCS membership). TOGAF gives your framework, processes, means to communicate, and the ITSO does well at explaining the technical considerations and could be mapped onto the TOGAF Technical Reference Model (TRM) and Standards Information Base (SIB).

The point, I wanted to get across was in the ITSO is an element on Management and Monitoring (E16583-03 if you want the document reference on the Oracle Technology Network). It makes a lot of really good points about monitoring challenges such as bottom up approach where people monitor the parts of the full capability that they’re responsible for, rather than developing monitoring from a business perspective. The rationale for adopting the business based approach is explained (this is not to say you don’t go into the technical measures & monitors of looking at your infrastructure, databases, services etc. But from the business approach you will capture the information to understand reporting from a user perspective which is how you’ll here about issues. Through your detailed monitoring decomposition to get the right specific data points you can then look at correlation of monitoring data for root cause analysis, but also see and .

What the I think the document misses, or at least underemphasises is the ever increasing importance of the monitoring and logging of what is happening as systems and environments become ever more elastic and self managing, and have as IBM call it autonomics. or self healing, self scaling characteristics. So consider trying to diagnose a problem when a user complains of intermittent performance issues, but you have Kubernetes or another tool scaling up your environment for a period and then back down. Only through measuring from a business context will you able to understand when the user might perceive performance as an issue. Then with excellent logging and audit data as to what components are doing at all levels – so services maybe behaving perfectly but your scaling mechanisms are scaling back too soon.

This leads to another consideration, for those organisations that absolutely committed to idea of self healing and proving in resilience production, as the famous Netflix Chaos Monkey does. You need to be able to correlate the monkey’s activities to what is happening in your environment. Has the monkey uncovered an issue that manifests in a manner you hadn’t expected and as a result your user see intermittent issues.

This all leads me to a rather good presentation from Jimmi Dyson at RedHat who showed the simple value of ensuring you can get semantic meaning from logging. As that means you and slice and dice the information to get understanding of what is happening and lead to root cause. In Oracle land Oracle Enterprise Manage (OEM) is ensuring the semantic understanding when it come to known products.

I’ve meandered a bit, so key points consider ITSO or any other vendor equivalent for sources of good practise. Monitor and measure from a business perspective, but still ensure your collecting detailed semantically meaningful metrics.

Oracle SOA Components

20 Saturday Jun 2015

Posted by mp3monster in General, Oracle, Technology

≈ Leave a comment

Tags

Licensing, Oracle, products, SOA, SOA Suite

Understanding all the different elements available to you with Oracle SOA Suite can be a little tricky to say the least as the often the building blocks available are shown in diagrams like:

As shown in http://docs.oracle.com/cd/E28280_01/doc.1111/e10223/01_components.htm

They say a picture says a thousand words. But, those thousand words can omit some details. Take this diagram for example, it only reflects the common elements in the main SOA server. But within the SOA license you also have Oracle Service Bus (OSB) and the Complex Event Processing (CEP) engine which run as separate servers. Which means you’d see the following:

as shown in http://docs.oracle.com/cd/E28280_01/doc.1111/e10223/04_osb.htm

as shown in http://docs.oracle.com/cd/E28280_01/doc.1111/e10223/14_cep.htm

Oh, but hang on; we still haven’t got a representation of JMS, and Coherence. JMS like CEP and OSB actually instantiates as a separate server as well. So got a handle on everything now? Well there is a little confusion still needing to be added to the mix. The following diagram is commonly used showing interoperability with other products with separate licenses:

shown in http://docs.oracle.com/cd/E28280_01/doc.1111/e10223/507_eda.htm

Taking the above diagram at face value, you could interpret things as actually you don’t have the CEP capability as it is the key part of the Oracle Event Driven Architecture Suite.

I have merged together the diagrams to show the bulk of all the SOA stack that comes into the main SOA license. As you can see ….

Microservices in a COTs and SaaS world

15 Monday Jun 2015

Posted by mp3monster in General, Technology

≈ Leave a comment

Tags

API, enterprise, ESB, Microservices, MSA, SOA, SOI

Moving my recent blogs on Microservices (Microservices & UI, Microservices) forward a bit further as a result of discussing the ins and outs of using the paradigm. Microservices as the very name suggests is the polar opposite of most COTs and particularly ERP solutions which are pretty much modularised monoliths.

It raises the question of can Microservice Architecture (MSA) deliver any benefit in this situation where buy dominates over build. I believe the answer is to an extend yes. Many consider MSA to be SOA++ although I’m not sold on this MSA does exhibit what has been referred to as Service Oriented Integration (SOI) characteristics. That is the key is not the pure service ideas that you would get if you applied the recommendations of Thomas Erl.

The difference between SOI and SOA is that SOI focuses on things like interface contracts and pulling components together (regardless of whether they embody SOA ideals). Where as SOA focuses more on the business process and capability composition. How components are pulled together is an area where MSA has a strong position.

Where SOA and to an extent SOI would need an ESB (or ESB like) platform to perform the business rules and decisioning we should be keeping the intelligence out of the ESB. You will probably still want an ESB or event registration framework so that all services can register to receive events and react as necessary – I.e. Pure pub-sub model.

One of the SOA patterns for dealing with monoliths was to promote the idea of wrapping such services with a SOA abstraction tier so that you can replace the ERP, build out custom capabilities etc. does this hold true friends a MSA approach. I would suggest yes, but rather than the purity of SOA the abstraction should be aiming for the goals of SOI and simplification both in the ERP interaction, but also moving orchestration intelligence out of an ESB into the services. You can seen a Genesis of this potential with Oracle’s Cloud Adapters whose base framework aims to simplify the integration.

So what might be he benefit of building the Microservice layer? We know MSA exchanges code complexity in the service for agility in service delivery. But when there is a monolith behind the services do you gain anything? The answer is potentially, but will be very dependent on the monolith and ESB. For example if you can actually patch your monolith quickly and easily I.e it doesn’t have huge dependency chains and deployment capabilities such as Oracle EBZ 12.2 includes improved deployment framework that reduces or removes downtime. Like wise if the middleware is exploiting the best of SCA (as offered by Oracle SOA Suite) and or an OSGi container such as Apache Karaf then the benefits start to become more marginal. It becomes more a devil you know style of debate.

Microservice UI Positioning

14 Sunday Jun 2015

Posted by mp3monster in General, Technology

≈ 1 Comment

Tags

CSS, Design, devops, Microservices, Redhat, UI

Last week I was fortunate enough to attend RedHat’s day in London on Microservices. There were some great presentations and some ideas that are both simple and potentially very effective. It wasn’t a simple Microservices solves everything get into out tech stack, there was some reality checks as well.

The challenge I have been unable to square up yet, is the idea that each Microservices would have its own UI. On the surface, it makes a lot of sense, after all the UI needs to reflect the capabilities of the service.

The challenge comes in the form, that a User Interface needs to have consistency across the board. Yes, many will immediately point to CSS., which undeniably provide a level of consistency. But UI needs run a lot deeper. Let me point out a couple of illustrations of this:

Recent switch to web interfaces reflecting the new ‘flat’ visual format
Adoption of app on a page through AngularJS
Lots of illustrations can be seen at elegant Theme

This goes beyond CSS3 in many cases, but the libraries being used – so impacting development. Now here is the rub, the backend service functionality won’t change but the UI implementation will and needs to be deployed consistently across the board in one go for B2B and critically for B2C. You can destroy a good product with a poor UI and sell a rubbish app with a good one. All of which would mean deploying updated all Microservices at once if they embody the UI. The linking of all the Microservices like this is completely contrary to the goals of agility driven the Microservices strategy.

Add to this, the Microservices approach promotes a DevOps approach, yet organisations may only employ 1 or 2 real user experience specialists rather than try and spread them across multiple service teams it maybe better to focus them into one or two service teams that just build the UI.

Which kind of leads me to the argument that I would suggest that your UI is a separate service or small group of services to the core functional side of things. So those PR driven website overhauls, and revisions to match user experience expectations can be done without impacting the core capabilities, demanding a total regression test and locking your entire set of services into a unified release cycle.

Citizen Integrators – Excel for Integration?

10 Wednesday Jun 2015

Posted by mp3monster in General, Technology

≈ 1 Comment

Tags

bimodal, Boomi, Citizen Integrator, enterprise, Excel, Gartner, integration, iTunes, Jason Bloomberg, OIC - ICS, Oracle

Over the last year or so I have been looking a lot at technologies that Gartner and others have branded as ‘Citizen Integrators‘ – products such as Dell Boomi, and the recently launched Oracle Integration Cloud Service. What I believe we are seeing is the appearance of a product family that in many respects will be to Integration what Excel has been to Finance systems. This is to say that Finance Systems such as large ERPs tend to be changed slowly when it comes to introducing process changes, but users can get reports easily to extract data into their Excel spreadsheets. We have the old joke that organisations finance can end up being run on Excel (http://www.wired.com/2014/03/many-spreadsheets-take-run-fortune-500-company/).

So don’t get me wrong, I’m not saying these tool are evil and should be banned or the such like, as such thinking is utter folly. I am looking at the quote much attributed to Spiderman (Stan Lee) but has been traced back to Voltaire:

With great power comes great responsibility

Why do I use this quote, well my experience (and that established by many others) is that with ease and agility comes a quick answer rather than a well thought out answer. That ease can be through cost (how many times have organisations discovered key systems solutions being run off someone’s desktop stuffed away in the corner of an office because they have been able to cheaply acquire the hardware and software get setup and then had viral adoption).

It is therefore beholden on those of us that understand the challenges of integration should be seeking to help our ‘citizens’ appreciate (not lecture, brow beat etc) the implications and some intelligent governance to ensure systems are not accidentally ‘poisoned with unexpected data’ and you don’t fall foul of legal obligations.

The biggest challenge, is for SME’s to ensure that their colleagues within the IT organisation who face into the business organisation understand and promote the right thinking. After all, developers and architects alike, think like all drivers -that they’re at least above average if not good drivers – after all why would we be in the job? But to set the average we can’t all be in that place (http://www.psychologicalscience.org/index.php/news/motr/when-it-comes-to-driving-most-people-think-their-skills-are-above-average.html).

This of course also touches upon the arguments with Gartner’s bi-modal approach to IT, such as those presented by Jason Bloomberg. Personally I believe pace layering is right, but bi-modal thinking can create opportunities for things to be done badly – not an absolute certainty, but to work needs some strong hands trusted by organisational executives to steer successfully – something that seems rather rare.

Do we really need a an app for everything?

07 Sunday Jun 2015

Posted by mp3monster in General, Technology

≈ Leave a comment

Tags

android, apps, download, ios

It is fascinating despite the fact that HTML5 and CSS3 give us very good user experience, when it comes solutions that largely push or marketing channels where caching has limited benefit there is still the ever growing perceived need to produce mobile apps. There are some great images which compares Facebook as an app vs Facebook as a web app.

So, if the labels didn’t give the game away then, you’d be looking for the tell tail signs of the browser – icons at the bottom of the page.
So the question begs, why build the mobile app and the mobile capable web app? The argument that the web app requires less traffic as you’re not downloading all he graphics has only limited credence as you have to pull all of this down with the app to start with. The browser will cache a lot of the content if you let it. Good web apps will make use of AJAX or Sockets to ensure only necessary content is retrieved so your download is going to be limited. The offline argument doesn’t work as you’re trying to provide current content.

To be honest, I think a lot of the answer comes down to a couple of simple factors:

is rather simple, a mobile app will find it easier to retrieve more information about the user
There is the mentality of I’m not a ‘digital’ player if I don’t have an app, and who isn’t going to challenge this?

So should we be pushing for OS providers to make it easier to install launch icons from your device (it is do able already but not in your face obvious). But also allow web apps that are designed to work well on mobile devices to be certified and listed within their stores. Perhaps even offer a configuration shell that setups a desktop link as if an app has been downloaded.

Doing so should take the burden off the app stores! may not benefit beyond a small download bandwidth, but has to be good news for Apple and others.

Microservices

07 Sunday Jun 2015

Posted by mp3monster in General, Technology

≈ 1 Comment

Tags

AMQ, Apache, camel, karaf, Microservices, Oracle, OSB, Redhat, SOA Suite

Microservices are a hot topic at present. But microservices is neither a standard or a specific technology. Like REST it is more a set of ideas. So what constitutes a microservices. The best description I have come across yet has been by Martin Fowler ( http://martinfowler.com/articles/microservices.html ).

We can focus down on a number of specific points that are central to the idea of Microservices:

the creation of small pieces of functionality that can be discretely deployed,
are connected typically by web APIs often using REST (but also seen using other abstracting protocols)
can be replaced with minimal dependency issues
microservices are typically built by small discrete teams usually in the range of 2-12 people (the so called 2 pizza rule)
services are usually orchestrated by dumb pipes (so publication/subscription strategies are often used, so the intelligence about how and what to do about each event is within the service not the orchestration).
design approach changes orientation from n-tier (presentation, orchestration/business logic, persistence) which could be described as horizontal separation to vertical separation where partitioning is functional/service centric (which internally may embody the horizontal partitioning but this is secondary and down to how the service delivery team wish to work).
Search service us running as their own CPU process – typically using container technologies such as Docker, Rocket, Spoon and Drawbridge
Any orchestration is dumb, the decisions of what to do and when to participate are taken by the service

The small container footprint (making the enforcement of the decoupling with minimal governance) means density of processes can remain high as the overhead compared to full VMs is a lot smaller but also means instantiating clean environments for fresh deployments and testing is very fast. This does not fit so well within many ESB environments such as Oracle’s SOA Suite as the pre-requisites create a substantial footprint that would need to reside within the container for the ESB (RedHat’s JBoss Fuse is one of the few exceptions if you consider the required footprint for Apache Camel for example).

However, some of the microcontainer principles can be pursued within the larger ESB environments utilising capabilities such as :

Service Component Architecture (SCA) provides a means to create isolated versions of solutions that can run concurrently. By exploiting proper versioning and version dependency controls you can start pushing out different solution pieces with great ease.
Exposing composites via we services REST or WSDL based and adopt a more SOI approach to artefacts so don’t tap into DVMs directly use web services to perform the lookups
Microservice implementations have a number of NFRs characteristics that are not (atleast in my exerpience) often utilised when rich ESB frameworks such as
- service compensation http://soapatterns.org/design_patterns/compensating_service_transaction
- standard implementation of Tolerant Reader patterns – http://servicedesignpatterns.com/WebServiceEvolution/TolerantReader (in conjunction with versioning patterns such as canonical versioning – http://soapatterns.org/design_patterns/canonical_versioning)

These approaches allow you adopt the dumb pipe approach (you don’t want services directly invoking each other except in case of utility services otherwise a lot of inter service dependency will build up). Using a publish & scribe framework or simple service sequencing we should be able to exploit OSB, Weblogic MQ in an Oracle Context and Weblogic as an OSGI container (for discovering technical services). In line with the Microservices ethos it would more than legitimate to build Microservices with other tools and then use an ESB like SOA Suite to provide the technology for weaving the services together.

In a Redhat product set there are more options as the solution footprints are smaller. But you would consider Karaf (OSGi container), Active MQ,and simple uses of Camel to weave microservices together.

With cloud middleware, adopting the goals of microservices will become easier as instantiating fresh environments and deployment approaches will become more akin to those of containers – for example Oracle Integration Cloud Service (ICS) deployment is simply an import of a whole set of configuration and integration process information.

It should be noted that Microservices does fit better with a number of organisational and management approaches, such as:

dev ops – the build team carry the role of operational support
product centric rather than project centric life cycles i.e. the team exists as long as the product, rather than existing until all the current funded features are complete
works for build rather than buy delivery (buy is likely to introduce artefacts too large for a Microservice model).

Each microservice is likely to contain its own copy of data – potentially leading to greater data duplication – therefore data reconciliation checks and management thinking maybe be needed.