While there is a deserved amount of publicity around the introduction of ARM compute onto OCI with the ARM Ampere CPU offering, and the amazing level of always free compute being provided (24GB of memory and 4 cores which can be used in any combination of servers). There have been some interesting announcements that perhaps haven’t drawn as much attention that they deserve. This includes OCI support for GitHub Actions, plus several new DevOps services and an Artifact Registry. We’ll comeback to the new services in another post. Today, let’s look at GitHub Actions.

Support for GitHub Actions

GitHub recently introduced a new feature called GitHub Actions. In essence it has become possible to establish a basic event driven CI/CD pipeline driven directly by events in GitHub. The Actions (sometimes also referred to as workflows) are executed by what are known as GitHub Runners, if you’re familiar with Jenkins, then you could think of these as slave nodes. The smart thing is that you can either use GitHub provided (aka GitHub Hosted) compute to execute the actions (effectively a bit of Azure) and pay for the service to execute the actions. If you’re using GitHub professionally or setup your own workers in other clouds or even potentially on-premise, these are known as self hosted GitHub runners. The other clouds includes Oracle who have provided a configuration that can build runner nodes (and yes you can use the Ampere free compute).

Launching a OCI Always Free GitHub Runner

There are a number of ways to get a runner working both using paid VMs or using the always free capacity.

For someone who produces utilities and makes them freely available; and wants to run at least unit tests on code and code checks. The ability to automatically run the unit tests is particularly handy. When extending an existing piece of functionality I tend to focus on testing that bit of code, this means I don’t need to rerun lots of tests, I can set the actions up to repeat those tests as I make changes and if I break something then I’ll know immediately. So combining the free compute with the ability to auto execute tests on a commit without needing to run my own dedicated Jenkins server is perfect.

The compute can be setup either by using the scripts generated by the GitHub UI for setting up runners, which can be seen when you navigate to the Runner configuration (settings menu at the stop of the page, and Runners on the left menu) as highlighted here with 1 & 2

We need to set the Operating System and Architecture for the runner (3), this will impact the binaries needed, and the code offered in the centre of the screen (4). This does mean that you need to run these scripts and ensure that the processes start and stop safely within the VM. The alternative in Oracle cloud is to use predefined image using a link like this (https://cloud.oracle.com/resourcemanager/stacks/create?zipUrl=https://github.com/oracle-quickstart/oci-github-actions-runner/releases/download/orm-deploy/orm.zip). As you can see from the URL itself, it is passing to the OCI a stack that has been already created (and made freely available in GitHub).

The library of Oracle quick start configurations can be found at https://github.com/oracle-quickstart

Following the URL will take us into the Resource Manager which presents a multi paged form capturing the relevant details that will enable it to generate the server and required supporting infrastructure in the correct compartment. The first step is to acknowledge an constraints, licensing conditions that the package imposes as highlighted with the red square.

Once complete we casn move onto the next page. We can see the example screens below the A1 (Ampere) CPU being selected using up 4 GB of RAM and 1 OCPU of the 4 available as free.

Aside from choosing the compute shape, I’ve selected a version of the Oracle Linux v8 suitable for the processor type.

You’ll not, you will also be promoted for the GitHub details, with your repository to be supported, and the associated token, which comes from the presented script values in the initial Runner setup, we highlighted the value in the first diagram as (5) and should correlate to (A) in the details below.

There is a gotchya to be careful of. When we put our repository into a browser URL it really isn’t bothered by the terminating slash or not in the name, but as shown below as (B) we have included the terminating slash. The authentication of communication is very sensitive to such details. People have reported that they see issues with the Runner being connected to GitHub, but all executions of actions fail (when we saw this within 1-2 seconds) and when you try to drill in, there is no information. To confirm the issue, you need to SSH into the runner server and navigate into the /actions-runner/_diags folder to see if things being reported in the log files are showing as going awry.

Once the runner is deployed, then it will show up after all while in the list of runners. Note in this screenshot we have a fresh runner having experienced the previously described connectivity issue. To see the runners defined, again we go through Settings (1) on the top menu and Actions (2) on the left. Once the runner has made successful contact then it will show in the list of runners.

Runner’s Job

With the runner ready we need to create a Job. This bit is a little tricky. You can develop from scratch your own Job definitions, but it will require some effort in mastering the YAML notation. The alternative is to see if there is a prepared job definition. It is possible to use pre-existing templates by clicking on the New workflow button as highlighted below. This will take you the catalogue of predefined flows.

The following image shows the predefined list of workflows. GitHub will make a recommendation based on its analysis of the repository, in this case it has filtered out to show Python workflows. But you’re not bound to these, this is just GitHub trying to simplify the effort involved.

To get a job definition started, from the workflow templates, select the Set up this workflow button. This will result in a copy of the YAML configuration file being put into a folder within the project as you can see in the next image.

Looking more closely at the YAML file describing the action, we see the following, which describes when the action be executed, in this case a push or pull on the main branch.

The default example is geared to Ubuntu which brings us to one the niggles we’ve encountered. The job will try to setup the environment for us.

GitHub Runner Constraint

You’ll note that there is a step that uses python-version: ${{ matrix.python-version }} – this is going to try setup Python for the three versions listed in the matrix list defined. That doesn’t sound problematic until you dig into the documentation provided by the link https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions. This web page describes how Python works with GitHub Runners. For a Python on a hosted GitHub runner you have to do things yourself. Fortunately the VM configuration provided mean Python 3.6 is installed that will work on the ARM architecture. But we do need to remove a chunk of logic that sets up the runner. Incidentally, if you look at the link to the supported Python versions, they work for both Windows and Linux but only on an x86 CPU.

We do still want to retain the use of PIP to ensure that latest versions of the packages are always in use.

# This workflow will install Python dependencies, run tests and lint with a variety of Python versions
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions

name: Python package

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

jobs:
  build:

    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        python-version: [3.7, 3.8, 3.9]

    steps:
    - uses: actions/checkout@v2
    - name: Set up Python ${{ matrix.python-version }}
      uses: actions/setup-python@v2
      with:
        python-version: ${{ matrix.python-version }}
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        python -m pip install flake8 pytest
        if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
    - name: Lint with flake8
      run: |
        # stop the build if there are Python syntax errors or undefined names
        flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
        # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
        flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
    - name: Test with pytest
      run: |
        pytest

Having removed the Python setup configuration, we need to ensure the utilities Flake8 and PyTest can be called either by modifying the PATH variable or tweaking the way the utilities are invoked. Rather than messing with the VM OOTB configuration or writing a script that is pulled onto the runner to perform further setup I’ve opted to adjust the call. These changes can be seen at https://github.com/mp3monster/oci-utilities/blob/main/.github/workflows/python-package.yml, and …

# This workflow will install Python dependencies, run tests and lint with a variety of Python versions
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions

name: Python package

on:
  push:
    branches: [ main ]


jobs:
  build:

    runs-on: oci
    strategy:
      fail-fast: false


    steps:
    - uses: actions/checkout@v2
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        python -m pip install flake8 pytest
        if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
    - name: Lint with flake8
      run: |
        # stop the build if there are Python syntax errors or undefined names
        python -m flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
        # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
        python -m flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
    - name: Test with pytest
      run: |
        python -m pytest

We’ve also made use of the requirements.txt to ensure the OCI Python SDK is installed for use. We now have an action that performs code analysis with Flake8 (details also here) which will spot any obvious coding errors and pytest which includes the ability to locate and run the native Python unittest based tests. All the results are then bundled up and returned back to GitHub.

By clicking on the build we can review all of the steps the run takes and the results of flake8 and pytest for example.

Possible Risk

GitHub’s documentation raises the risk of connecting Actions to public repositories. This comes down a couple of things. Firstly, the actions of public users could depending upon your configuration trigger the workflow (which is why we’ve removed the pull request from the triggers). Within GitHub you can limit on a public project who can push, in this case, we’ve put the maximum limits into the Settings, so only approved collaborators are allowed to perform any of the approved operations like fork etc.

Whilst this reduces the opportunities to keep triggering the workflow – consuming your resources, which may cost but also could be a form of denial of service by the fact it will be disrupting your genuine worker runs.

There are a couple of other mitigations that could be applied to further tighten this up:

• schedule the Runner so that it is shutdown and started up only when people are likely to be working on the code base
• Attach the job not to a public view of the code, but a private branch. All development is against the private branch and then, a successful result triggers the runner to merge the changes into the publicly visible branch.

Conclusion

Knowing the limitations, and the syntax of the workloads not looking difficult to master, we’ll probably develop more unit tests and deploy them. Plus also go back to the Groovy stuff as well. The big test will be how easy it is to reconstruct a container image of the Log Simulator.

Useful Links

• https://blogs.oracle.com/cloud-infrastructure/post/announcing-github-actions-arm-runners-for-the-arm-compute-platform-on-oracle-cloud-infrastructure
• https://docs.github.com/en/actions/hosting-your-own-runners/adding-self-hosted-runners
• Flake8 – package information here, and documentation here.