Phil’s Intro to Python Setup and other related stuff

Background

Python is a secondary language to me, so I don’t setup new projects and environments on a regular basis. These notes, are first and foremost an aide memoir, but I’ve also elaborated on why certain things are being done, this is less for my benefit but the newbie who might find this info helpful. But feel free to use the information as you wish. These notes are focused on a Windows based developer platform as most devs I know work with Python locally.

Windows Package Manager

Windows’ official package manager that operates like yum, apt etc. is still in the ‘works’ in the mean time its worth getting Chocolatey setup. As this make some install activities a lot easier. Details for this are located at – https://chocolatey.org/install.

One Python to rule them all

To get started and to exploit all the features and tooling we need an initial version of Python installed. Recommendation is regardless of the version you need to use for a specific use case – go for the latest. The downloads can be retrieved from https://www.python.org/downloads/. If you follow the links – you’ll find an exe installer which will go through setting up the path etc for you.

Python Package PIP installer

If you’re working with an older version of Python pre 2.7.9 or 3.4 then you’ll need to install the package manager – known as PIP manually – details at https://pip.pypa.io/en/stable/installing/

IDE

I’ve found that Microsoft Visual Studio Code to be a really good Python IDE – lightweight but well integrated. This can be installed from https://code.visualstudio.com/

Once installed you will need to install a set of plugins, we’ve been using:

TitleURLReason
Better Comments orhttps://marketplace.visualstudio.com/items?itemName=aaron-bond.better-commentsHelps with the visual formatting, particularly for comments
Auto Doc Stringhttps://marketplace.visualstudio.com/items?itemName=njpwerner.autodocstringAlternative to Better Comments – I’ve found myself preferring this plugin
Code Spell Checkerhttps://marketplace.visualstudio.com/items?itemName=streetsidesoftware.code-spell-checkerGet the code and comment spelling correct
Markdown Preview Enhancedhttps://marketplace.visualstudio.com/items?itemName=shd101wyy.markdown-preview-enhancedFor markdown documentation within VS Code (although, anything chunkier I opt for Typora)
Pythonms-python.python
SonarLintsonarsource.sonarlint-vscodeCode cleansing and code quality support

Virtual Environments to avoid ‘DLL Hell’

Python like just about every language needs a means to address the problem of ‘DLL Hell‘ (the challenge of dependencies and versions of the language). This can be overcome using PyEnv and virtualenv as a command line capability. Details of managing it VS Code can be found here which makes use of Conda. (Conda vs PyEnv info here) A colleague of mine also pointed out that PyCharm (part of the JetBrains family of IDEs has a graceful means to manage these challenges as well, explanation here, thanks @Nagesh).

Using correct Python version – pyenv

Eventually you’ll be working on projects that need or only support specific versions of Python, given going back to Python 2 (whilst End of Life) still used in places. As a result a util to make it easy to switch between different version of Python is essential. This is what Pyenv does. Details for installation are at – https://github.com/pyenv-win/pyenv-win#installation but with Chocolatey setup – the simplest solution is:

choco install pyenv-win

Once Pyenv is installed you can then use it to install the other versions of Python that you may need.

Configuration Management

You also need a config repository manager, if you’re using GIT then the recommendations are:

Unit Testing

We need Unit Testing capabilities, the simplest option here is to utilize the Python native capabilities. So no additional installation work.

Code Annotation and Doc Generation

Documentation relating to how code works is best when it is inline with the code rather than separate documents as this creates the best chance of the documentation actually reflecting the implementation.

The best way for documentation to be provided in Python is to use restructured (aka reST). There are a range of tools that can work with the reST approach to docs. The reST model works a lot like markdown for most part but not entirely.

With the code documented e.g. a function description:

def find(name=None, query_type=USER, query_msg="", print_find = False):

    """

    finds xxxx

    **Parameters**`
    * name : xx
    * query_type : xx 
    * query_msg : xx
    * print_find : xx

    **Returns**

      OCID if a single entity is located. If multiple entities are found this is a list of OCIDs
"""

We would recommend pdoc which can be installed by:

pip install pdoc

This will install the tool. Then to run the tool:

pdoc <pythpon file>

Running pdoc will launch a local webserver and browser page is opened accordingly. With it is an HTML page which dynamically updates inline with editing the document. So as documentation is applied, how it renders can be seen.

While HTML is perhaps not the best format to put into a repository such as GITHub it isn’t difficult to translate HTML to markdown using pandoc.

Terraform Python as a Data Source

Connecting Terraform to Python scripts to generate content e.g. algorithms for naming, creating JSON for tagging mechanisms can be done. On the Terraform side this is done by defining a Data Source. When used within Python it calls the Python script passing any data through as JSON string using stdout (sys.stdin.read) rather than the args being passed in.

If you want to determine whether the Python code has been invoked from the shell or Terraform, the give away is that sys.argv[0] won’t provide the name of the invoking module (it’s own name when from the CLI).

Remember this means you need to ensure no logging goes to the console as this will get read back by Terraform as it is looking at stdout for a JSON structured response and logging information will simply throw a spanner in the works.

Useful Links

Libs and SDKs