Develop Cloud Applications for OpenStack on Murano, Day 3: The application, part 1: Understanding Plone deployment

OK, so so far, in Part 1 we talked about what Murano is and why you need it, and in Part 2 we put together the development environment, which consists of a text editor and a small OpenStack cluster with Murano.  Now let's start building the actual Murano App.

What we're trying to accomplish

In our case, we're going to create a Murano App that enables the user to easily install the Plone CMS. We'll call it PloneServerApp.
Plone is an enterprise level CMS (think Wordpress on steroids).  It comes with its own installer, but it also needs a variety of libraries and other resources to be available to that installer.
Our task will be to create a Murano App that provides an opportunity for the user to provide information the installer needs, then creates the necessary resources (such as a VM), configures it properly, and then executes the installer.
To do that, we'll start by looking at the installer itself, so we understand what's going on behind the scenes.  Once we've verified that we have a working script, we can go ahead and build a Murano package around it.

Plone Server Requirements

First of all, let’s clarify the resources needed to install the Plone server in terms of the host VM and preinstalled software and libraries. We can find this information in the official Plone Installation Requirements.

Host VM Requirements

Plone supports nearly all Operating Systems, but for the purposes of our tutorial, let’s suppose that our Plone Server needs to run on a VM under Ubuntu.
As far as hardware requirements, the Plone server requires the following:
Minimum requirements:

• A minimum of 256 MB RAM and 512 MB of swap space per Plone site
• A minimum of 512 MB hard disk space

Recommended requirements:

• 2 GB or more of RAM per Plone site
• 40 GB or more of hard disk space

The Plone Server also requires the following to be preinstalled:

• Python 2.7 (dev), built with support for expat (xml.parsers.expat), zlib and ssl.
• Libraries:
  • libz (dev),
  • libjpeg (dev),
  • readline (dev),
  • libexpat (dev),
  • libssl or openssl (dev),
  • libxml2 >= 2.7.8 (dev),
  • libxslt >= 1.1.26 (dev).

The PloneServerApp will need to make sure that all of this is available.

Defining what the PloneServerApp does

Next we are going to define the deployment plan. The PloneServerApp executes all necessary steps in a completely automatic way to get the Plone Server working and to make it available outside of your OpenStack Cloud, so we need to know how to make that happen.
The PloneServerApp should follow these steps:

1. Ask the user to specify the host VM, such as number of CPUs, RAM, disk space, OS image file, etc. The app should then check that the requested VM meets all of the minimum hardware requirements for Plone.
2. Ask the user to provide values for the mandatory and optional Plone Server installation parameter.
3. Spawn a single Host VM, according to the user's chosen VM flavor.
4. Install the Plone Server and all of its required software and libraries on the spawned host VM. Well have PloneServerApp do this by launching an installation script (runPloneDeploy.sh).

Let's start at the bottom and make sure we have a working runPloneDeploy.sh script; we can then look at incorporating that into the PloneServerApp.

Creating and debugging a script that fully deploys the Plone Server on a single VM

We'll need to build and test our script on a Ubuntu machine; if you don't have one handy, go ahead and deploy one in your new OpenStack cluster. (When we're done debugging, you can then terminate it to clean up the mess.)
Our runPloneDeploy.sh will be based on the Universal Plone UNIX Installer. You can get more details about it in the official Plone Installation Documentation, but the easiest way is to follow these steps:

1. Download the latest version of Plone:

   $ wget --no-check-certificate https://launchpad.net/plone/5.0/5.0.4/+download/Plone-5.0.4-UnifiedInstaller.tgz

2. Unzip the archive: <pre?$ tar -xf Plone-5.0.4-UnifiedInstaller.tgz
3. Go to the folder containing the installation script...

   $ cd Plone-5.0.4-UnifiedInstaller

4. ...and see all installation options provided by the Universal UNIX Plone Installer:

   $ ./install.sh --help
   

The Universal UNIX Installer lets you choose an installation mode:

• a standalone mode - single Zope web application server will be installed, or
• a ZEO cluster mode - ZEO Server and Zope instances will be installed.

It also lets you set several optional installation parameters. If you don’t set these, default values will be used.
In this tutorial let’s choose standalone installation mode and make it possible to configure the most significant parameters for standalone installation. These most significant parameters are the:

• administrative user password
• top level path on Host VM to install the Plone Server.
• TCP port from which the Plone site will be available from outside the VM and outside your OpenStack Cloud

Now, if we were installing Plone manually, we would feed these values into the script on the command line, or set them in configuration files.  To automate the process, we're going to create a new script, runPloneDeploy.sh, which gets those values from the user, then feeds them to the installer programmatically.
So our script should be invoked as follows:

$ ./runPloneDeploy.sh <InstallationPath> <AdminstrativePassword> <TCPPort>

For example:

$ ./runPloneDeploy.sh “/opt/plone/” “YetAnotherAdminPassword” “8080”

The runPloneDeploy.sh script

Let's start by taking a look at the final version of the install script, and then we'll pick it apart.

1.  #!/bin/bash
2.  #
3.  #  Plone uses GPL version 2 as its license. As of summer 2009, there are
4.  #  no active plans to upgrade to GPL version 3.
5.  #  You may obtain a copy of the License at
6.  #
7.  #       http://www.gnu.org
8.  #
9.  
10. PL_PATH="$1"
11. PL_PASS="$2"
12. PL_PORT="$3"
13. 
14. # Write log. Redirect stdout & stderr into log file:
15. exec &> /var/log/runPloneDeploy.log
16. 
17. # echo "Installing all packages."
18. sudo apt-get update
19. 
20. # Install the operating system software and libraries needed to run Plone:
21. sudo apt-get -y install python-setuptools python-dev build-essential libssl-dev libxml2-dev libxslt1-dev libbz2-dev libjpeg62-dev
22. 
23. # Install optional system packages for the handling of PDF and Office files. Can be omitted:
24. sudo apt-get -y install libreadline-dev wv poppler-utils
25. 
26. # Download the latest Plone unified installer:
27. wget --no-check-certificate https://launchpad.net/plone/5.0/5.0.4/+download/Plone-5.0.4-UnifiedInstaller.tgz
28. 
29. # Unzip the latest Plone unified installer:
30. tar -xvf Plone-5.0.4-UnifiedInstaller.tgz
31. cd Plone-5.0.4-UnifiedInstaller
32. 
33. # Set the port that Plone will listen to on available network interfaces. Editing "http-address" param in buildout.cfg file:
34. sed -i "s/^http-address = [0-9]*$/http-address = ${PL_PORT}/" buildout_templates/buildout.cfg
35. 
36. # Run the Plone installer in standalone mode
37. ./install.sh --password="${PL_PASS}" -- standalone
38.
39. # Start Plone
40. cd "${PL_PATH}/zinstance"
41. bin/plonectl start

The first line states which shell should be execute the various commands commands:

#!/bin/bash

Lines 2-8 are comments describing the license under which Plone is distributed:

 #
 #  Plone uses GPL version 2 as its license. As of summer 2009, there are
 #  no active plans to upgrade to GPL version 3.
 #  You may obtain a copy of the License at
 #
 #       http://www.gnu.org
 #

The next three lines contain commands assigning input script arguments to their corresponding variables:

 PL_PATH="$1"
 PL_PASS="$2"
 PL_PORT="$3"

It’s almost impossible to write a script with no errors, so Line 15 sets up logging. It redirects both stdout and stderr outputs of each command to a log-file for later analysis:

 exec &> /var/log/runPloneDeploy.log

Lines 18-31 (inclusive) are taken straight from the Plone Installation Guide:

 sudo apt-get update

 # Install the operating system software and libraries needed to run Plone:
 sudo apt-get -y install python-setuptools python-dev build-essential libssl-dev libxml2-dev libxslt1-dev libbz2-dev libjpeg62-dev

 # Install optional system packages for the handling of PDF and Office files. Can be omitted:
 sudo apt-get -y install libreadline-dev wv poppler-utils

 # Download the latest Plone unified installer:
 wget --no-check-certificate https://launchpad.net/plone/5.0/5.0.4/+download/Plone-5.0.4-UnifiedInstaller.tgz

 # Unzip the latest Plone unified installer:
 tar -xvf Plone-5.0.4-UnifiedInstaller.tgz
 cd Plone-5.0.4-UnifiedInstaller

Unfortunately, the Unified UNIX Installer doesn’t give us the ability to configure a TCP Port as a default argument of the install.sh script. We need to edit it in buildout.cfg before carrying out the main install.sh script.
At line 34 we set the desired port using a sed command:

sed -i "s/^http-address = [0-9]*$/http-address = ${PL_PORT}/" buildout_templates/buildout.cfg

Then at line 37 we launch the Plone Server installation in standalone mode, passing in the other two parameters:

./install.sh --password="${PL_PASS}" -- standalone

After setup is done, on line 40, we change to the directory where Plone was installed:

cd "${PL_PATH}/zinstance"

And finally, the last action is to launch the Plone service on line 40.

bin/plonectl start

Also, please don’t forget to leave comments before every executed command in order to make your script easy to read and understand. (This is especially important if you'll be distributing your app.)

Run the deployment script

Check your script, then spawn a standalone VM with an appropriate OS (in our case it is Ubuntu OS 14.04) and execute the runPloneDeply.sh script to test and debug it. (Make sure to set it as executable, and if necessary, to run it as root (or using sudo)!)
You'll use the same format we discussed earlier:

$ ./runPloneDeploy.sh <InstallationPath> <AdminstrativePassword> <TCPPort>

For example:

$ ./runPloneDeploy.sh “/opt/plone/” “YetAnotherAdminPassword” “8080”

Once the script is finished, check the outcome:

• Find where Plone Server was installed on your VM using the find command, or by checking the directory you specified on the command line.
• Try to visit the address http://127.0.0.1:[Port] - where [Port] is the TCP Port that you point to as an argument of the runPloneDeploy.sh script.
• Try to login to Plone using the "admin" username and [Password] that you point to as an argument of the runPloneDeploy.sh script.

If something doesn’t seem to be right check the runPloneDeploy.log file for errors.
As you can see, our scenario has a pretty small number of lines but it really does the whole installation work on a single VM. Undoubtedly, there are several ways in which you can improve the script, like smart error handling, passing more customizations or enabling Plone autostart. It’s all up to you.
In part 4, we'll turn this script into an actual Murano App.