To facilitate being able to develop on the UQ Drupal CMS, a Virtual Machine (VM) has been developed that provides access on machines to the infrastructure, code base and standard behaviour of UQ CMS provisioned installations.

Please note that sites created in the VM will not be migrated onto the UQ Drupal hosting infrastructure.

Requirements

In order to be able to use the developer virtual machine, the following programs are required to be installed on the system:

  • vagrant (v1.7.2+)
  • virtualbox (v4.3.12+)  Download the latest stable 4.3.x build and not the 5.x series.
    • For windows machines with latest security updates, 4.3.34+ is required.

For Windows machines, it is suggested the following programs be installed:

For those needing to interact with repositories from http://git.its.uq.edu.au, a developer SSH key needs to be installed on your system -- the instructions are the same as those for github.com.

Installation

To install the virtual machine, and make it available for use, create an initial directory using either the terminal (Mac), or Git Bash (Windows) -- this will be called VIRTUAL_MACHINE_DIRECTORY in the documentation below.

Within this directory, execute the following commands:

$ vagrant init uq/uq-drupal-devbox https://boxes.devops.its.uq.edu.au/catalog/uq/uq-drupal-devbox

Set the IP address for the VM:

  • Open the Vagrantfile in a text editor.
  • Search for: config.vm.network "private_network"
  • Uncomment this line and set the IP whatever you are using for your VM. Default: 192.168.254.44
    config.vm.network "private_network", ip: "192.168.254.44"

Start the virtual machine:

$ vagrant up

There are some first time setup errors that occur when doing a vagrant up:

  1. chown: cannot access ‘/data/apps/sites/*/files/’: No such file or directory.
  2. Invalid argument supplied for foreach() backend.inc:713 [warning].

This will download and setup the vagrant box for you.

It will be necessary to add to your hosts file (/etc/hosts on linux machines and C:\Windows\system32\drivers\etc\hosts on window machines) the following lines:

192.168.254.44 phpmyadmin.dev.local
192.168.254.44 webgrind.dev.local
192.168.254.44 auth.dev.local

These domains are then accessible via any browser from your machine, and any sites created within the VM will need equivalent entries in the hosts file.

192.168.254.44 example.dev.local
192.168.254.44 example.uq.edu.au

In order to have access to repositories using SSH keys, it is necessary to copy the SSH keys (e.g. id_rsa and id_rsa.pub) generated under the requirements section to the VIRTUAL_MACHINE_DIRECTORY, and then run the following commands:

  • Locate the ssh files on your local machine. You should have an id_* and an id_*.pub file.
  • Copy the files to the root folder of your VIRTUAL_MACHINE_DIRECTORY.
  • Access the VM with the below command:
    $ vagrant ssh
  • Copy the ssh files from your local machine to the VM with the below command:
    $ cp /vagrant/id* ~/.ssh

For Windows machines, SSH key permissions must be secured to allow access to repositories:

$ chmod 600 ~/.ssh/id_rsa
$ chmod 644 ~/.ssh/id_rsa.pub

To be able to utilize git commands within the VM, the following commands need to be executed:

$ vagrant ssh
$ git config --global user.email "youremail@uq.edu.au"
$ git config --global user.name "Your Name"

Virtual Machine Usage

Turn on the VM

$ vagrant up

Turn off the VM

It is essential that the virtual machine be turned off before shutdown of the computer -- this is to ensure any changes to the filesystem or database are correctly saved.

$ vagrant halt

Update the VM

When the virtual machine gives a notice that it is out of date (when running vagrant up), it needs to be updated to the latest version. It is recommended that an archive be performed before regenerating the VM, so it can keep the sites created and being used.

$ vagrant ssh
$ usershell.sh
   Select archive, and follow the prompts.
$ exit
$ vagrant destroy --force
$ vagrant box update
$ vagrant up
$ vagrant ssh
$ usershell.sh

    Select Restore, and follow the prompts.

Warning: The archive and restore process only includes files that have been uploaded to Drupal sites on the developer virtual machine that are stored on the local file system.  Files stored in the local riak storage container are not included.  See Structure | File Storage on your Drupal site for more information about storage containers.

Update the CMS Codebase

$ vagrant ssh
$ usershell.sh
 Select "Update CMS Code"

Create a site

$ vagrant ssh
$ usershell.sh
 Select Add Site, and follow the prompts.

Sites created in this way, are accessible to login via the user/login menu path with the following credentials:

Username: root
Password: root

To execute drush commands on the newly created site, you can run from anywhere in the VM, the following example command:

drush @example.com/blah pm-enable coder

Connecting to the Virtual Machine (Samba)

Access to the code is available through a Samba connection, which is vital to be disconnected before the virtual machine is shutdown.

For Mac OSX:

  • Open Finder.
  • Select Go -> Connect To Server
  • Server address — smb://192.168.254.44/data
  • Connect with Guest Account.

For Windows:

First enable the windows Guest account:

  • Open Control Panel > User Accounts > User Accounts
  • Click Manage User Accounts
  • Go to the Advanced tab and click the 'Advanced' button
  • Click on the Users folder in the left column
  • Right-click on the Guest account and select Properties
  • Untick the 'Account is disabled' option. Click OK.

 

  • Click the Start button and select This PC.
  • Click Computer | Map network drive on the top toolbar
  • Enter the folder name as "\\192.168.254.44\data"
  • Click Finish to connect as Guest.

PHPStorm on Samba

On a Mac OS machine using PHPStorm through the Samba, a few minor settings need to be adjusted:

  1. Disable the Git plugin for the project.
    PHPStorm corrupts the git index, so disabling the git plugin stops this from happening. Instead use the command line to perform the git actions.
  2. Turn off safe writes.
    PHPStorm corrupts some files during saves, as the samba share does not keep up with how quickly PHPStorm performs the "safe" actions. The unsafe actions still successfully safely saves files correctly.

Accessing the databases (PHPMyAdmin)

Access to the databases can either be through the command line using:

$ vagrant ssh
$ mysql -u root
Enter password: root

Or through a browser at http://phpmyadmin.dev.local/.

Username: root
Password: root

Profiling a page (Webgrind)

For those wishing to profile code, xdebug provides a profiler which can be triggered to start by adding to the query string: XDEBUG_PROFILE=1, or by using the Chrome XDebug addon.

Access to the profiled pages is located at http://webgrind.dev.local/.

Debugging with the Virtual Machine (XDebug)

The virtual machine contains xdebug configured for PHPStorm.

Browser based debugging

It is recommended to install an xdebug extension for your browser, to facilitate debugging pages, although it is possible to trigger it on a page by adding XDEBUG_SESSION_START=1 to the query string of the address.

Command line debugging

For debugging drush commands, or behaviours, the following commands must be executed before running drush:

$ export XDEBUG_CONFIG="idekey=PHPSTORM remote_host=192.168.254.1"
$ export PHP_IDE_CONFIG="serverName=_"
$ export DRUPAL_SITE_CONFIG="/data/apps/uq-drupal/drushrc/includes/settings.php"

These are all necessary environment variables to have xdebug correctly notify PHPStorm, with the DRUPAL_SITE_CONFIG allowing the appropriate loading of the sites installed on the VM.

To turn drush debugging off again:

$ unset XDEBUG_CONFIG

For debugging the drupal test runner:

  1. Ensure /data/apps/uq-drupal/drupal/scripts/run-tests.sh is copied to /data/apps/uq-drupal/drupal/scripts/run-tests.php.
    PHPStorm doesn’t seem to correctly allow debugging with the run-tests.sh.

     

  2. From the /data/apps/uq-drupal/drupal directory, run the following command:
    sudo -u nginx -g www XDEBUG_CONFIG="idekey=PHPSTORM remote_host=192.168.254.1" PHP_IDE_CONFIG="serverName=_" DRUPAL_SITE_CONFIG=/data/apps/uq-drupal/drushrc/includes/settings.php php scripts/run-tests.php --url http://example.dev.local {TEST TARGET HERE}.

Alternative IDEs

The IDE key can be changed by modifying /opt/rh/php54/root/etc/php.d/xdebug.ini.

Once modified the php-fpm process must be restarted:

$ sudo service php54-php-fpm restart

Modifying this key will need to be done every time the virtual machine is updated.

Virtual Machine Internals

Code locations

  • /data/apps/uq-drupal-standard/drupal
    The location of the core drupal files for the CMS standard distribution.
  • /data/apps/uq-drupal-legacy/drupal
    The location of the core drupal files for the CMS legacy distribution.
  • /data/apps/sites
    The location of the sites that are generated by the usershell.sh script.
  • /data/apps/phpmyadmin
    The location of the PHPMyAdmin application code base.
  • /data/apps/webgrind
    The location of the Webgrind application code base.
  • /data/xprofile
    The location of the XDEBUG profiler generated files.

Configuration locations

  • /etc/nginx/conf.d
    The configuration directory for sites. Examples of sites with a base_path are located within the default.conf file.
  • /opt/rh/php54/root/etc/php.d
    The configuration directory for PHP. These are the values normally associated with php.ini.
  • /opt/rh/php54/root/etc/php-fpm.d
    The configuration directory for PHP-FPM. These are values associated with the server that executes PHP.
  • /etc/my.cnf.d
    The configuration directory for MySQL galera.

Log locations

  • /var/log
    The primary log location.
  • /var/log/nginx/drupal.log
    The syslog information for correctly setup drupal sites using the syslog module.
  • /var/log/puppet/puppet.log
    The syslog information for incorrectly setup drupal sites using the syslog module. This also contains all syslog entries that haven’t been assigned to a different syslog channel.

Service commands

  • service php54-php-fpm restart
    This restarts the PHP-FPM service, this is required whenever changing any of the php.ini settings.
  • service nginx restart
    This restarts the nginx service, this is required if making changes to the web server hosts.
  • service mysql restart
    This restarts the mysql service, this is required when making changes to the database configuration.
  • service redis restart
    This restarts the redis service, this is required when making changes to the caching of drupal cache tables, or session handling.
  • service riak restart
    This restarts the riak service, this is required when making changes to the shared file handling for drupal.

Frequently Asked Questions

How do I configure putty to use my developer virtual machine ssh key?

Run Puttygen.

Select Conversions | Import key and load the private key from where your vagrant virtual machine is stored.  

e.g. {VIRTUAL_MACHINE_DIRECTORY}/.vagrant/machines/default/virtualbox/private_key

Click Save private key to save your private key to the same directory in .ppk format.

e.g. {VIRTUAL_MACHINE_DIRECTORY}/.vagrant/machines/default/virtualbox/private_key.ppk

In putty, create a new session to your developer virtual machine.

Under Connection | SSH | Auth browse to the .ppk file you created above as your private key file for authentication.

Why am I being prompted for a password for git@git.its.uq.edu.au?

Have you copied your SSH key files to ~/.ssh (and set them to the correct permissions for Windows machines)?

If you have recently updated your developer virtual machine, this step must be done again.

See Installation for further details.