Development environment for the Wingpath website

The Wingpath website is developed using a local copy, which is then copied to the "live" version on the website hosting machine. There are two local copies on sage: one is used for making minor changes, and is normally identical to the live version; the other is used for major developments. If there is no major development in progress, the two local copies will be essentially the same. There are also local copies on nettle, but this document describes only the sage ones - the nettle ones are similar.

The local copy used for minor changes is at /www/website/web (/www is actually a symbolic link, so the full pathname is actually /var/www/htdocs/website/web). The copy used for major changes is at /www/website/web.new.

Both local copies are configured as "virtual hosts" in the sage Apache web-server configuration file (/etc/apache/httpd.conf), so that they appear as http://website.wingpath.private/ and http://website.new.wingpath.private/.

Both local copies are actually SVN (SubVersion) "working copies", with most of the files being under SVN control (i.e. if you edit a file, you should commit it to the SVN repository afterwards). Both local copies are owned by the user 'fog', so you will have to be logged in as 'fog' to change anything (I have been unable to persuade SVN to allow two users to share a working copy). Not all the files are under SVN control: some are copied from elsewhere and need to be updated elsewhere if you want to change them.

Copying to the live site

The shell script update-website (full pathname /usr/wingpath/bin/update-website) is used to copy files from /www/website/web to the live site. update-website takes no arguments, and should be run as user wingpath or fog (if you want to run it as another user, you will have to install an SSH key in ~wingpath/.ssh/authorized_keys2 on the website host for that user).

update-website uses rsync and ssh to copy files. Since it uses rsync, only modified files will be copied and the copying is very efficient.

update-website uses a list of files to be excluded from the copy. Any file not in the excluded list will be copied to the live website, so don't leave extraneous files lying around in the local-copy directories!

The update-website script is under SVN control, so if you need to modify it you should do so in an SVN working copy (/usr/wingpath/bin/update-website is not a working copy!). There is a working copy at /home/fog/Development/scripts/update-website that you could use (this is owned by fog, so you would need to log in as fog to modify it), or you could check out your own from svn://sage/trunk/development/scripts/.

Files not under SVN control

Some of the files in the local copies are not under SVN control - they are either generated by running the website or are copied from elsewhere. These are listed below:

GeoIP.dat
This is a database that maps IP addresses to country codes. It is automatically downloaded from www.maxmind.com to /www/website/web by the cron script /etc/cron.weekly/geoip. The cron script also copies the file to the live website (hence, the update-website script does not copy this file). If you need an up-to-date copy of GeoIP.dat in /www/website/web.new (or any other local copy of the website), you will have to manually copy it there.
bin/*
These are executables that needed on the website. To create the files regkey, dljar and geoip, cd to the src directory and type 'make'. The zip file is manually copied from /usr/bin/zip. Note that the dljar executable needs to be setuid on the live site so that it can write to the jar files and its output file (dljar_trace) - this should happen automatically, but is worth checking if jar downloads from the live site don't work.
dljar_trace
This is output from the executable dljar, which is run when a user wants to download a jar file. dljar_trace is normally empty, but may contain error output if anything went wrong with the download.
docs/*
Apart from modbus_tcp_specification.pdf (which is under SVN control), these are product manuals. They need to be copied from the SVN working copies of the respective products - see 'make.web' and 'make web.new'.
images/*
The images in the top-level of this directory are under SVN control, but those in sub-directories are screenshots, which need to be copied from the SVN working copies of the respective products - see 'make.web' and 'make web.new'.
jar/*
These are product jar files, which need to be copied from the SVN working copies of the respective products - see 'make.web' and 'make web.new'.
logs/*
These are registration log files, which are generated by running the website. log is the actual CSV-separated log file, id contains the identifier used for the last entry in the log file (it gets incremented for each entry), and uid contains the last-used "user identifier" (this is sent in a cookie to the user's browser to try and recognize returning users).

'make web' and 'make web.new'

Some of the files in the website directories are copied from product development directories:
These files are copied to the website local copy when 'make web' or 'make web.new' is executed in a product development directory. The command 'make web' copies files to /www/website/web, and the command 'make web.new' copies files to /www/website/web.new. There are working copies of the product development directories under the directory /home/fog/Development.

Since ModSlaveSim and ModMultiSim are developed elsewhere, there are only dummy product development directories on sage (/home/fog/Development/modslavesim and /home/fog/Devlopment/modmultisim respectively). These directories contain the screenshots to be used on the website. The manuals are not currently put on the website, so these are not in the dummy directories. The (obfuscated) JAR files for these products must be copied into the directory /home/fog/Development/build/bin before executing 'make web' or 'make web.new'.

Before a jar is copied, the script /home/fog/Development/Make/regjar.sh is run (by 'make web' or 'make web.new') to prepare the jar for downloading by users. See the comments in regjar.sh for details.

Screenshots are copied (by 'make web' or 'make web.new') using the script /home/fog/Development/Make/copyimages.sh, which also generates scaled-down versions of the screenshots (these are the ss.*.png files in the sub-directories of the images directory.

SVN branches

When a major development of the website is in progress, the web.new and web working copies refer to different SVN branches: web.new refers to the "trunk", whereas web refers to a "maintenance" branch. If there is no major development in progress, web and web.new both refer to the SVN trunk.

When a major development is about to be started, an SVN branch is created for maintenance of the current website, using something like the following commands:

svn copy svn://sage/trunk/website svn://sage/branches/website.3        # Create an SVN branch with name website.3
cd /www/website/web
svn switch svn://sage/branches/website.3                                        # Switch the web working copy to the new branch

When the major development is complete, we switch the web working copy back to the trunk, and also update the files that are not under SVN control:

cd /www/website/web
svn switch svn://sage/trunk/website
cd /home/fog/Development
make web