CoW: Users and Developers Guide

A Larson Cartoon

Contents

1. Introduction

Files and directories, documents and folders, disks and memory sticks, laptops and games machines, TV time-shifters and corporate IT systems. Data data everywhere, and never a drop to drink, as the Ancient Mariner could not have dreamt of saying. Wouldn't it be nice to be able to view your filesystem as a website, to be able to edit from multiple machines with and without network connections, to be able to have your own local copy and also share it with friends and colleagues, and to not have to worry about merging it all back together?

CoW is a "Controllable Wiki" and CMS that supports collaborative document creation with asynchonous off-line editing. CoW is desiged to make it easy to add interaction to static websites, and to support concurrent editing and off-line working with straightforward synchronisation (using Subversion). The system also serves as a test-bed for experiments in controlled languages for round-trip ontology engineering (from the GATE project: http://gate.ac.uk/).

CoW is based on Grails, Groovy, Subversion, Canoo and J2EE, and is hosted on SourceForge in the GATEWiki project.

Currently CoW includes:

The system has two modes, workstation and server; the former does no user management, the latter uses Spring Security.

2. Overview

Wikis are about allowing groups of people to create and edit sets of interlinked web pages with minimal effort and minimal learning of software conventions and features. Typically they achieve this by

Content Management Systems (CMSs) provide persistence, versioning, metadata management, upload and browsing of sets of documents and other resources.

CoW is different from other wikis and CMSs because:

No available wiki in Java that we could find has good SVN support. Using SVN as a backend gives us:

CoW is partly intended to be an experimental framework for a new type of website in which

(In fact perhaps that isn't a wiki at all, but a new type of literate database...)

The system is licenced under the GNU General Public Licence version 3 (GPL 3) except where otherwise stated.

3. Information for Users

This is incomplete :-(

3.1. Modes

CoW has two operational modes:

When you use CoW on your own machine it will be in workstation mode; when you're using it over the web on another machine it will be in server mode.

3.2. Register, Log in

When CoW is running in server mode, it is necessary to create an account and to log in before being allowed to edit pages. Some pages will also not be accessible for reading unless an administrator adds you to the appropriate group. To register, go to the login page and follow the register link.

3.3. Create a new wiki page

To create a new wiki page you can either:

Warning: if you create a new wiki page then the directory containing the page gets upgraded to SVN 1.4 format and you will then need to use an SVN 1.4+ client to access it.

3.4. Edit a wiki page

When you have permission to edit a page that you're viewing, two links will appear:

  1. "Edit": this will start a WYSYWIG editor (FCK Edit)
  2. "(with form)": this will forward to a web form page to edit the wiki page with

When using 1., you can generally ignore YAM syntax and work like you do in a word processor - e.g. hit the "B" button to make text bold, or the "I" button to make it italic, etc. When you're finished, hit the save button (it looks like an old floppy disk). There's one exception to ignoring YAM: because the edit is converted back to YAM afterwards, if you include YAM syntax in your edit you'll need to escape it. Do this by putting a backslash in front - e.g. \*.

When using 2. use YAM syntax (here's a short summary, which also appears below the form while editing).

3.5. Create a new wiki space

NEEDS UPDATING. currently only available to administrators, who should go to the top-level cow page and follow the links

To create a new wiki space you can either:

In the former case you can choose any existing SVN-controlled file tree and CoW will allow you to create, share and update YAM files in that tree.

3.6. Report a bug

To report bugs, first please check that they've not been reported already!

Then add a report to the bug tracker, including information about the platform you're running on and all details necessary to reproduce your problem.

4. Information for Developers

4.1. Roadmap

The development roadmap and wish-list is recorded in the backlog document, and currently active tasks list.

4.2. Checking out the code

To check out CoW from Subversion first decide if you want to check out a copy of Grails HEAD and Jetty while you're at it. If so do this:

svn co https://gatewiki.svn.sourceforge.net/svnroot/gatewiki/trunk gatewiki

If not, do this:

svn co https://gatewiki.svn.sourceforge.net/svnroot/gatewiki/trunk/cow cow

If you do the latter you'll need to set properties in build.xml to point to your own installation of Grails (and of Jetty if you want to run in production). (Also, if you put CoW in a directory not named cow you will need to change projectName in webtest/conf/webtest.properties to get the Canoo tests to work.)

4.3. Selecting Modes

To select the mode (see above) use

Workstations include laptops and are expected to be off-line from time-to-time; servers are expected to be always connected (and always have accses to the relevant SVN repositories).

4.4. Building and testing CoW

CoW is built with ant. For more documentation do ant help in the cow directory. To run CoW see next section.

Currently the build file

To build CoW from a clean checkout, assuming that grails is in the same directory:

The cruise target does a clean build and runs the Grails unit tests, the Java-only tests (for YAM etc.) and the Canoo webtests. The war target creates a WAR file which can be deployed using the deploy target. Other useful targets:

See ant help for more details.

Note that there are some bugs in ant target ordering (workaround for most of them is to run ant cruise once before doing anything else):

4.4.1. Upgrading to new Grails versions

4.4.2. Using the YAM Tests

The test suite uses a bunch of .yam and compiled .html files in cow/test/resources. After running the tests the script cow/bin/check-errors will check for failures and, when these are caused by incorrect translations, display a (tk)diff of the actual and the correct output.

Note: when updating to new versions of yam2html the test resource yam-minimal-no-includes.html needs to be selectively merged with yam-minimal.html. First update the latter to reflect the translation changes, then:

4.5. Running CoW

The ant run-dev target does a Grails run-app and ant run-prod runs a non-Grails Jetty instance on the CoW WAR. ctrl-c will shut down Jetty. ctrl-c of Ant does not, however, cleanly shutdown a Jetty forked from Ant. For a clean shutdown with correct execution of all shutdown code, run ant shutdown-prod You may supply an optional port on which Jetty will listen for the shutdown signal, and a password key to listen for, with -Djetty.shutdown.port and -Djetty.shutdown.key. Defaults are cleartext in the build file.

Note that the first time the system runs it will create a .cowrc.d directory (or cowrc.d on Windoze) in your home directory containing help documentation, a DB etc.

4.6. Structure

CoW is made up of the following components (and numerous 3rd party libraries):

4.6.1. Main classes and Grails objects

The CoW webapp is organised around the concept of wiki/content areas, which are file trees stored in SVN. What CoW then provides is

CoW is implemented using Grails. The Grails MVC model works like this:

Or, put another way, the Grails web application architecture has four main components:

In CoW we have:

Example:

4.7. Authentication and Authorisation

4.7.1. Wikis and Roles

This section gives some examples of roles and the URLs that they relate to (stored in Requestmap).

For example, roles on the gate.ac.uk site:

The default roles are:

The default accounts are:

Wikis, URLs and roles examples for the example (GATE) site:

Id Content Reader URL example Readers Writer URL example Writers
1 CoW help docs /show/1/ all /edit/1/ admin, gate-user
2 main GATE HTML (gate-server/html) /show/2/ all /edit/2/ admin, gate-user
3 public wiki (gate-extras/pubwiki) /show/3/support.html all /save/3/support.html other-user, admin, gate-user
4 SALE stuff (sale) /show/4/ all /preview/4/ admin, gate-user
5 SALE internal (sale/internal) /show/5/index.html admin, gate-user /edit/5/conventions.html admin, gate-user

The data model contains four classes:

4.8. Configuring mail for user registration

In cow/grails-app/conf/AcegiConfig.groovy

4.9. Referencing and regeneration

The problem is that a create/rename/delete/modify operation on a .yam should be cascaded through any files that link to or include it. The relevant data is what a file "linksTo" and "includes". Regeneration is needed:

This information needs to be maintained in a single dependency graph for each Wiki area. This is coded in class Dependencies.

If we can guarantee that all necessary regeneration and dependency maintenance is done whenever any generation is done, then the graph can be assumed to be consistent. In workstation mode that's an open issue because of updates from other tools but in the worst case we can periodically or on request regenerate all the output files to ensure consistency.

Referencing events

Event sources

Possible event handlers (using and maintaining a dependency graph)

The current design is to use YamFile.generate.

4.9.1. Serialization

Dependencies are serialized to the directory conf.gate.cow.dbs, one file per wiki. They are deserialized lazily, as required.

4.9.2. Speed of Running SVN Status

As noted above, YAM files managed by the wiki can be changed in several ways:

In each of these cases we need to regenerate any files that are dependent on the changes (e.g. a .html that is older than its .yam; a YAM file which refers to a wiki page which no longer exists; etc.).

There are a number of strategies that might be adopted to determine what needs regenerating:

  1. a low-priority process that inspects the filesystem directly
  2. a listener that receives notifications from the SVN repository
  3. a process that runs svn st to get a report on files that have changed

For option 3., the statistics in this table are interesting. They give an indication of how long it takes to do an svn st on the SALE tree (which is more than 5Gb of data in more than 100,000 files.

svn st on the sale tree (from cold) svn st -u on sale (filesystem cached)
First run Tenth run

real 0m58.839s
user 0m5.236s
sys 0m3.576s

real 0m2.380s
user 0m1.604s
sys 0m0.504s
First run Tenth run

real 0m33.331s
user 0m3.772s
sys 0m0.852s

real 0m31.082s
user 0m3.280s
sys 0m0.972s

The local status check (which doesn't go to the network to check on repository changes) takes around a minute from cold, but subsequently (when the operating system has had a chance to cache parts of the file system) only takes a couple of seconds. The network check (svn st -u) stays pretty constant around 30 seconds.

(Tests done on a 2GHz Pentium M with 1Gb RAM running Ubuntu Dapper on a broadband connection.)

4.10. Sourceforge notes

4.11. Misc notes

Footnotes

  1. SVN: the Subversion version control system.