Holy Chao

I started off by cloning our repository to a sandbox that I could muddy up with many pointless commits and broken hooks without disturbing other people's work. After that I set off to tackling the default config problem. We do not have a database.yml in our source trees for our rails projects, instead having a database.yml.example that is instructive of how your config should look. Additionally we use a config.yml to specify startup options for the server, but again, it is not in the default tree. Instead we use a config.yml.example that developers rename when they check out the tree.

To solve this first issue I created a config/defaults directory, which would have sensible defaults pointing at the database instances running on the build machine. In addition I made a rake task to copy the files from the config/defaults directory to the config directory, and placed this .rake file in the lib/tasks directory of our rails project.

Now, when cerberus checked out our code from the repository, I could specify that it should run the "copy default config files" target before the "test application" target and there'd be sensible defaults in place.

Next I set up cerberus for this repository. The cerberus setup was mostly straightforward according to cerberus' documentation. The only interesting work I did on it comes later.

In my testing I set up a SVN hook to run cerberus on checkin, by adding the line:

sudo -u cerberus cerberus buildall

to hooks/post_commit

This had the unfortunate consequence of making it so that the comitting developer was force fed all of the output of running cerberus. It also meant that the commit process had to wait around for the whole cerberus build to complete! This was going to slow down people (and arguably, commits shouldn't be rushed), but sucked either way. I tried just redirecting stdout to /dev/null but svn holds onto the error output pipe of child processes spawned by hooks as well. Ultimately to fix the problem, I ended up having to change the post-commit to read as follows:

sudo -u cerberus cerberus buildall >>dev/null 2>&1 &

This worked great! Next I wanted RCov to fire on checkin and successful build. Cerberus allows you to define hooks in the config.yml for specific projects and the global config.yml that fire at certain lifecycles in the cerberus execution cycle. They run just like they're invoked from a shell, with the working directory being the root of the local sandbox of whatever you have checked out from subversion. Between adding the hook, and making the config file copy fire, my ~/.cerberus/config/<project>.yml ended up with the following additions:

builder:
  rake:
    task: config:default_files test
hook:
  RCOV_HOOK:
    on_event: successful
    action: 'rake test:test:rcov'

Testing with a few checkins, cerberus was building properly and code coverage reports were getting fired! Success!

Moving it over to the real repository snagged a few problems on the way though. They were all solvable.

First, our repository is hosted "officially" over https. I had done my testing over svn+ssh, since that does not require bumping the http daemon, and the test sandbox was going to go away, so there was no point in polluting our apache config with some temporary information. Of course, this meant I wasn't testing the path of coming in from https. Since I was authenticated to the system as as user with sudo permissions, everything worked great. When coming in as wwwrun, pam auth was puking all over my attempts to sudo. This was not going to be acceptable, further wwwrun lacked some of the environment information stored for my cerberus user.

First I needed to make a constrained way for wwwrun to trigger the cerberus build. This was solved with a quick script in my cerberus user's home directory. This would execute the cerberus buildall command and only the cerberus buildall command with the correct home directory (We are running on an X-Serve).

Finally we granted wwwrun the permission to run this script and ONLY this script without authentication via the /etc/sudoers file. I fired up visudo and added the following line:

wwwrun ALL = (cerberus) NOPASSWD: /Users/cerberus/cerb-cerb.rb

The post-commit hook needed tweaking, becoming:

sudo -u cerberus /Users/cerberus/cerb-cerb.rb >>dev/null 2>&1 &

Also, I wanted the code coverage reports viewable by all developers. This was done by tweaking the hook in my cerberus config file to the following:

  RCOV_HOOK:
    on_event: successful
    action: 'rake test:test:rcov && cp coverage/test/* /srv/www/htdocs/rcov/<project>/'

Now we copied the code coverage reports to a publicly accessible directory.

Finally the problem of a re-arranged source tree reared its head again. This time I was going to solve it by starting from a blank slate on every build. The means to do this was, by my judgement, best served by having the sandbox deleted on every successful build. On broken builds it would be possible to go into the sandbox and see what was breaking if it was a problem with the build system and not with the commit. Thus we had one final addition to the cerberus hook:

  RCOV_HOOK:
    on_event: successful
    action: 'rake test:test:rcov && cp coverage/test/* /srv/www/htdocs/rcov/<project>/ && rm -rf /Users/cerberus/.cerberus/work/<project>/sources'

This was all we needed, and now we had builds and code coverage running on every checkin.