Working Towards a Vision

What Was Past

One of the changes that I recently added to CruiseControl.NET (for the 1.5 release) is a common communications library. This project is a cut-down version of Remote – I stripped out everything that had nothing to do with communications (e.g. exceptions, NetReflector, lots of interfaces, etc.)

The aim for this library is simple – it contains everything that is needed for a client to communicate with a server. This includes all communications protocols – currently HTTP and .NET Remoting – plus an implementation of every possible method. Additionally it contains a number of classes to simplify the process of communicating with a server.

Considering a lot of this already existed within Remote, you might ask why I have added another library to do this? The answer is simple – Remote contained the basic components for communications, it didn’t actual provide anything to facilitate the process. Instead, each client had to write its own mechanisms for communications – which is why CCTray has one way for communications, the dashboard has another and the server uses yet another way (when it needs to communicate with other servers.) Making a change in one wouldn’t affect the other – so if there was a common change it needed to be made to all three!

Finally, these mechanisms were deeply entrenched within each client. I couldn’t just take the communications part of CCTray and use it – I had to include everything else (and CCTrayLib has a lot of stuff in it!) I certainly tried and did produce a simple command-line client that used CCTrayLib. The problem was it needed all the other dependencies for CCTray, and it wasn’t the easiest thing to use.

What Is Now

As I wrote above, I have added a new communications only project to CruiseControl.NET. Remote still contains everything, plus the new communications work, while the communications library only contains the communications classes (hence it is a subset of remote.)

The heart of this library is a class called CruiseServerClientBase. This class is actually an abstract class, but it exposes all of the communications methods. The actual implementations of this class take the methods and pass them onto the server for processing. This means a developer does not need to worry about how the methods are transmitted, just that they are transmitted.

Since CruiseServerClientBase is an abstract class there is a static factory class called CruiseServerClientFactory that generates instances of it. This can be as simple as just saying “I want a client class” or as complex as saying “I want a HTTP client class that communicates to the localhost server”. Eventually over time this factory will handle a whole variety of communications protocols – while hiding the complexity from the developer.

The following shows the only line that is needed to instantiate a CruiseServerClientBase:

var client = CruiseServerClientFactory.GenerateClient("tcp://localhost:21234");

This will create a client for connecting to the server on localhost. Since the protocol is tcp:// it will create a .NET remoting client. There are also methods to create HTTP or .NET Remoting clients directly – GenerateRemotingClient() and GenerateHttpClient().

Under the hood these methods all create an instance of CruiseServerClient. This is another facade class that provides common functionality. With the 1.5 release CruiseControl.NET is moving to a message-style method interface. This style means every message has a request message and a response message. CruiseServerClient generates the request messages, passes them onto a IServerConnection instance and then processes the response. The IServerConnection instance is actually responsible for transmitting the messages.

As you can guess, there are HTTP and .NET Remoting implementations of IServerConnection. These serialise the message, send it using the correct protocol and then deserialise the response. This response is then returned from the implementation.

As such, it is possible to completely bypass CruiseServerClientFactory and CruiseServerClient and use the IServerConnection instance directly. However this would mean you’d need to write the code to generate the messages yourself, and you’d need to process the responses yourself. CruiseServerClient does all of this plumbing for you.

The following shows how it is possible to trigger a force build with just two lines of code:

var client = CruiseServerClientFactory.GenerateRemotingClient("tcp://localhost:21234");
client.ForceBuild("A project");

As an added bonus, it is very easy to change from .NET Remoting to HTTP, you’d just change the address. The communications library hides all the underlying complexity for you.

However, before I say everything is roses, there is one current limitation: this will only work with the new 1.5 version server. There is no backwards compatibility.

Which brings me to…

What Is Coming

This work all started out of the security work I did for CruiseControl.NET. I got frustrated by the number of places that needed changing, and the number of different ways, so I merged and simplified them. This opened up some ways of making things simpler under the hood.

However there is still the question of previous versions of CruiseControl.NET. The new library will only work with 1.5 or later versions of CruiseControl.NET servers. It would also be nice to allow backwards compatibility. Now this is much more challenging as a number of base classes have been changed to work with messaging, but I still think it is a worthwhile goal.

I should also mention that HTTP compatibility is broken between 1.4.4 and 1.5. Since CCTray promises that HTTP can be used between versions, I am aiming to ensure that at least HTTP compatibility is ok, and then if possible .NET Remoting. That is the reason why there is a CruiseServerClientBase base class, with only one implementation. These older versions do not use the messaging style method calls, and so won’t be able to use CruiseServerClient. Instead they will need to make their own implementations.

Another step for current functionality is to convert CCTray, the dashboard and the server to all use the new client. These currently still have their existing ways of doing things (although I have started converting CCTray). This means in future, we will only need to change one place to change all three clients! Now that is exciting news to me

And finally I can write about what I think is really exciting – a CruiseControl.NET SDK.

One of the great points about CruiseControl.NET is its flexibility. It has a lot of functionality in-built, plus a lot of ways of configuring this functionality. And if that is not enough, people can write their own extensions. But that’s where the good news stops – there isn’t exactly a lot of documentation on how to write extensions, how to configure them or how to use them. Plus just to complicate things, the code of CruiseControl.NET isn’t very well documented.

For the communications library I’m going to build up an SDK. This will include:

• The library itself (the API)
• A help document (both compiled and HTML)
• Examples (both code and help-based)
• Quick starts (processes to get someone up and running)

The idea behind this is to provide a knowledge base for a person new to CruiseControl.NET (although no new to programming) and help them to quickly build applications for CruiseControl.NET.

So as you can see, I have a vision that I’m aiming for with the new Communications library. Over time I’ll try and write up some more posts about this library. This library – together with the server extensions – opens up a whole world of possibilities for expanding CruiseControl.NET.

Stay tuned…

Throttling Build Integrations

My Server Can’t Handle It

Recently on the mailing group there was a request to limit the number of simultaneous build integrations. Well, the person actually was asking how to get an integration queue to fire off more than one request at a time, but the underlying intention was to limit the number of integrations. Now playing around with integration queues can be a risky thing (just trying to figure them out can be challenging!) so I thought I’d use the new integration events to add a build integration request throttler.

The principal behind the throttler is simple – it will limit the total number of concurrent builds on a server. This is completely independent of queues, so both queues and build throttling can be active on a server. Since not everyone will want this functionality, it must be optional, and it also should allow people to set the number of concurrent builds allowed.

Right, that’s the design brief, let’s start building it.

Extension Framework

First, a quick review of server extensions. These extensions are designed to plug in to CruiseServer and allow direct manipulation of CruiseServer (i.e. they can call any methods on it). They can also subscribe to a number of events that have been added to ICruiseServer, which now includes events for before and after an integration request is triggered.

All extensions must implement ICruiseServerExtension, which provides the following methods:

• Initialise()
• Start()
• Stop()
• Abort()

Initialise() is called when the extension is first loaded (i.e. when CruiseServer starts up) and can be used to subscribe to any events. Start(), Stop() and Abort() are called by the matching methods on CruiseServer and allow the extension to do processing at any of these points.

The Throttler Extension

I have added a new extension to the Core project called IntegrationRequestThrottleExtension. This implements the four methods required by ICruiseServerExtension – but it ignores the last three methods (i.e. does nothing).

The Initialise() method checks the configuration to see if the number of requests allowed is set (more on this later) and then subscribes to the IntegrationStarted and IntegrationCompleted events. So far, so good.

In terms of the actual throttling, I store a list of project names within the extension. When a new request is started the project name gets added to the list. If the list is over the limit, then the extension sets the result code to Delay, otherwise it sets the result to Continue. Likewise, when a request is completed it is removed from the list.

Since the server is multi-threaded I needed to add some locking code – otherwise the extension would get garbled data. Additionally, requests must be processed in order – so I added some code to prevent requests “jumping” the queue.

And with that throttling is now available for CruiseControl.Net.

Configuration

Now that the extension has been added, the next question is how to use it? The answer to this lies in configuration. In the application configuration (ccnet.exe.config or ccservice.exe.config) there is a <cruiseServer> section. The extensions need to be added to here.

The following is an example of how to add the throttling extension with a limit of three requests:

  1: <cruiseServer>
  2:   <extension type="ThoughtWorks.CruiseControl.Core.ServerExtensions.IntegrationRequestThrottleExtension,ThoughtWorks.CruiseControl.Core">
  3:     <limit>3</limit>
  4:   </extension>
  5: </cruiseServer>

The default limit is five – so if the <limit> element is omitted (it is optional) then only five can be run concurrently.

Final Note

At the moment the code for this extension (and the extensions framework) is all in the security branch. This is because we’re still waiting on the final release for 1.4 (1.4.3 is still being delayed!) and all this new functionality will be part of the 1.5 release.

The New Improved Validator

Some History

A little while back I wrote a post on the problems I was having getting the XML formatted nicely (read it here). The main problem was the code formatter I was using was slow

Despite my attempts I was unable to get any decent speed improvements, so I ended up moving it to a background worker. Now it was still slow, but it didn’t impact the user.

Moving Forward

One of the readers of my blog saw my post and suggested Scintilla.Net as an alternate. So I downloaded it and tried to use it, but without any success I hate to say it but the documentation on this control is very poor, so I ended up giving up.

Thankfully Patrick didn’t, and he was able to provide an example of how it should work. From here I was able to see my mistake (needed to turn the IsReadOnly off before setting the text) and now it works beautifully!

This is the new improved CCValidator:

Notice the loading time – only 2.28s (this previously was over a minute!) Also the display includes the expandable sections for the XML – making it much easier for browsing.

Many thanks to Patrick for not only suggesting this control, but helping me to get it working.

Speeding Up Validation

The Problem

The validator tool that I built recently works reasonably well. It has all the same validation options as both the console and the service (it’s built on top of the same code), and it allows someone to check all their projects in one go. Additionally the version in the security branch allows some of the extra features that I’ve added into that branch (e.g. internal validation).

But, there is one problem: as the number of projects increases the time to run the tool increases, and sometimes quite significantly! On my machine I find it tends to slow down after around 10-15 projects, and by the time I’m up to 50 projects I’m waiting over a minute – not so good

The Cause

Anyway, I started inserting some timing statements into my code to try and work out the slow point. And in the end I found it, but it wasn’t where I expected!

As some background, here is the sequence of processing:

1. Load and pre-process the XML
2. Format and display the original XML
3. Validate each XML element and write the results to the display
4. Pass each element through NetReflector to regenerate the XML
5. Format and display the processed XML

Each of the first four steps worked without any problems. For my config with 50 projects in it they all took around 2.5s to achieve. It is the last step, formatting the XML, that took the rest of the time.

Now to do the formatting I use the CSharpFormat library from manoli.net (http://www.manoli.net/csharpformat/) which normally works very nicely (this is the first time I’ve had an issue with it). Now I haven’t tried modifying the code, but I know it uses regular expressions under the hood to do the formatting, so there might be an issues with this

However, on further investigation it looks like there is more of an issue with the amount of XML that  NetReflector generates. Basically NetReflector generates XML for nearly every item in the configuration, plus it uses elements for everything, instead of attributes in places.

The Solution

Originally I wanted to par down the amount of XML that NetReflector generates. There are some approaches that can be used, such as eliminating empty elements (i.e. those with empty strings or empty lists) and changing to use attributes instead of elements.

The eliminating empty elements pared down the resultant XML from ~1800 to ~1600 lines, but this still took a while to format.

Next I tried switching to using attributes, but I kept getting run-time errors, so in the end I gave up on this.

I then added some DefaultValue attributes into the code and modified NetReflector to check for these, which saved me an extra 300 lines, but not enough to make a speed difference.

After this, I gave up! Obviously I’m not going to be able to trim down the generated XML enough to make a huge difference, when the issue obviously lies in the amount of time it takes to format the code. This left me with two choices – finding another code formatter or removing the formatting.

I had a look around, but couldn’t find anything that would be better, and I didn’t want to totally remove it, so in the end I settled for a compromise. I moved the XML formatting step into a background worker.

Once the validator has finished the first four steps I pass the XML into the background worker to do the actual formatting. Once this has finished it is displayed like it did originally.

The two main differences the end user will see is faster processing of the validation, plus when they attempt to view the processed XML they will get a message saying it is processing. Then once the background worker has finished the formatted code will be displayed.

Validating with Preprocessing

Oops, I Missed Something

Recently I added a validator for the configuration to CruiseControl.Net (full posting on it here). While this works nicely, and has a couple of nice features, I missed one important area – preprocessing!

Preprocessing was added in release 1.3 and allows the definition of preprocesser elements. These elements are blocks of configuration that are inserted into the configuration. As an example, a preprocesser element could contain the configuration for the e-mail publisher. Then, rather than repeating this configuration every time the e-mail publisher is needed, the config would point to the preprocesser element.

Preprocessing breaks the configuration loading into two stages: preprocessing and loading. The preprocesser literally goes through the configuration and generates a “standard” configuration out the other side.

Unfortunately when I wrote my validator I omitted this preprocessing step and just built a replacement for the loading stage. Happily I have now rectified this – and this post will cover the details.

How Configuration Loading Works

In the server configuration loading is handled by IConfigurationFileLoader, with a default implementation called DefaultConfigurationFileLoader. DefaultConfigurationFileLoader is responsible for loading the configuration file, generating an instance of the preprocesser and executing it, and finally handing it onto an instance of NetReflectorConfigurationReader to do the actual loading.

It also does a couple of extra bits and pieces like validating the XML and expanding DTD entities, which also should be included in the validator.

Now the actual preprocessing is done by ConfigPreprocessor, which looks like it uses XSL-T internally. But since this is all wrapped in DefaultConfigurationFileLoader I shouldn’t need to call it directly.

However, with the current state of the code there is a problem – DefaultConfigurationFileLoader expects an instance of NetReflectorConfigurationReader, and the critical methods in NetReflectorConfigurationReader  are not virtual (and therefore can’t be overridden!)

So, that leaves me with two choices:

1. Make my own implementation of DefaultConfigurationFileLoader in the validator
2. Modify DefaultConfigurationFileLoader or NetReflectorConfigurationReader in core

Some Minor Modifications

In the end I decided to go with approach two. I changed DefaultConfigurationFileLoader to expect an implementation of INetReflectorConfigurationReader (a new interface). This interface has one method (Read()) and one event (InvalidNodeEventHandler) – which map to the current public members of NetReflectorConfigurationReader.

With this in place I then modified NetReflectorConfigurationReader so it implements INetReflectorConfigurationReader, and the core changes are done.

Since DefaultConfigurationFileLoader has a constructor that automatically generates an instance of NetReflectorConfigurationReader, and a second constructor that now expects a INetReflectorConfigurationReader, everything continues to work as expected.

In the validator I had to make a couple more changes. First I changed MainForm so it also implements INetReflectorConfigurationReader. Then I changed the method names so they match the interface.

So far, so good – everything still works.

The final change is to modify the loading process. Now the loading process generates an instance of DefaultConfigurationFileLoader and passes in a reference to MainForm. Then, rather than calling the load methods directly, the code calls the Load() method on the DefaultConfigurationFileLoader instance, which in turn does all it’s fancy processing and passes the preprocessed configuration back to MainForm.

Simple

One Extra Enhancement

With preprocessing in place, the actual validated configuration can now look different from the original. To handle this I added a TabControl to the form and moved the XML viewer to this. I also added a second tab that displays the preprocessed XML.

This works by loading the configuration elements and then passing them back through NetReflector to regenerate the XML. While this strips all the comments it shows the actual XML that will be used by CruiseControl.Net (and more importantly it shows how NetReflector views it). Hence this means the processed XML will look slightly different from the original.

The following picture shows how the validator now looks:

And the contents of the processed tab:

One Final Word

I should mention two things:

1. These changes have been made to both the trunk (1.4.3 release) and the security branch (1.5 release). While the basic functionality is the same for both, the security branch has a bit more functionality in it (validation of security plus internal validation).
2. This changes occurred because of feedback to my initial post. I’m still learning my way through CruiseControl.Net and am by no means an expert yet. So continue to send in your feedback and I’ll continue to improve things

Builds, Installers and Frustration

A.K.A Why I Love Documentation

I recently put together a small tool to help diagnose errors in the configuration (i.e. ccnet.config). After getting some good feedback it on it, I decided it would be nice to add to the trunk, and even nicer if it ended up in the final installer (since this is what most people will use).

Like a lot of areas in CC.Net, there is no documentation on developing for the installer. But given that installers are reasonably simple, I thought I’d give it a try. And that is where my frustration began!

I should add as a side note, I’m a keen user of WiX. This is a set of open-source tools that take in an XML file and generate an installer out the other end. It is very simple to use (although I do have some problems with it) and has a great price tag – free!

CC.Net uses NSIS instead, which is more of a scripting tool for building installers. It is also free, but I find it a little harder to read and understand (of course I’m probably just used to WiX and find it hard to convert).

Anyway, back on topic, this post is about what I have learnt to get a new project included in the installer.

Where I Started

Now, since I knew that CC.Net uses NSIS I thought all the information for the installer would be in ccnet.nsi (this is the script file for the server). I was wrong

The file has all the instructions for installing the product, and nothing about the files to be included. I knew the names of the files currently included, but nowhere did I find these in the script file.

Thus I was lost!

First Things First: The Build

As we start out with code, not binaries, the first step is to generate the executables. CC.Net already has a pretty good build file that uses NAnt. This does all sorts of wonderful things, including compilation, unit tests, code coverage and most importantly for me – it generates the installer.

Again, it doesn’t have any lists of file, but it has three important pieces of information:

1. First, it has what is actually built and what parameters are passed in
2. Next it has when the installer is generated, and the build steps that must happen first
3. Finally it has a build step for generating a deploy folder

I’m going to shuffle the order of these around a little because I didn’t discover them in this order.

Instead, the first piece I found out was the order of build steps. The build step (or target) that generates all the installer is called “dist”. This depends on the deploy targets for CCTray, server and dashboard (there’s an intermediary target in between called “deploy”).

Now while “dist” is the actual build step that generates the installer, it needs all the files copied to a deploy folder first. This turns out to be critical because the .nsi file refers to this folder and builds the installer using the files in this folder!

Thus to include new files in the installer all I needed to do was modify the build step for generating the deploy folder (see important piece of information #3).

Where Do You Think You’re Coming From?

Looking at the existing executables I saw they came from the build folder, and where slightly re-arranged to be in the correct locations (plus there were some exclusions). However, when the executables for the validator went into another folder – the standard binrelease folder from C#.

Again I searched and searched through the build script to find the step that copies them – but I couldn’t find it anywhere.  Which brings me to important piece of information #1 – the parameters that go into the compilation.

It turns out that CC.Net projects have an extra build configuration called “Build”. This is very similar to the standard “Debug” and “Release” builds, but (and this is a big but) it sends the outputs of the build to a folder underneath the global build folder!

Once I had figured out these important key points, everything else fell into place!

A Recipe for New Projects

So, here are the steps for adding a new project so it is included in the installer:

1. Add a new build target to the project called “Build”.
2. Configure this project so it outputs the binaries to a folder underneath the global build folder.
3. Add some lines to the NAnt build script to copy the binaries into the correct location within the deploy folder.

And that’s it! It took me almost three hours to figure out these three steps

Addendum: Adding a Shortcut

It is also very easy to add a shortcut. This is covered in the NSIS documentation, so it didn’t take me long to figure it out.

In the .nsi script file there are a number of “Section” entries. There are three main sections:

• CruiseControl.NET Server (SEC01)
• Web Dashboard (SEC02)
• Examples (SEC04)

Each of these sections map to an option the user can choose. In these sections are the commands for installing the product, including creating short cuts.

So, to add a new shortcut, select the correct section and add a CreateShortCut command with the correct parameters (location and target).

Even simpler than adding files to the installer

Building a Better Validator

But My Configuration is Broken!

One of the common issues we see on the mailing lists is people having problems with their configuration. Normally it’s sometime along the lines of “Hi, I’m getting this error when the app starts, can you help?”. They also normally include the error message (when they are being helpful) and sometimes their config (when they are being extra helpful). But while we don’t mind being helpful, it can be frustrating to see similar errors occurring over and over.

Now, yes, I know the configuration for CruiseControl.Net can be very painful at times (I have spent many hours trying to work out why my config is breaking at times), but it is this way to allow maximum flexibility. There there is a configuration tool available (another source project – you can find it here), but sometimes this can be out-of-date (especially when we have just added new functionality).

So, I thought it would be nice to build a validator that goes through and validates the entire config file at once. There is a validate option on the console version, but this is text-based and stops at the first error found. In contrast I’m going to make this tool a bit friendly, plus hit the entire configuration.

And on a final intentions note, I’m going to use the current libraries as much as possible. This means when we change something in the configuration we don’t have to modify the validator as well (as much as possible).

The Design

First off, this will be a Windows application. While I could make it a web application, it would be harder, less user-friendly and need to be hosted somewhere (if the config is invalid, then the server won’t start and thus we can’t use the dashboard).

This will be a separate stand-alone application that will operate in the same directory as the server. Being in the same folder means it will pick up the same assemblies (including plug-ins) as the server – again with the idea of reducing the amount of work I have to do

This will be very simple – a single window application. When the user starts it will display a blank window with a menu bar and status bar. In the menus will be an “Open” option which will prompt the user for the file. When the user selects a file it will open, load and validate the file. The results will then be displayed in the window (using the WebBrowser control).

The actual validation will be done using the classes from the Core project. However, instead of throwing exceptions, it will display error details and move onto the next configuration item.

The Implementation

Starting the project was nice and easy, I just started a new C# Windows Application project and modified the main form to include my menu bar, status bar and web browser. I added a menu option for “Open” and added the code behind to display a dialog box prompting the user to select a file.

When the user selects a file, it first attempts to load it into an XmlDocument. If there are any errors it will display them in the browser. Otherwise it will check for the root element, and then validate each separate element under the root.

This is basically a duplication of the functionality in NetReflectorConfigurationReader (from Core). I couldn’t use NetReflectorConfigurationReader as it stops on the first error (by throwing an exception), so I did a cut and paste job instead. This means changes to NetReflectorConfigurationReader also need to be made in the validator, but this file doesn’t change too often.

To display the results I use a combination of an embedded HTML template in the project and manipulating the DOM. The HTML template provides the basic layout (including the styles), while the DOM manipulation gives me the dynamic results.

With this in place, I now have a very basic validator:

I also added a bit of eye-candy around the loading process (i.e. status messages and a progress bar). And with that a simple validator is now ready.

But, I wanted to make it more user-friendly…

Adding Eye-Candy

The first thing to add was the ability to reload files. This will just reload the currently selected file.

Next I added the ability to print the results, so the user can get a hard-copy (if desired).

Finally I added a history of the files opened. This now gives me the following menu options:

Additionally, I didn’t like jumping back and forward between the configuration and the results, so I added a read-only display of the configuration (plus allowed the user to select their view):

Finally I added some persistence to the application. It will persist the location of the application, plus which display settings were chosen.

Another Small Tool Completed

This completes another tool to help with using CruiseControl.Net. I know it’s not perfect, but it is better than the current version. At the moment, the main issue is sometimes messages from NetReflector are not very helpful – but there is not much I can do about this currently.

If I have more time I will look at adding some additional functionality in future (e.g. context-links, in-line editing for the config, etc.)

The binaries for this tool are at http://sites.google.com/site/automatedcoder/Home/CCValidator.zip. I can also apply to have this tool added to the trunk if it is liked (and so will be included in the main install for CruiseControl.Net.)

New Faces – Old Body

Looking for a New Face?

As anyone who has used CruiseControl.Net can tell you CCTray is one of the public “faces” of CruiseControl.Net. By a face I mean what the end-users see (the other one is the dashboard). CCTray is actively maintained by the developers and is constantly getting new features.

However, sometimes CCTray is not the answer CCTray is a Windows application with its own nice little GUI, but what about if we wanted to add it to the Vista sidebar? Or how about a plain console app so we could include it in batch files? Or what if we’re being adventurous and want to try WPF?

The good news is it’s not that hard to make our own client application. CCTray is really just an entry point to the DLL that does the real work – CCTrayLib. This library does all the work of connecting to a CruiseControl.Net server, retrieving updates, passing requests, etc. It also contains all the UI code, but we don’t have to use it if we don’t want to.

So, in this post I’m going to look at building a simple command-line interface that can be used to retrieve the status of a server or project, force or abort a build and start/stop a project.

Introducing CCCmd

CCCmd is the new project I’m adding. This will simply parse the command-line parameters and then call CCTrayLib to perform the expected function. Here is a high-level design of the application:

High level design

These are some example calls:

• cccmd retrieve -server:tcp://localhost -all
• cccmd retrieve -server:http://dashboard -project:”CruiseControl.Net (Live)”
• cccmd forcebuild -server:tcp://localhost -project:cc.net
• cccmd stopproject -server:http://dashboard -project:”CruiseControl.Net (Live)”
• cccmd help

The parameters are very simple. The first parameter is the name of the action. The current actions I’ll implement are:

• help: displays help on this command (default action)
• retrieve: retrieve the details on either a project or server
• forcebuild: forces a build on a project
• abortbuild: aborts a project build
• startproject: starts a project
• stopproject: stops a project

The remaining parameters just specify which project and server to use.

The first thing I did was quickly add a shell that parses the args and stores them in a known format. Then I added the help action (it just displays an embedded text resource). Finally I added a few helper methods for use later on. Now I’m ready to start integrating with CCTrayLib.

Retrieving Details

The first action to implement is retrieve. This is actually two actions, depending on whether a project is supplied or not. So, first thing, validate the parameters to ensure everything is correct. Now based on whether the project is supplied or not I can implement the action.

However it turns out the easiest way to implement this is to treat both requests as the same – but the server request displays all the projects, whereas the project request only displays the desired project. So, how is it done?

To retrieve the project statuses I’m going to use GetCruiseServerSnapshot() on ICruiseServerManager. To instantiate this I need a few other instances first. Basically the steps are:

1. Create an instance of BuildServer and set the URL (since I’m allowing remoting I play around with the URL a bit so the user only has to enter the protocol and server name, it will guess at the port)
2. Create an instance of ICruiseManagerFactory (this is actually a RemoteCruiseManagerFactory)
3. Create a new CruiseServerManagerFactory passing in the ICruiseManagerFactory instance
4. Call the Create() method on the CruiseServerManagerFactory instance passing in the BuildServer instance

Now I have a ICruiseServerManager instance that I can use. All I need to do is call GetCruiseServerSnapshot() and then display the required projects.

Performing Actions

All the rest of the actions are just a different method call – they use the same underlying routine to retrieve the correct project manager. Unlike retrieving details, a difference instance is needed, this time a ICruiseProjectManager. This requires a bit more work to instantiate. The steps are:

1. Create an instance of BuildServer (same as for the server)
2. Create an instance of ICruiseManagerFactory (again the same)
3. Create a new CruiseProjectManagerFactory passing in the ICruiseManagerFactory instance
4. Create a new CCTrayProject instance passing in the BuildServer instance and the project name
5. Create a new Dictionary<BuildServer, ICruiseServerManager> instance – add the BuildServer instance and a newly created ICruiseServerManager (I used my existing method above)
6. Finally call the Create() method on the CruiseProjectManagerFactory instance passing in the CCTrayProject instance and the dictionary

Once I have the ICruiseProjectManager instance, everything else is easy! Just call the correct method on this instance and voilá, I’m done.

Note: Some of the implementations of ICruiseProjectManager swallow communications errors, which means the application can’t tell whether the request succeeded or not. Not good!

Tidying Up

That’s pretty much all there is to this application. The only hard part is setting up the manager, and then everything else is easy! I’ve gone through and added some validation checks (server and project names are set, -all not called on the project actions), plus added some diagnostic messages. I also added a ‘quiet’ mode so the console dosn’t get cluttered if it is being consumed from another application. Plus I set the exit code if something is wrong (1 = invalid parameters, 2 = unexpected error).

This was a fairly simple project to do – it only took me a couple of hours. The only down side that I’m aware of is CCTrayLib comes with all the UI stuff for CCTray in it – so the DLL is larger than it would be if it was a purely worker DLL, but that’s not too bad.

The source code for this project is here and the binaries are here. If there is enough support for this I can apply to have it added to the trunk for CruiseControl.Net.

Enjoy

Monitoring CC.Net

Are You Sure CC.Net is Running?

With our modern 24/7 world we often expect things to be constantly up and running. However, as any sysadmin would tell us, expectation doesn’t always match reality.

Recently I had a problem where my CC.Net server crashed (well, it was my WCF code that crashed it, but it was still a crash). Now I assumed it was all running happily, made several check-ins to our code base and carried on working knowing that CC.Net would alert me to any problems. However since CC.Net was down I didn’t get any warnings – it wasn’t until I went to get the deployment files that I realised it was down.

That was when I thought wouldn’t it be nice to know when CC.Net isn’t running. Now at my work we have a monitor server that will send out notifications if a server (or service) goes down. One of the mechanisms for this is the server sends out a message telling the monitor it’s still happy (or that it’s unhappy but still running). If the monitor doesn’t receive a message within a certain timeframe then it sends out notifications.

This monitor server also tries to make things easy for us developers. We can tell it our service is running by passing in a simple HTTP request. So I thought this would be a fun mini-project using my new extension points in CC.Net.

ServiceMonitor

I quickly started up a new project and added a class that implemented ICruiseServerExtension. Since all I wanted to do is send out a message every time period (I choose one minute) I added a Timer (from System.Timers) to the class and attached the Elapsed event to a handler. Then all I had to do was get the Start()/Stop()/Abort() methods to start/stop the timer and I was done!

I initialised the new timer in the Initialise() method (including attaching the handler and hard-coding the interval). All the handler does is send a request off to the monitor server (again hard-coded, I was feeling lazy this morning).

The End

Once I compiled the code I deployed it to my test CC.Net and left it running there. It seems to be working quite happily. I then configured our monitor server and did some simple testing. I tried killing the service and the monitor sent me a notification. I left the service running for a while – no notifications. Now I’ll just leave it running for a week or so and if there are no issues I’ll deploy to our prod CC.Net.

Nice and easy!

The Code

Since I have no immediate plans to add this to the CC.Net code base (since the values are hard-coded), here is the complete source for my extension:

using System;
using System.Timers;
using System.Net;
using ThoughtWorks.CruiseControl.Core.Util;
using ThoughtWorks.CruiseControl.Remote;

namespace ThoughtWorks.CruiseControl.ServiceMonitor
{
   /// <summary>
   /// Adds an extension to send the server status to a monitor application using HTTP.
   /// </summary>
   public class ServiceMonitorExtension
      : ICruiseServerExtension, IDisposable
   {
      #region Private fields
      private ICruiseServer _cruiseServer;
      private Timer _monitorTimer;
      #endregion

      #region Public properties
      #region IsRunning
      /// <summary>
      /// Gets whether this extension is currently active.
      /// </summary>
      public bool IsRunning
      {
         get { return _monitorTimer.Enabled; }
      }
      #endregion
      #endregion
 
      #region Public methods
      #region Initialise()
      /// <summary>
      /// Initialises the service host to use.
      /// </summary>
      /// <param name="server">The CruiseServer that is initialising this extension.</param>
      /// <param name="extensionConfig">The configuration for the extension.</param>
      public void Initialise(ICruiseServer server, ExtensionConfiguration extensionConfig)
      {
         // Validate the input parameters
         server.ValidateNotNull("Cannot pass in a null server");

         // Store the parameters that we need for later
         _cruiseServer = server;

         // Initialise the timer
         _monitorTimer = new Timer(60000);
         _monitorTimer.Elapsed += new ElapsedEventHandler(CheckServerStatus);
      }
      #endregion
 
      #region Start()
      /// <summary>
      /// Starts the monitor timer.
      /// </summary>
      public void Start()
      {
         CheckServerStatus();
         _monitorTimer.Start();
      }
      #endregion

      #region Stop()
      /// <summary>
      /// Stops the monitor timer.
      /// </summary>
      public void Stop()
      {
         _monitorTimer.Stop();
      }
      #endregion

      #region Abort()
      /// <summary>
      /// Stops the monitor timer.
      /// </summary>
      public void Abort()
      {
         _monitorTimer.Stop();
      }
      #endregion

      #region Dispose()
      /// <summary>
      /// Make sure everything is closed.
      /// </summary>
      public void Dispose()
      {
         if (_monitorTimer != null)
         {
            _monitorTimer.Stop();
            _monitorTimer.Dispose();
            _monitorTimer = null;
         }
      }
      #endregion

      #region CheckServerStatus()
      /// <summary>
      /// Checks the status of the ICruiseServer and tells a listening server (event handler signature).
      /// </summary>
      /// <param name="sender"></param>
      /// <param name="e"></param>
      private void CheckServerStatus(object sender, ElapsedEventArgs e)
      {
         CheckServerStatus();
      }

      /// <summary>
      /// Checks the status of the ICruiseServer and tells a listening server (event handler signature).
      /// </summary>
      private void CheckServerStatus()
      {
         Log.Info("Sending server status");

         // TODO: Figure out how to check whether the ICruiseServer is running
 
         // Tell the remote server we're still running
         WebRequest request = HttpWebRequest.Create("http://monitorserver/monitor.aspx?app=ccnet&status=active");
         try
         {
            WebResponse response = request.GetResponse();
            response.Close();
         }
         // If the remote server is down we don't want to crash CC.Net as well!
         catch (Exception) { }
      }
      #endregion
      #endregion
   }

}