Documentation Has Arrived

Some New Stuff, Some Old Stuff

Like any system, documentation is an important part of CruiseControl.NET. With the 1.5.0 release, we’re adding a lot of new functionality – all of which needs documentation. So I’ve taken some time out of resolving issues and adding code coverage to document some of the new things that are coming.

As well as documenting the new stuff, I’ve also been updating some of the older documentation, although this is mainly around adding what has changed. Some of our changes has had a major effect on all the tasks and publishers. But I’ve tried to also add some value to the documentation around versioning.

Finally, I’ve also been trying to add some more contextual documentation. A large part of the current documentation describes the various configuration options, and what can be set. We also have some stuff on using the system, including with other tools and components, and a little bit of development documentation. Basically, I’m trying to fill out the last two sections a bit more!

What’s New?

We have lots of new stuff for 1.5.0, some of which has been requested for years! In terms of configurable items, we have added the following:

• Security (big change)
• Dynamic values and parameters (wide-ranging change)
• New tasks and publishers (parallel, sequential, conditional, NCover, PowerShell and FTP)
• New source control providers (RoboCopy, VSTS, FTP, Git and Mercurial)

Most of which have now been documented (just the RoboCopy and VSTS source control blocks outstanding).

Before I move on, a quick comment on the security. This is a very big change, and one that has a lot of associated documentation. I’ve gone through and added the basics on all the configuration options, but as Ruben pointed out, the sheer volume of configuration for this area can make things very confusing. As such, I’ll be going through and adding more over time.

What’s Updating?

One of the widest-ranging changes is the additional of dynamic parameters. Nearly every single task and publisher can use them! As such I’ve gone through and added this to all the tasks and publishers that can use them.

Additionally, we have an issue where we have only one documentation wiki – which has to handle all versions of CruiseControl.NET. To try and make sense of what works in various versions, I have added two additional items.

First, all new items have a Version section. This section says when the item was first added (e.g. 1.4.4, 1.5.0, etc.) This should help people who want to try a new item, but they are still using an older version. Of course, there are a huge number of items that have been around for a long time – these I have not documented. Basically, if there no Version section, then it will work in 1.4.0, not sure about older versions!

Second, to the configuration elements table I have added a Version column. This works in the same way as the Version section, it says when the element was first added. Again, since there is a lot of old stuff, not everything will have a version number. Perhaps if I am bored one day…

What’s Contextual?

Since both security and dynamic values and parameters are new, I have tried to explain what they are for, how they work and what some of the possibilities are. For security, I’ve started adding some scenarios of how it can be used. Actually, at the moment there is only one, but there are two others in this blog that I need to update and migrate over.

Finally, I’ve also started writing about how security can be extended. When I designed it, I added all sorts of interfaces for different things. So, if you don’t like what I have done, it should be easy enough to swap out a part and use your own implementation.

What’s Missing?

While I’ve added a lot to the documentation, I don’t consider it anywhere near finished yet!!! here are some of the things I would like to see done around documentation:

• Review everything: a lot of the configuration documentation is inconsistent – not surprising considering the number of people who have worked on it over time – it would be nice to go through and put together a consistent style (e.g. what is included, etc.) through the entire documentation.
• Client apps: there is some documentation on these, but a lot of it is out of date, especially considering the number of things that have changed.
• Developer documentation: my first few months on CruiseControl.NET was just trying to figure out what is what, heck, even now I am still confused at times!

Anyway, these are just my thoughts, at least we have some documentation!!!

So, in closing, please take a look at some of the new documentation and let me know what you think. As always, I’m too close to the trees to see the forest

Migration Woes

Oops, We’ve Broken Some Eggs

As we slowly move forward and add new features or fix broken features, it is inevitable that we make some breaking changes. Sometimes it is possible to work within the current structures and file formats, other times it is either inefficient, very painful or downright impossible. So, where possible we don’t make breaking changes, but sometime we have no choice.

While this is a common feature of software development, this can still cause pain to our users. Generally, people don’t like to think, even less so when it is painful thinking! And often, people just don’t read the upgrade notes – they just expect it to work.

Moving to 1.5.0 of CruiseControl.NET, we are making some major changes to the underlying storage locations. This is because all older versions assume the Program Files is readable and writable! Unfortunately, with Windows Vista, Windows Server 2008 and newer versions, this is no longer the case. We’ve been locked out! So, if we want to write data to disk, we need to move to a new location.

While this is a minor change for new users, it does have a major impact on existing users. They will need to move all their files that are stored in Program Files to the new local (e.g. ProgramData, All Users, etc.) Knowing how people don’t read the upgrade notes, we’re going to get a lot of queries about this.

A New Wizard

While I know we’re not going to be able to totally avoid the pain, I’d like to reduce it as much as possible. Originally I had added a check to the dashboard to enforce the migration, but this was disliked. So, instead I’ve added a new WinForms application to handle the migration.

At this point I should mention, this wizard is still in the very early days of development! DO NOT USE it for production systems, you have been warned!!!

This wizard will provide a step-by-step process for collecting user details on what they want to migrate. Once this has finished, it will migrate everything for them. If it runs into any errors in the migration process, it will (theoretically) roll-back any migration changes.

So, that’s the theory, here’s what I have put together so far.

Following the Yellow Brick Road

When the user starts the wizard, they will get a general introduction window. I’ve tried to put relevant information on it, and hopefully not too much information.

Clicking on next will prompt you to enter which version you are migrating from. Currently, I only handle migrating from 1.4.4 to 1.5.0, but I plan to add additional migrations from older versions (up to 1.4.0). This is because we added a number of breaking changes through the various versions, so it would be nice to allow the wizard to handle all of them. Hopefully, this will also encourage users of older versions to upgrade.

Once the user has selected the version, we enter into the meat of the settings. First off, the user has the choice of just which items they want to migrate. Here, they are being prompted if they want to migrate the server settings. The wizard also asks where the user wants to migrate from, since they may have installed in a different location (I’m not aware of an easy way to find the location, and I don’t want to try and scan for it!)

Migrating the server settings will move any state files, the overall projects state file and any project folders. If the working or artefact folders are not set in the config, they are added under the server folder, which is why they need to be moved.

Each page has validation checks built in. In this case, since we are asking for a folder location, it will validate the folder is valid and that it contains a ccnet.config file (this is needed later on).

Next, the user will be prompted if they want to migrate the configuration settings. This will change ccnet.config so it works in 1.5.0. Sometimes, but not very often, we are forced to change the layout of ccnet.config, this will handle the changes automatically.

On this page, the user is also prompted if they want to backup their original configuration. This is because the wizard will actually modify the file (as opposed to just move it), so they might like the safety of seeing their original file is preserved.

Now for the bad news, I’m not actually doing anything with these settings yet. But I know we do have some breaking changes, I just need to track them down and then add them to the wizard.

Next, the user is asked if they also want to migrate the web dashboard. A lot of times, the web dashboard is installed on the same machine as the server, but sometimes they are on different machines. Here we give the user the choice of whether to migrate both, one or the other.

Again, the user is prompted for the current location – they might have installed it to a non-default folder. The wizard will default to the standard location, but as you can see in my example, it is easy to change (I don’t have CruiseControl.NET installed on my machine, I have several development copies of the source instead.)

The user is then asked to confirm the settings. Currently it is just a text blob with all the details in it, I might see if I can figure out a better way to display it.

When they click on next, it then begins the migration. This is done on a background thread, so the user can still interact with the wizard. In time, I’d like to add the ability to cancel a migration part-way through (this would perform a rollback), but I haven’t gotten around to it yet.

Also, if you noticed the (Not Responding) message on this title bar, it’s just because the process is so fast on my machine, the only way I could take this snapshot was putting a breakpoint in the code! Normally it does respond (and very nicely).

Finally, the user will told what has happened. I’m trying to keep this very minimalistic – just the statuses for each item, plus any warnings or errors.

If the user wants to see the full log, this is available by clicking on the View Log button:

This is pretty much all the messages that are displayed in the progress window.

Welcome to the Wizard

So, that’s a quick tour of the wizard from start to end, at least in terms of its UI. The next question is what does it actually do?

Currently, it will move the following folders and files:

• Project working and artefact folders (if under Program Files)
• Project state files
• Overall project states (started/stopped statuses)
• Web dashboard configuration
• Web dashboard packages

I am also going to look at moving the log files, but this also require modifying the app.config files,so I want to be able to do this in a reversible manner.

I will also update ccnet.config to handle any breaking changes, but I need to find out what these are first.

And finally, I need to go through the old change logs to see if there are any other breaking changes, and then add them in.

Additional Features

I’ve also tried to build this with a few additional features in mind, but I can’t guarantee that I will add them. These are:

• Command-line execution (i.e. no UI)
• Cancelling migration part-way through migration
• Saving the log file
• Migrating to a different location

Plus whatever other people want to suggest…

So, this is what I am currently working on. I’ll add it to Subversion, so anyone who is brave can try it out, but I won’t add it to the installer until it is nearly complete.

Let me know what you think about the wizard.

Fun With Leases and Lifetimes

Yet Another Issue

One of the features we added to the 1.4.4 release of CruiseControl.NET was the ability to hot-swap the DLLs. This means you can xcopy new DLLs in over top of the existing DLLs and CruiseControl.NET would play nice, i.e. just reload itself with the new DLLs.

Unfortunately, after this enhancement we started hearing of people having problems with stopping the service. It appears that people would attempt to stop the service using the SCM, but it would fail. They would get an error message telling them a problem happened, and the service would continue running. Not very good

Anyway, this week-end I spent some time tracking down the issue. Actually, it was more like ten minutes here, another ten minutes a few hours later, and so on…

Normally, this would be a disruptive process, but today, it actually solved the problem!!!

Cross-Domain Calls

In order to get the hot-swapping working, we fire up a second AppDomain in CruiseControl.NET and load all the DLLs there. The reason this works, is the DLLs are shadow-copied (more details on this are available here). This is where the fun starts – in order to make this work, we needed to make some cross-domain calls. The following diagram shows the two AppDomains and the calls needed:

Basically, CCService starts up in the primary AppDomain. It starts up a second AppDomain and instantiates an instance of AppRunner in this domain. AppRunner then interacts with CruiseServer – which is one of the classes in the DLLs. CruiseServer then does all the real work.

Now, this part all worked pretty well – the problem comes with the cross-domain calls between the two AppDomains.

Introducing MarshalByRefObject

In order to make cross-domain calls, the called class must inherit from MarshalByRefObject. This class abstracts all the logic of making cross-domain calls, so we don’t need to worry about them.

However, to solve this problem, we do need to know a little bit about them. Basically, MarshalByRefObject generates a couple of proxies – one on each side:

When either side needs to talk with the other, they go via these proxies. These proxies then handle the communications between the two AppDomains (since this is handled by .NET, I won’t go into how this is done). What is important to know, is there is no direct referencing between the two instances. Instead CCService holds a reference to the AppRunner proxy, and vice-versa for AppRunner to CCService.

What does this mean for our problem?

The garbage collection in .NET works by checking if there are any references between instances. If there are, it checks to see if any of the referenced instances are active, and so on. If it checks all the references for an instance and finds no active instance instances, it garbage collects the instance.

Now, as I understand it, garbage collection only works within an AppDomain (or at least it functions that way). So, the garbage collection for the primary AppDomain knows about CCService, but not AppRunner, and vice versa for the secondary AppDomain. Which causes an issue for garbage collection, since there are cross-domain objects.

But, there is no actual reference to the cross-domain object, instead it is only to the proxy instances. This means, the two AppDomains don’t know when to garbage collect these objects!

Leases to the Rescue

To get around this problem, each proxy has a “lease”. This is like a property lease – the two sides have agreed to keep the lease “active” for a certain period. After this period is up, either side can clean up.

Now, by default, a MarshalByRefObject instance has a lease period of (I think) five minutes. This means, after the proxies have been active for five minutes, they can be garbage collected (although this can happen later).

So, that’s the background, hopefully by now you’ve figured out what has happened. AppRunner inherits from MarshalByRefObject. When the cross domain calls are required, it set up the proxies automatically for us, and then generated a five minute lease. The initial calls work fine,but after a while garbage collection comes along (after the five minutes), sees the lease has expired and so cleans up the proxies!

Sometime after this, someone decides to shut down CCService. CCService receives the call, and tries to pass it onto AppRunner – expect the proxies in-between have been cleaned up and no longer exist! Poor .NET gets confused and just spits the dummy

Now, during development and testing, we didn’t detect this. Why? Because when we did our testing, we’d fire up the service, make the various calls, and then shut down the service. And normally, this was all done before the proxies were cleaned-up, so the issue never raised its head.

After the week-end, because my testing was interspaced with gaps, I suddenly came across an error when I returned from a break and tried to shut down the service. Basically, the error was telling me a reference couldn’t be found – and that’s when it clicked – leases!

And Finally, a LifetimeService

So, to round off this post, there is a very simple solution – we just need to extend the leases. This is done by a LifetimeService in AppRunner – the actual method to do this is InitializeLifetimeService().

Now, we could just set a longer time-out period for the lease, but considering some CI servers run for a very, very, very long time between shut-downs, how would we know what it should be?

Instead, there is an alternate approach – disable the LifetimeService altogether. Doing this has the effect of setting infinite leases or leases that only expire when both sides shutdown. To do this, we merely return null from InitializeLifetimeService(). Simple!

Why didn’t we do this earlier? Well, we do for other MarshalByRefObject-inherited classes, like the remote cruise server, etc. It was just forgotten in this case

Anyway, problem now fixed. In the next release of CruiseControl.NET, we won’t have to worry about it.

Now, in future, we just have to remember to always check the lease on any MarshalByRefObject-inherited classes, but that’s a task for another day…

Time Zone Mayhem

We just resolved an interesting problem tonight with some unit tests in CruiseControl.NET. But before I get to the resolution, here is some background.

We have a number of developers around the world. For example, I live in New Zealand, which is UTC+12. Some of the other developers live in Europe, which is UTC+1 (although it is currently UTC+2 due to daylight savings). And just to confuse things, our build server – CCNetLive – is in Chicago, Illinois, UTC-5 currently.

One of the developers had written some unit tests that was checking some date/time comparisons in the underlying code. When he tested it on his machine, it worked beautifully – no problems whatsoever. Same for my machine, but CCNetLive was failing the test!

So, what was happening?

The code was parsing a date/time in a string and then checking to see if it was in a valid date/time range. The string included the time zone (2009-06-13 10:37:42 +0000), which was UTC+0.

Since the time zone was UTC+0, he generated a date using a UTC date/time kind – new DateTime(2009, 06, 13, 10, 00, 00, DateTimeKind.Utc). This date was the lower bound, while the upper bound was DateTime.Now.

Here is where things got interesting – he assumed, and I did also, that since the string contained the time zone that it would be a UTC date/time. Therefore, the UTC date/time would work nicely. And it did – on any machine that was ahead of UTC. On CCNetLive, which was behind UTC, it was failing!!!

Why?

DateTime.Parse generates a Local date/time, no matter what the incoming string has. If the string contains the time zone, it uses this to generate a date/time and then moves it to the local date/time. So, the string that was being parsed was being converted to 5:36:42am, instead of 10:36:42am. This was then being compared to 10:00:00am and naturally failing.

This is where it got us – the 10:00:00am date/time was UTC, and the 5:36:42am was Local. Shouldn’t the .NET date/times automatically convert to the same type in order to perform the comparison?

It turns out, they don’t. They just literally compare the date and time components, without taking into consideration the different date kinds.

So, I don’t know whether this is a bug or an oversight (or even deliberate), but it is worth while making sure you are comparing the same date/time kinds when doing a date/time comparison. Otherwise, like us, you’ll end up wondering why things aren’t working in different places around the world.

So what was the resolution to our issue – we converted the UTC date/time to a local date/time using .ToLocal(). After this, the test worked beautifully!

Building the Basics: Towards a Better Task

It’s All Changing!

I was chatting with one of the other developers on the CruiseControl.NET project yesterday. During the chat he mentioned he was concerned about how much things have changed from 1.4.4 to 1.5.0, and that he was having a hard time keeping up with all the changes.

So, in this post I’ll cover one of the areas that has recently changed – TaskBase.

Warning: This is a CruiseControl.NET development topic. If you don’t care about developing tasks for CruiseControl.NET, then this post is not for you!!!

Some History

In CruiseControl.NET up to version 1.4.3, a task could only be asked to do one thing – run. You could not ask it to do anything else. This lead to a number of assumptions and limitations. As an example, it was assumed that if NetReflector could load the task from the XML, it was valid. Another example, if a task completed without an exception, then whatever the status of the result would be the status of the task (definitely not a valid assumption for publishers!)

Basically, there was no way for the system to interact with a task beyond asking it to run.

Now, this is because a task must implement ITask. This interface has only one method, Run(), so the system is limited to calling this method.

So, how then can the functionality of a task be expanded? The answer was to add additional, optional interfaces to a task. The system checks to see if these interfaces are present when it needs them, and then calls the methods on those interfaces. This allowed tasks to expose new functionality, without breaking previous tasks.

An example of this is IConfigurationValidation. This was a new interface added in 1.4.4 that would allow a task to be validated after it has loaded. After the configuration was loaded, the configuration would be checked to see if any of the loaded classes had this interface. If so, it would be called, and the class could validate itself. Nice and simple.

An Explosion of Interfaces

In 1.5.0, we added a couple more interfaces to allow even more system/task interaction. These interfaces are IParamatisedTask and IStatusSnapshotGenerator. IParamatisedTask allows dynamic build parameters (i.e. values that the user can select when the task is built, as opposed to be hard-coded in the configuration) and IStatusSnapshotGenerator allowed for individual task statuses (prior to this, you could only see the status of the overall build).

If task now wants to validate itself, accept parameters or generate a status item, all it has to do is implement the required interface.

But, this is where it started to get messy!

Nearly every task in the standard build would like to have parameters, plus people like seeing what is happening inside their builds. And most of the time, this functionality is the same, no matter which task.  As you can guess, this would lead to a lot of duplicated code (never a good thing).

A New Base

There are a couple of approaches we could use to minimise the duplicate. We could add a helper class that handles the duplicate code, but then we’d still need to add the interfaces and methods to all of the tasks. An alternate approach is to make an abstract base class that tasks can inherit from.

So, as you might have guessed, I went with the abstract base class approach and added a new class called TaskBase.

This class implements ITask, IParamatisedTask and IStatusSnapshotGenerator. There are default implementations of each of the interfaces. These implementations are virtual, so they can be overridden.

Additionally, TaskBase provides a number of other common items. It exposes Description and Name properties (Description comes from the config, Name defaults to the class name). It includes a default skeleton for the Run() method, the actual functionality must now be implemented in the Execute() method.

Finally, it adds a method called PreprocessParameters(). This method uses a recent extension to NetReflector that allows the XML of a class to be pre-processed before it is loaded by NetReflector. This allows us to convert $[…] blocks within a property into dynamic value definitions – thus providing a simpler way to configure tasks.

Multi-task Tasks

We have actually added two new base task types to CruiseControl.NET – TaskBase and TaskContainerBase. TaskContainerBase is a further specialisation of TaskBase. Given that we have TaskBase, why do we also need TaskContainerBase?

In CruiseControl.NET we have added some task-containing tasks. These tasks contain other tasks (child tasks). Again, these tasks have a number of common functionality areas. All of these tasks need to validate (just in case any of their children require validation), they need to handle parameters (same reason) and also need to generate status.

So TaskContainerBase was born.

The functionality that this class provides is running child tasks (including applying parameters), validation and status generation. It is up to the actual task implementations to define the order of running the tasks. For example, ParallelTask runs all the tasks at the same time (using threads) while SequentialTask runs them one after the other.

Finally

Hopefully this shows how useful the new TaskBase and TaskContainerBase classes can be. We don’t have to worry about breaking existing 3rd party tasks by changing the underlying interface, and at the same time we’re not duplicating lots of code.

If you are developing tasks for CruiseControl.NET, you can continue to use ITask, with or without the extra interfaces. However, if you’re feeling lazy, TaskBase and TaskContainerBase will save some time, plus ensure your tasks utilises the new functionality we will be adding in future.

Inner Workings: The Anatomy of the Dashboard

An Introduction

On the mailing list there was a question today about the parts of the dashboard and how they match up with the configuration. So I thought it’s time for me to write down what I know about the parts of the dashboard and how they work.

First, there are four basic levels to the dashboard. These levels are:

• Farm – the areas of the dashboard that do not relate to any build server
• Server – the overall summary of a build server (e.g. an instance of the CC.NET service or console)
• Project – a project on a build server
• Build – an instance of an integration for a project on a build server

Each level has its own set of plug-ins, as would be expected since they show very different information. To show these levels, here is a screen shot of each.

Note: These are running on my development instance of CC.NET, hence they don’t have a proper version number or very many plug-ins installed.

Farm Level View

Server Level View

Project Level View

Build Level View

It is important to know about these levels, because putting a plug-in in the wrong location will put items in an unexpected place!

The Parts of a Page

Every page has four basic parts – and a plug-in only has control over one part. These four parts are:

The header and the footer are both generated by the main page template. These contain common items to every page, such as the name and version of CC.NET, common links, breadcrumbs and when the date/time the page was rendered. These do change slightly depending on the level, but this is outside the control of a plug-in.

The side bar contains a set of links. Again, these vary slightly depending on the level, but they are controlled by the system again. However, these links are defined in the configuration – each plug-in exposes one or more links to add to the side bar. More on this in a little while.

Finally, there is the content area. This is entirely up to the plug-in to populate. The system completely ignores the content area and assumes the plug-in will generate meaningful content. This content is accessed by clicking on one of the links in the side bar.

Overlying these parts onto a farm level view looks like the following:

Where the Config Comes In

Hopefully by now, you will have some ideas of where things fit. But let’s make things perfectly clear and delve into how the configuration relates to the pages.

First, here is the configuration I used to generate the above screen shots:

  1: <dashboard>

  2:   <remoteServices>

  3:     <servers>

  4:       <server name="local" url="tcp://localhost:21234/CruiseManager.rem" allowForceBuild="true" allowStartStopBuild="true" backwardsCompatible="false" />

  5:     </servers>

  6:   </remoteServices>

  7:   <plugins>

  8:     <farmPlugins>

  9:       <farmReportFarmPlugin />

 10:       <cctrayDownloadPlugin />

 11:       <administrationPlugin password="********" />

 12:     </farmPlugins>

 13:     <serverPlugins>

 14:       <serverReportServerPlugin />

 15:     </serverPlugins>

 16:     <projectPlugins>

 17:       <projectReportProjectPlugin />

 18:       <viewProjectStatusPlugin />

 19:       <latestBuildReportProjectPlugin />

 20:       <viewAllBuildsProjectPlugin />

 21:     </projectPlugins>

 22:     <buildPlugins>

 23:       <buildReportBuildPlugin>

 24:         <xslFileNames>

 25:           <xslFile>xsl\header.xsl</xslFile>

 26:           <xslFile>xsl\modifications.xsl</xslFile>

 27:           <xslFile>xsl\NCoverSummary.xsl</xslFile>

 28:         </xslFileNames>

 29:       </buildReportBuildPlugin>

 30:       <buildLogBuildPlugin />

 31:       <xslReportBuildPlugin description="NCover Report" actionName="NCoverBuildReport" xslFileName="xsl\NCover.xsl"></xslReportBuildPlugin>

 32:     </buildPlugins>

 33:     <securityPlugins>

 34:       <simpleSecurity />

 35:     </securityPlugins>

 36:   </plugins>

 37: </dashboard>

Like I said, it is very simple. The part that we are interested in the the <plugins> section. This defines all the plug-ins that can be seen. For the moment, I’m going to ignore the <securityPlugins> as this is specific to 1.5.0 and I’ve already covered it in a previous post.

There are four sections of plug-ins: <farmPlugins>, <serverPlugins>, <projectPlugins> and <buildPlugins>. These map to each level in the dashboard. Within each section there are one or more plug-ins. These plug-ins define the links that appear in the side bar. For example, in the <farmPlugins> section:

• <farmReportFarmPlugin> maps to the “Farm Report” link
• <cctrayDownloadPlugin> maps to the “Download CCTray” link
• <administrationPlugin> maps to the “Administer Dashboard” link

Most plug-ins have these link titles as hard-coded values within them – so they can’t be changed. Clicking on a link will pass control to the plug-in, which generates the content to be displayed (this is actually a simplification, but it will do for this post.)

Normally, there is only one instance of a plug-in per section – having multiple plug-ins generates some unexpected results, so it is recommended against. However, some rules are made to be broken.

Breaking the Rules – <buildPlugins>

The above details cover most plug-ins in the dashboard. However, the one area where the rules get broken is in the <buildPlugins> section. This section defines two special plug-ins – <buildReportBuildPlugin> and <xslReportBuildPlugin> (1.5.0 will be adding a third – <htmlReportPlugin>).

First, the <xslReportBuildPlugin> element. This breaks the rules by allowing multiple instances. It does this because of two properties: description and actionName. The description is the text of the link to appear in the side bar, while the actionName is the command name to be passed to the server. The actionName MUST be unique, otherwise the poor dashboard will get confused!

This plug-in takes in a XSL-T template and transforms the build log for a project into an HTML report (which is the third parameter in the element). This means we don’t need to develop lots and lots of plug-ins (e.g. one for each report), instead we can just write a style sheet and get it to transform the results.

The second rule breaker is <buildReportBuildPlugin>. This is a required plug-in and there can only be one instance of it. The reason it is different is it has an <xslFileNames> section in it. This section is similar to the xslFileName attribute in the <xslReportBuildPlugin>, but it has one major difference. The <xslReportBuildPlugin> generates a link in the side bar, the <xslFileName> element doesn’t. Instead, it’s transform gets merged into one big page – the “Build Report”.

The following picture shows how these relate:

Clicking on an <xslReportBuildPlugin> will generate a completely different content area, one that is not affected by <buildReportBuildPlugin> at all.

Summary

There are four general levels – farm, server, project and build. Each has its own config section and allows a different set of plug-ins.

Within each page, there are four areas – header, footer, sidebar and content. The plug-in generates the content area and defines links to go into the sidebar – everything else is handled by the system.

For the build level, there can be multiple <xslReportBuildPlugin> – each defines a link in the side bar, with custom content. The <xslReportBuildPlugin> section defines the items to appear within the “Build Report” – these do not appear as links within the side.

Hopefully this provides a better understanding of the parts of the dashboard.

Parallel and Sequential Tasks

That’s an Old One

Now that 1.4 is officially finished, it’s time to start gearing up for 1.5. Already we have a large range of improvements (security, dynamic parameters, communications improvements, etc.) However, while these improvements are nice, we also need to review any outstanding issues to see what we can resolve.

One of my pet dislikes is patches that get ignored! People have put an effort into enhancing CruiseControl.NET and then nothing happens. Personally I think this is very sad, although now that I have dev rights to Subversion I can understand how this happens. That aside, I decided to spend some time reviewing the old issues to see what patches can be applied.

Back in August of 2007 Patrick Boyd submitted a patch to allow tasks to run in parallel (CCNET-964). This task is fairly straight-forward – it has an array of child tasks to be run. When the task is run, it starts up a new thread and fires off each task. So, I thought I’d go and commit it (especially as there are five votes for it).

Some Revisions

While the original patch “works”, there are some issues it does not take account of. The one that got me is the way it handles the results from a child task – it just hands the parent result to each child. Unfortunately, this can cause issues as the result is not guaranteed to be thread safe.

So I when through and revised the patch and (hopefully) improved it. So, here is what I have changed:

1. Changed to use ThreadPool for generating the threads. I changed to this to help manage the number of threads used by CruiseControl.NET. Already we fire up one thread per project (nasty!) and now we’re firing up more threads for parallel tasks. Using the ThreadPool helps ensure we don’t overload the system (too much!)
2. Generated an instance of IIntegrationResult for each task. This is actually a clone of the parent result, when all the tasks have finished running, the main task goes through and merges all the results (this is actually implemented in IntegrationResult). This guarantees the thread safety of the results.
3. Added some error handling. When a child task now runs, I wrap it in a try/catch block. I did this because I got some weird results in the unit tests – mainly because the ManualResetEvent instance for the thread was not being set! Adding the try/catch block helps with this.
4. Generated lots of log entries. Well, not that many, but enough to hopefully show what is happening within this task. These log entries now provide a view of what is actually happening inside the task (and within each child task thread). I could probably add some more logging, but I tried not to go overboard.

Additionally, I fully unit tested this code. This was a combination of TDD and TAD – I added some extra logic after I wrote the tests that introduced some more scenarios. So I went back and added tests for more.

What About Sequential Tasks

In his original patch Patrick provided a parallel and a sequential task. Originally I didn’t want to add the sequential task as I didn’t see the value of it. However, I eventually saw why he added it – within the parallel task you might want a child task that has several tasks. But they should all run in sequence.

So I also added a sequential task. This uses a lot of the same code as the parallel task. Actually, there are two main differences between the two:

1. Parallel uses threads to run each task, sequential uses the project thread.
2. Sequential has the ability to cancel pending tasks if a task fails, parallel just runs everything.

Both tasks use a clone of the results, both have improved error handling and both use logging extensively.

An Example

As an example of how these work, here is a small block that shows how parallel and sequential tasks work together:

  1: <tasks>

  2:   <parallel>

  3:     <tasks>

  4:       <sequential continueOnFailure="false">

  5:         <tasks>

  6:           <exec>

  7:             <executable>RunTest.cmd</executable>

  8:             <baseDirectory>D:\Temp</baseDirectory>

  9:           </exec>

 10:           <exec>

 11:             <executable>RunTest.cmd</executable>

 12:             <baseDirectory>D:\Temp</baseDirectory>

 13:           </exec>

 14:         </tasks>

 15:       </sequential>

 16:       <exec>

 17:         <executable>RunTest.cmd</executable>

 18:         <baseDirectory>D:\Temp</baseDirectory>

 19:       </exec>

 20:     </tasks>

 21:   </parallel>

 22: </tasks>

Of course, there is nothing stopping people using these tasks within other tasks.

In Conclusion

So, sorry Patrick for taking so long to get to your patch. Hopefully this now provides an improved version of the original patch.

And hopefully this will be useful to other people as well (I can already think of some scenarios at my work where it will be useful!)