Extending Queues

New Queue Functionality

There have been a couple of requests in the mailing lists recently about queues and how they work. I recently wrote a post on how they work (read it here). Now it’s time to look at extending the queue functionality.

One of the issues about queues is the way new projects are added to it. We have the following scenario:

• Project A depends on Project B – so a ForceBuild trigger has been added to Project A.
• Project A also has an interval trigger with an IfModificationExists.
• Project B uses an interval trigger to build

In this scenario it is possible for Project A to be added to the queue with a IfModificationsExists trigger while Project B is building. The outcome people would like is the IfModificationsExists build to be replaced with a ForceBuild instance. This is the specific functionality I’m going to look into.

Note: Before I begin I should tell you that James Chaldecott has already posted a solution to this in CCNET-1095. I’m not going to replace his work, instead I’m going to expand on his work to give people the choice of what will happen.

Planned Work

Since James has already done most of the hard work, here’s what I’m planning on doing:

• Add a new configuration setting called queue to the config file. This will allow the queues to be configured.
• Modify the queue start-up process to associate the configuration information.
• Modify James’s patch to only be applied when the configuration asks for it.
• Add an additional option to queues to replace the existing build request. This is instead of removing and re-adding the build request.

So, let’s start our journey of expansion.

First Step: Configuration

Recently I posted on how the configuration works in CruiseControl.Net and how to extend it (here for the basics and here for extending it). I’m going to build on this knowledge to add a queue item to the configuration file, which will be a top-level item since it will involve multiple projects. Here is how the configuration will look when I’m done:

<cruisecontrol>
   <!-- Project definitions -->
   <queue name="CruiseControl.Net" duplicates="ApplyForceBuildsReAdd"/>
</cruisecontrol>

You’ll see that I’m adding a <queue> tag at the same level as <project> tags. To do this I first defined an interface and then an implementation of the interface. Since this is purely for configuring a queue I’ve added it to the configuration folder. I also added an enumeration for the duplicates value, so the value will be validated.

Queue Configuration Types

The ReflectorType name for DefaultQueueConfiguration is “queue”. The Name and HandlingMode properties have been decorated with ReflectorProperty attributes. That’s the first part.

Next I opened up IConfiguration and added a new property called QueueConfigurations to hold the queues. This required me to also add the property to Configuration. And one last change – I modified NetReflectorConfigurationReader to see if the item is a project or a queue and then add it to the correct location (this post explains why.)

Next Step: Associating with the Queue

The method were queue names are converted to actual queues is the Initialize() method in IntegrationQueueManager. This method takes in an IConfiguration instance and generates the queues. It does this by calling the Add() method on IntegrationQueueSet. This in turn generates a new IntegrationQueue instance and sets the name. Since IntegrationQueue is an implementation of IIntegrationQueue, that’s where I started my changes.

The first thing I did was add a property called Configuration to IIntegrationQueue. Then I modified IntegrationQueue to implement the property. This is read-only, so I also had to modify the constructor to load the property. Next, add a parameter to the Add() method in IntegrationQueueSet to take in the configuration and pass it onto the new IntegrationQueue instance. Finally I modified the Initialize() method in IntegrationQueueManager to find the configuration and pass it into the Add() method.

To make things a bit easier I added a FindQueueConfiguration() method to IConfiguration. This takes in a queue name and returns an IQueueConfiguration. It simply iterates through the lists of queues checking the names. If a match is found, then this is returned. Otherwise it creates a new default instance of DefaultQueueConfiguration and returns it. This way there will always be configuration associated with a queue.

Applying the Patch

Now that I’ve done the ground work, I’m ready to apply James’s patch. His patch modifies the Enqueue() method in IntegrationQueue. So, first I added his patch and made sure it works (it does ) Once his changes were in place I added a switch statment to check the HandlingMode property of the configuration. If the mode is UseFirst I stick with the original logic. If the mode is ApplyForceBuildsReAdd I use James’s logic. Otherwise I throw an exception (since I’m not quite ready to implement ApplyForceBuildsReplace).

Give it a quick test, and it’s still working

Final Step: Replace Mode

Now that I’ve enabled configuration of the way duplicate builds are handled, I wanted to add an additional mode. This is the same as James’s patch, but instead of removing the old item and adding the new item, I will replace the old item (that way the queue order doesn’t change).

The current code (for re-adding the request) removes the requests, calculates the new queue position and adds the new request. What I want to dos is exactly the same, expect without the calculation of the queue position. To handle this I added an override to the AddToQueue() where I could specify the queue position. Then I all had to do was copy the existing code, change the messages slightly and call my new override with the old position.

Turns out this change was quite easy afterall

In Summary

I’ve now built on my knowledge of how both the configuraton and queues work in order to add some requested functionality. The main changes were:

• Adding configuration settings for queues (three new types)
• Based on these settings, allowing the user to specify how duplicate requests should be handled (seven types modified)
• Updated the unit tests (not included in this post, but still needed to be done)

Now, if people are happy with the existing functionality, they don’t have to change (since the existing functionality is the default). However, if people want force builds to be considered more important, they can configure their queues to work this way. Hopefully a win/win situation for everyone involved! I’ll get this posted on Jira and then hopefully it’ll be added to the trunk sometime soon.

Coming up next, security for CruiseControl.Net…