The Missing Piece: Encrypted Communications

Introduction

In my last post (read it here) I wrote about an issue with security – that the actual communications between the client and the server were not secure. I also outlined the approach I was taking to remedy the issue.

Before I move onto my solution, I must mention that this is not a perfect solution. As one of my readers commented, this does not really cover a man-in-the-middle attack. It will prevent network sniffing and replay attacks, it is still possible to insert a listener between the client and the server.

It is possible to configure .NET Remoting to intrinsically secure communications – in which case there is no need to worry about this from the client side at all. However, it is not easy easy as it might appear, so I decided to finish my partial solution.

In time, I hope to be able to look at expanding the communications channels. Currently the only channel for communicating with the server is .NET Remoting – which was because of the way methods used to be called. Now with the new messaging-based framework, it is easier to add new channels. Hopefully I will get some time to look at using SslStream and NegotiateStream to secure communications.

Client Communications

First, how does the new code send encrypted messages to the server? This involved expanding how messages are currently sent.

Currently messaging uses the following model:

Using the new communications model, each client application will generate an instance of a CruiseServerClient. Actually they generate a CruiseServerClientBase instance, but the class that implements this abstract class for the new framework is CruiseServerClient. This class does not actually send any messages – instead it translates the method calls (e.g. ForceBuild) into messages (e.g. ProjectRequest), sends the message and then processes the response.

The actual sending is done by an instance of IServerConnection. This instance takes the specified messages and pushes them over the specified channel (either .NET Remoting or HTTP). Because the actual channel is encapsulated by an IServerConnection, it is easy to replace the channel with a different one.

For encrypted communications I added a new channel called EncryptingConnection. This channel is then inserted between the client and the underlying communications channel:

This means that the messages are encrypted (both ways), but the underlying transports (e.g. HTTP or .NET Remoting) can still be used without any changes.

There are a couple of extra changes. First, the original request message gets replaced by an EncryptedRequest message. This contains the original message, but encrypted (of course), and the original action that was requested. Secondly, the action has been changed to ProcessSecureRequest – more on this when I cover the server side changes.

EncryptingConnection is also responsible for the hand-shaking required to set up the encrypted communications. When a message is sent, the connection checks to see if it has a password. If it does not have a password, sets up a new common password by the following process:

1. Ask the server for its public key
2. Generate a new secret (using RijndaelManaged)
3. Encrypt the secret using the server’s public key (using RSACryptoServiceProvider)
4. Send the secret to the server and wait for the response
5. Store the secret locally

Once this has been done, all messages are then encrypted using the common secret (again with RijndaelManaged). This includes both request and response messages.

The actual sending of the messages is then done by passing the message onto the underlying channel.

Server Communications

Now that the client can send messages, we need to look at the server side. After all, if the server can’t understand the messages, then nothing will be done!

All communications into the server come in via CruiseServerClient. This is a MarshalByRefObject-derived class that uses .NET Remoting. All HTTP communications actually go via the dashboard, which passes them onto the server using .NET Remoting. So this model simplifies things – I only need to change CruiseServerClient!

Currently CruiseServerClient has one overloaded method for processing messages – ProcessRequest(). This takes the incoming message, works out which method to call, converts the message into the required objects (if required) and then calls the method. It also handles generating the response and adding any error information.

To handle secure messages I have added a ProcessSecureRequest (). This method is called by ProcessRequest() and acts as a middle-man. Basically, when ProcessRequest() gets a secure message, it passes it onto ProcessSecureRequest(), which then works out which methods to call, converts the message, etc.

I have also added three other new methods:

• RetrievePublicKey() – this will send the server’s public key to the client. If the server does not have a public/private key pair it will generate the pair (using RSACryptoServiceProvider).
• InitialiseSecureConnection() – this receives the common secret from the client, decrypts it and stores it locally.
• TerminateSecureConnection() – this will remove the common secret for a client from the local store.

These methods are called as part of the hand-shaking between the client and the server.

Additionally, as part of these changes, I have added a new property to the messages called ChannelInformation. This property has information on how the message was transported – i.e. metadata about the message itself.

Currently, there is only one implementation – RemotingChannelSecurityInformation. This says the messages was transported using .NET Remoting. On this class is a property called IsEncrypted. ProcessRequest() generates an instance of this class and sets IsEncrypted to false, while ProcessSecureRequest() generates an instance with IsEncrypted set to true.

Turning On Encryption

Encryption is turned on from the client-side. When a client wants to communicate with a server using encrypted communications, they need to set UseEncryption to true in ClientStartUpSettings.

Currently, this is not possible with the dashboard, but I have added it to CCTray. When a new server connection is added (for either .NET Remoting or via the dashboard), there is a new option of turning on encryption:

To use encrypted communications, all that is needed is to check this box.

Unfortunately, due to the way CCTray is configured, it is not possible to turn on encryption for an existing server. Instead the server needs to be removed and re-added (yuck!) This is probably a good area for tidying up in a future release of CCTray.

Enforcing Encryption

All this work so far has been to set up encrypted communications. At this point, it is up to the client to decide whether the channel is encrypted or not – the server doesn’t really care. We need some way of telling the server to only accept encrypted communications (i.e. to enforce encryption).

To do this, I have added a new configuration option to the server security settings – channel. This is an optional setting and has one required property – the type of channel. The following shows how to configure a channel to use encrypted messages:

  1: <internalSecurity>

  2:   <channel type="encryptedChannel"/>

  3:   <users>

  4:     <!-- Omitted for brevity-->

  5:   </users>

  6:   <permissions>

  7:     <!-- Omitted for brevity-->

  8:   </permissions>

  9: </internalSecurity>

This tells the security manager it will only accept communications over an encrypted channel.

When a message is received by the message processer (i.e. after it has been received by CruiseServerClient, processed and passed on), it calls a method called ValidateRequest(). This does some simple validation checks (the message is less than a day old and it hasn’t been duplicated). I have expanded this validation check to detect a channel definition and if present to validate it.

The Basics Of Encryption

This provides some of the basic pieces for encrypted communications. It certainly is not a comprehensive solution – there are still some areas uncovered (like validating the server is who it says it is).

In terms of where this can go, there are the following items that can be included:

• Additional encryption mechanisms (e.g. DES, triple DES, AES, etc.)
• Additional security channels (e.g. SSL, negotiated)
• Validation of the server (perhaps via X.509 certificates)
• Use encryption from the dashboard

But this provides a start. Let me know what you think so far.

Migration Wizard

I have completed the first cut of the migration wizard. This wizard is now included in the installer – screenshots on how this looks are available here.

The wizard will migrate the following items when converting from 1.4.3 or 1.4.4:

• State and log files on the server are moved to ProgramData for both the console and the service
• E-mail groups configuration changed to use multiple notification types instead of a single attribute
• Dashboard configuration and packages are moved to ProgramData

The wizard will migrate the following items when converting from 1.4.2 or older (up to 1.4.0):

• Convert statistics file so it doesn’t cause double tabs
• Plus all the above migrations from 1.4.3/1.4.4

At the moment, I have only tested this on Windows machines – this will probably fail on *nix-based machines due to the slightly different folder structure.

Let me know if there are any issues.

Migration Woes

Oops, We’ve Broken Some Eggs

As we slowly move forward and add new features or fix broken features, it is inevitable that we make some breaking changes. Sometimes it is possible to work within the current structures and file formats, other times it is either inefficient, very painful or downright impossible. So, where possible we don’t make breaking changes, but sometime we have no choice.

While this is a common feature of software development, this can still cause pain to our users. Generally, people don’t like to think, even less so when it is painful thinking! And often, people just don’t read the upgrade notes – they just expect it to work.

Moving to 1.5.0 of CruiseControl.NET, we are making some major changes to the underlying storage locations. This is because all older versions assume the Program Files is readable and writable! Unfortunately, with Windows Vista, Windows Server 2008 and newer versions, this is no longer the case. We’ve been locked out! So, if we want to write data to disk, we need to move to a new location.

While this is a minor change for new users, it does have a major impact on existing users. They will need to move all their files that are stored in Program Files to the new local (e.g. ProgramData, All Users, etc.) Knowing how people don’t read the upgrade notes, we’re going to get a lot of queries about this.

A New Wizard

While I know we’re not going to be able to totally avoid the pain, I’d like to reduce it as much as possible. Originally I had added a check to the dashboard to enforce the migration, but this was disliked. So, instead I’ve added a new WinForms application to handle the migration.

At this point I should mention, this wizard is still in the very early days of development! DO NOT USE it for production systems, you have been warned!!!

This wizard will provide a step-by-step process for collecting user details on what they want to migrate. Once this has finished, it will migrate everything for them. If it runs into any errors in the migration process, it will (theoretically) roll-back any migration changes.

So, that’s the theory, here’s what I have put together so far.

Following the Yellow Brick Road

When the user starts the wizard, they will get a general introduction window. I’ve tried to put relevant information on it, and hopefully not too much information.

Clicking on next will prompt you to enter which version you are migrating from. Currently, I only handle migrating from 1.4.4 to 1.5.0, but I plan to add additional migrations from older versions (up to 1.4.0). This is because we added a number of breaking changes through the various versions, so it would be nice to allow the wizard to handle all of them. Hopefully, this will also encourage users of older versions to upgrade.

Once the user has selected the version, we enter into the meat of the settings. First off, the user has the choice of just which items they want to migrate. Here, they are being prompted if they want to migrate the server settings. The wizard also asks where the user wants to migrate from, since they may have installed in a different location (I’m not aware of an easy way to find the location, and I don’t want to try and scan for it!)

Migrating the server settings will move any state files, the overall projects state file and any project folders. If the working or artefact folders are not set in the config, they are added under the server folder, which is why they need to be moved.

Each page has validation checks built in. In this case, since we are asking for a folder location, it will validate the folder is valid and that it contains a ccnet.config file (this is needed later on).

Next, the user will be prompted if they want to migrate the configuration settings. This will change ccnet.config so it works in 1.5.0. Sometimes, but not very often, we are forced to change the layout of ccnet.config, this will handle the changes automatically.

On this page, the user is also prompted if they want to backup their original configuration. This is because the wizard will actually modify the file (as opposed to just move it), so they might like the safety of seeing their original file is preserved.

Now for the bad news, I’m not actually doing anything with these settings yet. But I know we do have some breaking changes, I just need to track them down and then add them to the wizard.

Next, the user is asked if they also want to migrate the web dashboard. A lot of times, the web dashboard is installed on the same machine as the server, but sometimes they are on different machines. Here we give the user the choice of whether to migrate both, one or the other.

Again, the user is prompted for the current location – they might have installed it to a non-default folder. The wizard will default to the standard location, but as you can see in my example, it is easy to change (I don’t have CruiseControl.NET installed on my machine, I have several development copies of the source instead.)

The user is then asked to confirm the settings. Currently it is just a text blob with all the details in it, I might see if I can figure out a better way to display it.

When they click on next, it then begins the migration. This is done on a background thread, so the user can still interact with the wizard. In time, I’d like to add the ability to cancel a migration part-way through (this would perform a rollback), but I haven’t gotten around to it yet.

Also, if you noticed the (Not Responding) message on this title bar, it’s just because the process is so fast on my machine, the only way I could take this snapshot was putting a breakpoint in the code! Normally it does respond (and very nicely).

Finally, the user will told what has happened. I’m trying to keep this very minimalistic – just the statuses for each item, plus any warnings or errors.

If the user wants to see the full log, this is available by clicking on the View Log button:

This is pretty much all the messages that are displayed in the progress window.

Welcome to the Wizard

So, that’s a quick tour of the wizard from start to end, at least in terms of its UI. The next question is what does it actually do?

Currently, it will move the following folders and files:

• Project working and artefact folders (if under Program Files)
• Project state files
• Overall project states (started/stopped statuses)
• Web dashboard configuration
• Web dashboard packages

I am also going to look at moving the log files, but this also require modifying the app.config files,so I want to be able to do this in a reversible manner.

I will also update ccnet.config to handle any breaking changes, but I need to find out what these are first.

And finally, I need to go through the old change logs to see if there are any other breaking changes, and then add them in.

Additional Features

I’ve also tried to build this with a few additional features in mind, but I can’t guarantee that I will add them. These are:

• Command-line execution (i.e. no UI)
• Cancelling migration part-way through migration
• Saving the log file
• Migrating to a different location

Plus whatever other people want to suggest…

So, this is what I am currently working on. I’ll add it to Subversion, so anyone who is brave can try it out, but I won’t add it to the installer until it is nearly complete.

Let me know what you think about the wizard.

Inner Workings: The Anatomy of the Dashboard

An Introduction

On the mailing list there was a question today about the parts of the dashboard and how they match up with the configuration. So I thought it’s time for me to write down what I know about the parts of the dashboard and how they work.

First, there are four basic levels to the dashboard. These levels are:

• Farm – the areas of the dashboard that do not relate to any build server
• Server – the overall summary of a build server (e.g. an instance of the CC.NET service or console)
• Project – a project on a build server
• Build – an instance of an integration for a project on a build server

Each level has its own set of plug-ins, as would be expected since they show very different information. To show these levels, here is a screen shot of each.

Note: These are running on my development instance of CC.NET, hence they don’t have a proper version number or very many plug-ins installed.

Farm Level View

Server Level View

Project Level View

Build Level View

It is important to know about these levels, because putting a plug-in in the wrong location will put items in an unexpected place!

The Parts of a Page

Every page has four basic parts – and a plug-in only has control over one part. These four parts are:

The header and the footer are both generated by the main page template. These contain common items to every page, such as the name and version of CC.NET, common links, breadcrumbs and when the date/time the page was rendered. These do change slightly depending on the level, but this is outside the control of a plug-in.

The side bar contains a set of links. Again, these vary slightly depending on the level, but they are controlled by the system again. However, these links are defined in the configuration – each plug-in exposes one or more links to add to the side bar. More on this in a little while.

Finally, there is the content area. This is entirely up to the plug-in to populate. The system completely ignores the content area and assumes the plug-in will generate meaningful content. This content is accessed by clicking on one of the links in the side bar.

Overlying these parts onto a farm level view looks like the following:

Where the Config Comes In

Hopefully by now, you will have some ideas of where things fit. But let’s make things perfectly clear and delve into how the configuration relates to the pages.

First, here is the configuration I used to generate the above screen shots:

  1: <dashboard>

  2:   <remoteServices>

  3:     <servers>

  4:       <server name="local" url="tcp://localhost:21234/CruiseManager.rem" allowForceBuild="true" allowStartStopBuild="true" backwardsCompatible="false" />

  5:     </servers>

  6:   </remoteServices>

  7:   <plugins>

  8:     <farmPlugins>

  9:       <farmReportFarmPlugin />

 10:       <cctrayDownloadPlugin />

 11:       <administrationPlugin password="********" />

 12:     </farmPlugins>

 13:     <serverPlugins>

 14:       <serverReportServerPlugin />

 15:     </serverPlugins>

 16:     <projectPlugins>

 17:       <projectReportProjectPlugin />

 18:       <viewProjectStatusPlugin />

 19:       <latestBuildReportProjectPlugin />

 20:       <viewAllBuildsProjectPlugin />

 21:     </projectPlugins>

 22:     <buildPlugins>

 23:       <buildReportBuildPlugin>

 24:         <xslFileNames>

 25:           <xslFile>xsl\header.xsl</xslFile>

 26:           <xslFile>xsl\modifications.xsl</xslFile>

 27:           <xslFile>xsl\NCoverSummary.xsl</xslFile>

 28:         </xslFileNames>

 29:       </buildReportBuildPlugin>

 30:       <buildLogBuildPlugin />

 31:       <xslReportBuildPlugin description="NCover Report" actionName="NCoverBuildReport" xslFileName="xsl\NCover.xsl"></xslReportBuildPlugin>

 32:     </buildPlugins>

 33:     <securityPlugins>

 34:       <simpleSecurity />

 35:     </securityPlugins>

 36:   </plugins>

 37: </dashboard>

Like I said, it is very simple. The part that we are interested in the the <plugins> section. This defines all the plug-ins that can be seen. For the moment, I’m going to ignore the <securityPlugins> as this is specific to 1.5.0 and I’ve already covered it in a previous post.

There are four sections of plug-ins: <farmPlugins>, <serverPlugins>, <projectPlugins> and <buildPlugins>. These map to each level in the dashboard. Within each section there are one or more plug-ins. These plug-ins define the links that appear in the side bar. For example, in the <farmPlugins> section:

• <farmReportFarmPlugin> maps to the “Farm Report” link
• <cctrayDownloadPlugin> maps to the “Download CCTray” link
• <administrationPlugin> maps to the “Administer Dashboard” link

Most plug-ins have these link titles as hard-coded values within them – so they can’t be changed. Clicking on a link will pass control to the plug-in, which generates the content to be displayed (this is actually a simplification, but it will do for this post.)

Normally, there is only one instance of a plug-in per section – having multiple plug-ins generates some unexpected results, so it is recommended against. However, some rules are made to be broken.

Breaking the Rules – <buildPlugins>

The above details cover most plug-ins in the dashboard. However, the one area where the rules get broken is in the <buildPlugins> section. This section defines two special plug-ins – <buildReportBuildPlugin> and <xslReportBuildPlugin> (1.5.0 will be adding a third – <htmlReportPlugin>).

First, the <xslReportBuildPlugin> element. This breaks the rules by allowing multiple instances. It does this because of two properties: description and actionName. The description is the text of the link to appear in the side bar, while the actionName is the command name to be passed to the server. The actionName MUST be unique, otherwise the poor dashboard will get confused!

This plug-in takes in a XSL-T template and transforms the build log for a project into an HTML report (which is the third parameter in the element). This means we don’t need to develop lots and lots of plug-ins (e.g. one for each report), instead we can just write a style sheet and get it to transform the results.

The second rule breaker is <buildReportBuildPlugin>. This is a required plug-in and there can only be one instance of it. The reason it is different is it has an <xslFileNames> section in it. This section is similar to the xslFileName attribute in the <xslReportBuildPlugin>, but it has one major difference. The <xslReportBuildPlugin> generates a link in the side bar, the <xslFileName> element doesn’t. Instead, it’s transform gets merged into one big page – the “Build Report”.

The following picture shows how these relate:

Clicking on an <xslReportBuildPlugin> will generate a completely different content area, one that is not affected by <buildReportBuildPlugin> at all.

Summary

There are four general levels – farm, server, project and build. Each has its own config section and allows a different set of plug-ins.

Within each page, there are four areas – header, footer, sidebar and content. The plug-in generates the content area and defines links to go into the sidebar – everything else is handled by the system.

For the build level, there can be multiple <xslReportBuildPlugin> – each defines a link in the side bar, with custom content. The <xslReportBuildPlugin> section defines the items to appear within the “Build Report” – these do not appear as links within the side.

Hopefully this provides a better understanding of the parts of the dashboard.

Dynamic Build Parameters – Some Examples

Some New Exciting Functionality

In the 1.5.0 release we are adding the ability to define dynamic build parameters. A dynamic build parameter is a value that can change for different builds – and it can be set by the user when they perform a force build.

This addresses a simple issue in the current CruiseControl.NET – if even one parameter in a project needs to differ between builds, multiple projects are needed! Now, an administrator can define just one project and allow multiple different variations on it.

In this post I’ll go through some of the possibilities for this feature. I won’t cover everything, and the feature itself is still in its infancy, but hopefully this will provide a start for people to trial it.

A Very Basic Configuration

To keep things simple, I’m only going to work with a very simple configuration (ccnet.config). This is the base config:

<cruisecontrol>

  <project name="Test Project">

    <description>A demonstration project to show the features of the dynamic build parameters.</description>

    <sourcecontrol type="svn">

      <trunkUrl>svn://svn.mycompany.com/firstproject/trunk</trunkUrl>

      <workingDirectory>C:\SourceControl\FirstProject\</workingDirectory>

    </sourcecontrol>

    <triggers>

      <intervalTrigger />

    </triggers>

    <tasks>

      <nant>

        <buildFile>App.build</buildFile>

        <targetList>

          <target>Dev</target>

        </targetList>

        <buildArgs>-D:reason=testing</buildArgs>

      </nant>

      <gendarme>

        <assemblies>

          <assemblyMatch expr="*.dll"/>

        </assemblies>

        <limit>100</limit>

      </gendarme>

    </tasks>

    <publishers>

      <rss />

      <xmllogger />

    </publishers>

  </project>

</cruisecontrol>

This project uses Subversion as the source code repository, using a repository called firstproject. It has the default interval trigger (checks for modifications every 60 seconds). When a build is triggered it will run an NAnt build script called App.Build and then perform a Gendarme analysis. Finally it will generate an RSS feed and log the XML results (so the dashboard can use it.)

Basically I’m going to add three dynamic parameters:

• Define the target to build
• Change the reason for the build (a buildArg for the NAnt script)
• And vary the limit on the Gendarme analysis

And to finish things off, I’ll show how the same project can be used to generate different builds via a schedule.

So, let’s get the show on the road.

Defining a Target

The first parameter I want to vary is the build type. This is a target in the NAnt script, and I’m going to allow three different targets: Dev, Test and Prod (this could also be done as a buildArg, but I want to leave that for later!)

The first thing to do is to add the parameter to the project. Parameters are defined at the project level for two reasons:

• It saves duplicates if the parameter needs to be used in multiple places
• It makes it simpler for the server to work out which parameters need to be sent to the client (thus it is faster)

Since the build target can only be one of three values, I’m going to define a range parameter:

<parameters>

  <rangeParameter name="Type">

    <description>The type of build to perform.</description>

    <allowedValues>

      <value>Dev</value>

      <value>Test</value>

      <value>Prod</value>

    </allowedValues>

    <default>Dev</default>

  </rangeParameter>

</parameters>

All parameters must be defined within a parameters block within the project. As a minimum definition, each parameter must have a name – which is what is sent to the client.

This definition defines a parameter called “Type”, it is a range parameter with three possible values: “Dev”, “Test” and “Prod”. Finally, the default value is “Dev”. When a force build is triggered, the user will be displayed a combo box (or a drop down list depending on the client) with these three values in it, and “Dev” will be the initial selection.

The next step is to use the parameter in a task (currently they are limited to tasks and publishers, this may change if there is enough support for it.) The following shows how I would modify the <nant> task to use the parameter:

<nant>

  <!-- Omitted to save space -->

  <dynamicValues>

    <directValue parameter="Type" property="targetList.target[0]">

      <default>Dev</default>

    </directValue>

  </dynamicValues>

</nant>

This simply says replace the first target in the targetList with the parameter value. This is a direct replacement – the entire value will be replaced with whatever value was selected for Type.

Note that there is also a default value here. This is used for when a value has not been selected (i.e. when the interval trigger detects changes).

This is a very simple example of a pre-defined value that entirely replaces the existing value. Now, let’s see some of the other parameter types.

A Numeric Parameter – Gendarme Limit

The next thing I’ll show is how to use a numeric parameter value. Again, this needs to be defined at the project level in the parameters block:

<parameters>

  <!-- Omitted to save space -->

  <numericParameter name="GendarmeLimit">

    <description>The limit of Gendarme errors.</description>

    <minimum>50</minimum>

    <maximum>500</maximum>

    <default>100</default>

  </numericParameter>

</parameters>

This says we want to also have a GendarmeLimit parameter, this is a numeric value and must be between 50 and 500. The default value will be 100.

To use this, it is the same as the target type, just add a direct replacement dynamic value:

<gendarme>

  <!-- Omitted to save space -->

  <dynamicValues>

    <directValue parameter="GendarmeLimit" property="limit">

      <default>100</default>

    </directValue>

  </dynamicValues>

</gendarme>

Note, the value will automatically be converted into the correct data type for use in the task. If there is a data type mismatch (e.g. a test value is put into a numeric property), then the build will fail with an error.

Now, let’s add one more parameter – one that doesn’t use direct replacement.

A Build Reason

This is a bit of a contrived example, but here we are going to pass a reason for the build into the NAnt task.

First, let’s define the parameter:

<parameters>

  <!-- Omitted to save space -->

  <textParameter name="Reason">

    <description>Reason for the build being forced.</description>

    <minimum>10</minimum>

    <maximum>255</maximum>

    <required>true</required>

  </textParameter>

</parameters>

This is a text parameter with a minimum length of 10 and a maximum length of 255.

This time, to use the parameter we are going to use a replacementValue:

<nant>

  <!-- Omitted to save space -->

  <dynamicValues>

    <!-- Omitted to save space -->

    <replacementValue property="buildArgs">

      <format>-D:"{0}"</format>

      <parameters>

        <namedValue name="Reason" value="Triggered" />

      </parameters>

    </replacementValue>

  </dynamicValues>

</nant>

This has the same property attribute, but otherwise is very different. A standard .NET format string is required (internally this is literally passed onto string.Format()!) And there is also a list of parameters to use. As you might have guessed, this dynamic value can use multiple parameters. If a property needs to use more than one parameter, then this is the syntax that is required.

Note for the parameters a namedValue is used. The name is the name of the parameter, while the value is the default value. This is needed because there can be multiple parameters, each with its own default value!

Scheduling A Prod Build

The final item I’ll cover is how to use parameters together with triggers. In this case, I’m going to schedule a build every night that uses the Prod target.

This is fairly easy to do, all we need to do is add a new parameterTrigger that contains the schedule trigger. The following shows how to do this:

<triggers>

  <intervalTrigger />

  <parameterTrigger>

    <trigger type="scheduleTrigger">

      <time>1:00</time>

      <buildCondition>ForceBuild</buildCondition>

    </trigger>

    <parameters>

      <namedValue name="Type" value="Prod"/>

      <namedValue name="Reason" value="Scheduled"/>

      <namedValue name="GendarmeLimit" value="500"/>

    </parameters>

  </parameterTrigger>

</triggers>

This will schedule a build every night at 1am and pass in “Prod” for the Type, “Scheduled” for the Reason and 500 for the GendarmeLimit.

The Final Configuration

Hopefully this has provided a quick introduction to how parameters and dynamic values can be used. If you can think of how this can be improved, let me know and I’ll see what I can do.

And to finish up, here is the complete configuration:

<cruisecontrol>

  <project name="Test Project">

    <description>A demonstration project to show the features of the dynamic build parameters.</description>

    <sourcecontrol type="svn">

      <trunkUrl>svn://svn.mycompany.com/firstproject/trunk</trunkUrl>

      <workingDirectory>C:\SourceControl\FirstProject\</workingDirectory>

    </sourcecontrol>

    <triggers>

      <intervalTrigger />

      <parameterTrigger>

        <trigger type="scheduleTrigger">

          <time>1:00</time>

          <buildCondition>ForceBuild</buildCondition>

        </trigger>

        <parameters>

          <namedValue name="Type" value="Prod"/>

          <namedValue name="Reason" value="Scheduled"/>

          <namedValue name="GendarmeLimit" value="500"/>

        </parameters>

      </parameterTrigger>

    </triggers>

    <tasks>

      <nant>

        <buildFile>App.build</buildFile>

        <targetList>

          <target>Dev</target>

        </targetList>

        <buildArgs>-D:reason=testing</buildArgs>

        <dynamicValues>

          <directValue parameter="Type" property="targetList.target[0]">

            <default>Dev</default>

          </directValue>

          <replacementValue property="buildArgs">

            <format>-D:"{0}"</format>

            <parameters>

              <namedValue name="Reason" value="Triggered" />

            </parameters>

          </replacementValue>

        </dynamicValues>

      </nant>

      <gendarme>

        <assemblies>

          <assemblyMatch expr="*.dll"/>

        </assemblies>

        <limit>100</limit>

        <dynamicValues>

          <directValue parameter="GendarmeLimit" property="limit">

            <default>100</default>

          </directValue>

        </dynamicValues>

      </gendarme>

    </tasks>

    <publishers>

      <rss />

      <xmllogger />

    </publishers>

    <parameters>

      <rangeParameter name="Type">

        <description>The type of build to perform.</description>

        <allowedValues>

          <value>Dev</value>

          <value>Test</value>

          <value>Prod</value>

        </allowedValues>

        <default>Dev</default>

      </rangeParameter>

      <numericParameter name="GendarmeLimit">

        <description>The limit of Gendarme errors.</description>

        <minimum>50</minimum>

        <maximum>500</maximum>

        <default>100</default>

      </numericParameter>

      <textParameter name="Reason">

        <description>Reason for the build being forced.</description>

        <minimum>10</minimum>

        <maximum>255</maximum>

        <required>true</required>

      </textParameter>

    </parameters>

  </project>

</cruisecontrol>

Extended Configuration – Beyond NetReflector

Some Background

In CruiseControl.NET we have a file merge task. This task will load one or more files and merge them into a single XML document. From this document the reports get generated.

Recently we made some changes to CruiseControl.NET to allow reports and report artefacts to come from a different location, which now allows images and HTML reports to be included. This was actually a flow-on effect from the new NDepend and NCover tasks, as these tasks needed to display non-XML data.

These worked so well, one of the other developers asked how to include items from other tasks in these reports, and even potentially user-specified files. The bad news at the time is this was not possible. The only merging task we had was the merge files task (<merge> element) and this task merges into the XML document.

This left us with two options:

1. Add a new task to merge non-XML files
2. Modify the merge task to do non-merge copying

Since I didn’t particularly like the idea of a new task just for this specialised functionality, I decided to look at extending the current merge task.

The Problem

An example configuration for the current merge task would look something like this:

<merge>

    <files>

        <file>File 1</file>

        <file>File 2</file>

        <file>File 3</file>

    </files>

</merge>

In contrast, this is what I wanted to achieve:

<merge target="somewhere">

    <files>

        <file>File 1</file>

        <file action="Copy">File 2</file>

        <file action="Merge">File 3</file>

    </files>

</merge>

The problem with this lies in the action attributes – NetReflector does not handle them. The closest we can get is something like:

<merge target="somewhere">

    <files>

        <file>File 1</file>

        <file><action>Copy</action><name>File 2</name></file>

        <file><action>Merge</action><name>File 3</name></file>

    </files>

</merge>

Which would break existing config files! So, based on plain-vanilla NetReflector we would be stuck.

But, there is a way around this, and one that doesn’t involve modifying NetReflector at all.

Expanding the Attribute

All of the configuration is based on NetReflector attributes and most of it uses ReflectorPropertyAttribute. Like all the NetReflector attributes it has a constructor that requires a name, which is the name of the element. However, this attribute is unique – it also has a second constructor allows the developer to pass in a factory Type.

The factory argument allows the developer to override the default serialisation/de-serialisation behaviour with their own custom behaviour.

The factory type must implement ISerialiserFactory. This is a simple interface with one method that creates a new serialiser. This serialiser can then serialise/de-serialise in any way it wants!

The serialiser must implement IXmlMemberSerialiser, which also includes IXmlSerialiser. IXmlSerialiser has a Read() and a Write() method, while IXmlMemberSerialiser provides two properties and an extra method (I haven’t figured out what they are needed for yet!)

While it is possible to directly implement these interfaces, a simpler way is to inherit from XmlMemberSerialiser. This class implements all of the methods and properties – we just choose which ones we want to override (normally Read() and Write()).

Both the Read() and the Write() methods work directly with XML. The Read() method receives in the node to be de-serialised, while the Write() method receives an XmlWriter to work with. Generally, if you going to override one, you should override both – especially as both are used in CruiseControl.NET.

The only down-side to this approach is you have to completely implement the serialisation/de-serialisation behaviour. The default serialiser automatically handles whether it is an array, a list or a single value, our custom serialiser needs to do this for itself.

Reading a Node

As an example of how to implement a custom serialiser, I’ll go over what I have done for the new serialiser for the merge file task. This serialiser handles the desired XML I showed above.

The code for the Read() method is:

public override object Read(XmlNode node, NetReflectorTypeTable table)

{

    var fileList = new List<MergeFileInfo>();


    if (node != null)

    {

        // Validate the attributes

        if (node.Attributes.Count > 0)

        {

            throw new NetReflectorException("A file list cannot directly contain attributes.\r\nXML: " + node.OuterXml);

        }


        // Check each element

        foreach (XmlElement fileElement in node.SelectNodes("*"))

        {

            if (fileElement.Name == "file")

            {

                // Make sure there are no child elements

                if (fileElement.SelectNodes("*").Count > 0)

                {

                    throw new NetReflectorException("file cannot contain any sub-items.\r\nXML: " + node.OuterXml);

                }


                // Load the filename

                var newFile = new MergeFileInfo();

                newFile.FileName = fileElement.InnerText;


                // Load the merge action

                var typeAttribute = fileElement.GetAttribute("action");

                if (string.IsNullOrEmpty(typeAttribute))

                {

                    newFile.MergeAction = MergeFileInfo.MergeActionType.Merge;

                }

                else

                {

                    try

                    {

                        newFile.MergeAction = (MergeFileInfo.MergeActionType)Enum.Parse(

                            typeof(MergeFileInfo.MergeActionType),

                            typeAttribute);

                    }

                    catch (Exception error)

                    {

                        throw new NetReflectorConverterException("Unknown action :'" + typeAttribute + "'\r\nXML: " + node.OuterXml, error);

                    }

                }

                fileList.Add(newFile);

            }

            else

            {

                // Unknown sub-item

                throw new NetReflectorException(fileElement.Name + " is not a valid sub-item.\r\nXML: " + node.OuterXml);

            }

        }

    }

    return fileList.ToArray();

}

The arguments are an XmlNode and a NetReflectorTable. The node is the complete node to be de-serialised, in this case the <files> element and everything in it. The NetReflectorTable is a type custom to NetReflector – it basically contains a dictionary of all the available types that can be loaded. In this scenario we are not going to use it, but it is used in other serialisers (e.g. for hash-tables).

The majority of the work in this method is validation – since this will override the automatic behaviour we have to do everything ourselves! Since the end objective is to return an array of MergeFileInfo instances, I need to make sure the user isn’t trying anything that is not allowed.

The main working lines are instantiating a new MergeFileInfo, setting the file name from the text of the element and then loading the action from the attribute. If the attribute is not there, then it assumes a Merge action. Once this is done it is added to a List<MergeFileInfo>. This gets done for every child element, and then at the end it returns an array of MergeFileInfo items.

Writing an Element

The Read() was a fairly long method because it included lots of validation (plus it might need some more later one). In contrast the Write() method is very simple. Here is the code that I wrote for it:

public override void Write(XmlWriter writer, object target)

{

    var list = target as MergeFileInfo[];

    if (list != null)

    {

        writer.WriteStartElement(base.Attribute.Name);

        foreach (var file in list)

        {

            writer.WriteStartElement("file");

            writer.WriteAttributeString("action", file.MergeAction.ToString());

            writer.WriteString(file.FileName);

            writer.WriteEndElement();

        }

        writer.WriteEndElement();

    }

}

The writer is where the XML will be written to, the target is the item to be serialised.

The only check is that the target is actually an array of MergeFileInfo. Once this is verified, then is is a simple matter of iterating through the array and dumping all the values to XML.

Beyond the Ordinary – Serialisers

I should mention that once this was implemented, the rest of the modifications were very simple. Now the <merge> task has the ability to copy files, rather than just merge them. And not only that, but it is also backwards compatible with previous versions!

So, while this post started off talking about the <merge> task, the real focus is how we can extend NetReflector serialisation by using a custom serialiser and type factory. if you want to see how it is done, take a look at the MergeFileSerialiser, MergeFileSerialiserFactory and MergeFilesTask classes.

On a final note, I should mention this did not need any changes to NetReflector. This functionality has always been there, even though I have only just discovered it. As well as the new merge files, time-outs also use this functionality – take a look at the TimeoutSerializer and TimeoutSerializerFactory classes.

So hopefully this opens some more possibilities, happy coding…

What’s New in 1.4.4: Miscellaneous Improvements

We’ve Updated the Current Stuff Too!

As well as adding new functionality, we’ve also taken some time to improve some of the existing components. Some of these improvements are corrections (i.e. making things work like they should have originally), others are brand-new.

Areas that have been improved include: the validator, configuration, source control, the console/service and the RSS feeds.

The Validator

The validator was originally added in the 1.4.3 release, but unfortunately due to an oversight by the development team (actually, by me!) it was not included in the normal distribution. Instead the code needed to be downloaded and compiled (not good for non-developers!)

This has now been fixed and the validator will be installed as part of the normal server install. As well as the console and uninstall short-cuts, a new short-cut will be added for the validator (CCValidator). Additionally, the validator now has its own separate installer – so it can be installed without also installing the server components (or the dashboard). Hopefully this will make it easier to get the validator and use it.

As well as tidying up the install process, the validator now offers some internal validations. Previously the validator would only validate that the file could be loaded (i.e. there were no load errors), it wouldn’t perform any checks to ensure that the settings were actually valid. Now the validator will start to offer some internal validation, and we certainly plan on improving this in future releases.

Configuration

This is kind of an overflow from the validator, but we’ve also tried to make errors in the configuration easier to understand. This involved some work on NetReflector (since this is what converts the XML into the .NET objects), and mainly involved adding more meaningful messages.

In a nutshell, we’ve tried to replace any generic exceptions with specific NetReflector exceptions. These exceptions provide some more information on the error (e.g. unable to convert to specified type, base types do not allow attributes, etc.), plus they also include the erroneous XML element. This information will make it easier to track the problem (where do we start with a NullReferenceException!) and hopefully easier to pinpoint the exact cause.

However, this is a work in progress – we can’t guarantee we’ve got all the exceptions covered. So if you find any errors that don’t make sense, feel free to let us know.

Source Control

In the 1.4.3 release we finally started logging errors with source control. This certainly opened a can of worms as a lot of installations worked on the assumption that source control errors were ignored.

Based on the feedback we’ve added a couple of new options to define how source control errors should be handled.

maxSourceControlRetries defines how many source control errors are allowed before the next stage happens

stopProjectOnReachingMaxSourceControlRetries (yes, it’s a long name) controls what happens in the next stage. If this is set to true, then the project will be stopped. This allows an administrator to work out what is wrong and then fix things without overloading the source control server. Setting it to false means the project will continue running.

sourceControlErrorHandling defines the options for what happens when an error occurs. The valid options are:

• ReportEveryFailure – this is the current functionality of 1.4.3 – every source control exception will be reported.
• ReportOnRetryAmount – only report once when the maximum number of attempts has been made. Any subsequent errors will be ignored.
• ReportOnEveryRetryAmount – only report when the maximum number of attempts has been made. After this the counter is reset and the error reported again after the next number of attempts has failed.

The retry attempt counter is always reset back to zero after a successful source control operation – no matter which option is chosen.

Additionally cleanUp and revert options have been added to the Subversion source control block.

Console/Service

Both the console and the service now have the ability to hot-swap DLLs. This is similar to ASP.NET where a binary can be x-copied into the folder and the application will automatically restart.

This feature has been added to make it easier to upgrade running instances of CruiseControl.NET. Previously the server (either console or service) would need to be stopped, the binaries copied over and then the server restarted. Now it is just a simple matter of copying the new binaries in and the server will take care of the rest.

The only file that cannot be copied over is the application executable (e.g. ccnet.exe or ccservice.exe).

RSS Feeds

Previously the RSS publisher in CruiseControl.NET only reported on the last build – previous builds were overwritten. The publisher now can publish multiple builds – which allows the feed to have a history of what has happened.

By default, the last twenty builds will be in the RSS feed, but this can be configured by adding an items property to the task.

<rss items="15"/>

In Conclusion

So, as well as adding new functionality, we’ve been tweaking the existing functionality to make things easier. However these new tweaks and features will remove some of the headaches people have been having.

As always, if you find any issues or problems, please let me know and I’ll look into them.

Now, this pretty much wraps up a lot of the changes for 1.4.4 (excluding bug fixes). At the moment we’re preparing for a second RC and then we will do the final release! So, please try out RC2 when it comes out and let us know if you find any issues. Otherwise, we’re gearing up for 1.5.0 (and that will definitely be an exciting release!)

What’s New in 1.4.4: New Tasks

Gendarme, NDepend and ZIP Files

Previously in CruiseControl.NET we have shied away from adding new tool-based tasks in CruiseControl.NET. This has mainly been because these tools have third-party dependencies, a lot of which we cannot guarantee their availability.

However, this approach has made things harder for administrators. The administrator needs to modify the build script to execute the tool, and then they need to modify the config to merge the results from the tool. It places the onus on the administrator to ensure the correct files are merged.

We are now trying a new approach and started adding some tool-based tasks to CruiseControl.NET. As well as executing the tool, these new tasks will merge all the results from the tool (including non-XML files). Now an administrator just needs to add the relevant task to ccnet.config and everything else is handled (although it is still possible to do things the old way as well).

In the 1.4.4 release we have added two new tasks and a new publisher.

Gendarme Analysis

Gendarme is a tool produced by the Mono project to inspect assemblies for common problems. The full details on this tool is available at http://mono-project.com/Gendarme.

This task will run the Gendarme tool and then merge the results. The tool is highly configurable (http://confluence.public.thoughtworks.org/display/CCNET/Gendarme+Task has the full details on what is available). Additionally there are now some Gendarme reports that can be included in the dashboard – including a package to install them.

The following is a brief example of how this tool could be configured:

<gendarme>

	<executable>Tools\gendarme.exe</executable>

	<assemblies>

		<assemblyMatch expr='*.dll' />

		<assemblyMatch expr='*.exe' />

	</assemblies>

	<limit>200</limit>

	<failBuildOnFoundDefects>true</failBuildOnFoundDefects>

</gendarme>

This will run the Gendarme application (tools\Gendarme.exe) and analyse all the .dll and .exe files in the working folder. It will report a maximum of 200 defects and fail the build if any defects are found.

NDepend Analysis

I’ve already posted recently on this task (read it here), so I won’t go into too many details. As the basic details, NDepend is a third party tool to analyse a code base for complexity (their website is http://www.ndepend.com/). This new tool will run the application (again with a full set of options) and then merge the results, plus there is a dashboard package to automatically install the reports.

One of the major enhancements for this task is not in the actual task itself, but in the dashboard. As part of the reporting phase of NDepend it generates a number of helpful images. Previously, these images have not been available in the dashboard (or at least not available with a work-around.) The 1.4.4 release now allows these images to be displayed, so the NDepend reports are available in their full glory.

Since this is a general fix, it also means that images can be displayed for other reports as well. Sometime in the future I’ll write a post on how this works, so other reports can also utilise this functionality.

The following is a brief example of how this task could be configured:

<ndepend>

  <project>NDepend-Project.xml</project>

  <executable>tools\NDepend.Console.exe</executable>

  <emitXml>true</emitXml>

  <outputDir>NDepend-Reports</outputDir>

  <baseDir>project\</baseDir>

</ndepend>

This will run the NDepend application (tools\NDepend.Console.exe) on the NDepend-project.xml file (this will need to be generated using NDepend). The base directory for the tool will be the project folder under the working folder and the results will be written to NDepend-Reports.

Full details on the NDepend task are available at http://confluence.public.thoughtworks.org/display/CCNET/NDepend+Task.

Package (ZIP) Publisher

It has always been possible to generate ZIP packages of files in CruiseControl.NET – just not directly from CruiseControl.NET itself (normally it’s been done via a NAnt, MSBuild or other build script task.) The 1.4.4 release now includes a built-in ZIP package generator.

This publisher will compress the specified files into a single ZIP file. The options include the level of compression, the files to include (including wild-cards), and whether to include the directory structure or not. This will generate the ZIP file, copy it to the artefacts folder and add it to a list of generated packages for the project (this list will be utilised more in future releases.)

One of the additional feature for this publisher is the ability to generate a manifest. A manifest is a list of all the included files, plus additional metadata. CruiseControl.NET uses manifest files in the dashboard packages to control the installation of the package, just to provide one example of how a manifest can be used.

Rather than forcing a set format for the manifest, it is possible to choose a manifest generator (or even write one of your own.) Currently there are only two choices – a custom format that just lists the files plus some build metadata, or a manifest importer. Hopefully in future we’ll expand the number of options (including the ability to generate a dashboard package manifest.)

The following is a brief example of how this publisher could be configured:

<package>

  <name>Binaries</name>

  <compression>9</compression>

  <manifest type="defaultManifestGenerator" />

  <files>

    <file>*.dll</file>

    <file>*.exe</file>

  </files>

</package>

This will compress all the .dll and .exe files from the working folder and store them in a ZIP file called binaries.zip using maximum compression. Additionally it will generate a CruiseControl.NET style manifest.

Again, full details on this new publisher is available on Confluence at http://confluence.public.thoughtworks.org/display/CCNET/Package+Publisher.

A Quick Wrap-Up

In this post I’ve briefly mentioned the two new tasks and the new publisher for the 1.4.4 release. We are working on adding additional tasks and expanding the current functionality for future releases, so if you think there are other tools we should integrate into CruiseControl.NET let me know.

Configuration Woes, Plus Improvements

How Do I…?

One of the common issues with CruiseControl.Net is it can be challenging to configure it – now of it’s parts have any configuration tools (except CCTray). Looking at some of the other CI tools out there (e.g. Cruise, TeamCity, etc.), it is obvious they also have picked up on this. Not only that, but they have done something about it, so much so, that there are people who are changing to these products just because they are easy to configure!

So, this is an issue for CruiseControl.Net, and not something we can easily ignore.

The Bigger Picture

One of the reasons why CruiseControl.Net doesn’t have any configuration tools is it’s flexibility. There is a huge range of possibilities, plus it is pretty easy to write extensions to CruiseControl.Net. So, how do we build a tool (or set of tools) that can cover all the possibilities?

Additionally, there is not just one part of CruiseControl.Net to configure, but multiple. Most people know about ccnet.config, fewer know about dashboard.config and even fewer know about ccnet.exe.config, ccservice.exe.config and web.config. So, the first question is why so many? And can we remove any?

But before I answer this question, what are these files?

First ccnet.config contains the project and queue configurations (and security from 1.5). This is the “main” configuration for CruiseControl.Net, in that it defines the build process. There is no way we can not have this configuration file.

Dashboard.config provides similar functionality, but for the web dashboard. It defines the servers to monitor and the plug-ins to use. Again, it is critical to the system, otherwise the dashboard would be empty! However, there are a few quirks around this file, especially around the way it is cached.

The other three files (ccnet.exe.config, ccservice.exe.config and web.config) are the .NET configuration files. They contain the settings required for things like .NET Remoting, log (i.e. Log4Net settings), a few miscellaneous settings (like where to find the actual config file, etc.) Now, we could remove these files, but it would probably break some things! Therefore we won’t be looking at removing them, although we may try to tidy them up a bit.

Now that I’ve covered the bigger picture, let’s look at improving one (little) area.

Dashboard Plug-ins

The dashboard works on a plug-in model, most of the information that is available comes from plug-ins. If the administrator doesn’t want the users to see some information, they can remove that plug-in. Likewise, new plug-ins can be added to provide additional functionality. Some plug-ins are hard-coded (e.g. the interfaces for CCTray, etc.), but otherwise the administrator has a lot of control of what is included.

To add or remove plug-ins, the administrator needs to modify dashboard.config (in the dashboard folder) and then start IIS. Additionally, if they want to add a new plug-in, they need to copy all the required files (e.g. templates, XSL-T files, etc.) into the right place. This makes adding a new plug-in a fragile process, as there is no guarantee the administrator will get everything right.

Now, in my previous job, there was the concept of a package. A package contained all the necessary files, plus a manifest that described how the files were to be used. Our application would open the package, read the manifest and then correctly deploy the files (at least it would if the manifest was right!) This would be a neat concept for CruiseControl.Net.

This would work in the following way:

1. A developer writes a new plug-in/theme/etc.
2. When it is completed, they zip all the files into a package and then add a manifest saying what needs to be copied and configured.
3. This package is then distributed.
4. An administration retrieves the package.
5. They start up the dashboard, and import the package – this then displays the package details.
6. If the administrator is happy they then install the package, which loads all the files into the correct places and adds the configuration.
7. The administrator is then done

This involves a little more work for the developer, but a lot less work for the administrator! Additionally the risk of user error is reducing significantly.

Sound good? If so, read on, because I have put together a working prototype.

Package Importer

First off, what is the package and what does it contain?

I’ve defined a package as a zip file, that contains all the necessary files, plus an XML manifest. I’ve put together an example package for the download CCTray functionality:

This contains the installer for CCTray, plus the manifest file (there are no templates or binaries required because this is part of the standard install). The manifest file is called “manifest.xml” – this is very important because the package importer needs to find the file, so I’ve based the search on the file name.

The contents of the manifest looks like this:

  1: <package>

  2:   <name>CCTray Download</name>

  3:   <description>Allow a user to download the installer for the CCTray application.</description>

  4:   <type>Plugin</type>

  5:   <folders>

  6:     <folder>

  7:       <location>CCTray</location>

  8:       <files>

  9:         <file>CruiseControl.NET-CCTray-1.4.3-Setup.exe</file>

 10:       </files>

 11:     </folder>

 12:   </folders>

 13:   <configuration>

 14:     <setting>

 15:       <path>/dashboard/plugins/farmPlugins</path>

 16:       <name>cctrayDownloadPlugin</name>

 17:     </setting>

 18:   </configuration>

 19: </package>

This defines all the files (in folder elements), plus the configuration changes to make. I’ve tried to keep the format as simple as possible, but hopefully still cover everything.

The actual package importer is a dashboard plug-in. This means an administrator can remove it if they want to (it’ll also be secured under 1.5 so the user must have admin rights).

The farm level menu for the dashboard will look like this:

There are two things to notice:

1. First there is the new package menu (I’ve highlighted it to ensure people see it, the green bits aren’t in the HTML)
2. There is no CCTray download option – I’ve removed it to show that things do work

Clicking on the menu option displays the following view:

This allows the administrator to select the package to import. Selecting a file and clicking import will copy the package to the server and validate it (it needs a manifest file). However, it doesn’t install it, instead it just tells the administrator about the package:

This allows the administrator to check that the package contains what they actually want! From 1.5, the administrator will be able to install both plug-ins and themes, hence the type label above.

If the administrator is happy, they click on “Yes” and the package is installed. Rather than forcing the administrator to guess about what has happened, a log view is displayed:

The files in gray are extra information – more useful for the developer, rather than administrator – but I’m trying to plan ahead. If there are any warnings or errors, they will also be displayed in this log.

As well as copying the files and changing the configuration, the package loader will also force a reload of the configuration. This gives the following result:

I included the before image in this screenshot, just so it is obvious what has changed. The “Download CCTray” option has been added, and I didn’t even need to restart IIS!

Where To From Here?

At the moment, this is still a very rough prototype. Here are my plans:

I want to do some more work around packages. The process currently has two steps: load the package and then install it. I’m planning on making a packages repository, where the administrator can upload multiple packages. We’ll also pre-install a number of packages to simplify installation of the dashboard.

This repository means the administrator can load packages, and then install or un-install as required. Plus they’ll be able to delete packages from the repository.

Also, as I stated above, I want to make packages reversible. If an administrator decides they don’t want a plug-in or theme in the future, they can go to the repository and un-install the package.

Finally, I want to do some tidying up around the log – it would be nice to hide the developer-only messages by default, but still give the developer the option to view them.

Anyway, these are my thoughts, what do you think of the concept?

Inner Workings: Configuration in the Dashboard

Introducing Configuration

One of the nice things about the dashboard is the ability to customise the plug-ins that it uses. These plug-ins are configured in the dashboard.config file, which sits in the root folder of the web site.

While it is easy to configure plug-ins, the question is how can we modify how the code reads the configuration to include new configuration items?

First off, configuration in the dashboard uses NetReflector. This takes care of the actual serialisation and deserialisation of the configuration. In order to add a new property it is merely a matter of adding a ReflectorPropertyAttribute to the property that needs to be serialised/deserialised. The harder questions is where should this attribute be added?

Configuration Interfaces

A quick look into the configuration folder will show the following interfaces:

These three interfaces define the information that can be retrieved from the configuration. The top interface – IDashboardConfiguration – defines the entry level configuration. This interface maps to the <dashboard> element in the configuration file.

The other two classes map to the next level in the configuration – IPluginConfiguration maps to plug ins, while IRemoteServicesConfiguration maps to <remoteServices>. Within each of these interfaces are the actual configuration properties, most of which map to other interfaces or classes.

The main point to know from here, is the dashboard stores these interfaces and uses them, instead of the actual implementations of the interfaces. The first place to add new configuration information is these interfaces.

Next, since these are interfaces, they don’t contain the NetReflector attributes – these attributes are defined on the actual implementations.

Configuration Implementations

There are four classes that implement the configuration interfaces:

NetReflectorPluginConfiguration and NetReflecotRemoteServicesConfiguration are both very straight-forward. They implement the associated interfaces and contain the NetReflector attributes. There is a simple one-to-one mapping between the properties in the interfaces and the classes, and also to the configuration elements.

DashboardConfigurationLoader is the class responsible for loading the configuration settings. Internally it uses NetReflector to do the actual serialisation and deserialisation. It also implements IDashboardConfiguration and is responsible for exposing the second-level configuration, which is where things start to get a little messy.

DashboardConfigurationLoader uses lazy-loading for retrieving the configuration. When the instance is instantiated it calls GetTypeTable() to initialise NetReflector and then stops. Later, when the application wants to retrieve either the plug ins or remote services configuration it calls the relevant method (LoadPluginsConfiguration() or LoadRemoveServicesConfiguration()) to load the configuration and then stores it internally. These methods both rely on the Load() method to find the relevant XML node in the configuration and then use NetReflector to load it.

As such DashboardConfigurationLoader does not have any NetReflector attributes in it – instead it needs to be coded to handle new configuration items. As such it is easier to add new configuration items to either <plugins> or <remoteServices> than it is to add it to <dashboard>.

The final class in configuration is CachingDashboardConfigurationLoader. This class has a internal reference to a DashboardConfigurationLoader which it uses to do the actual work. The main work this class does is storing the DashboardConfigurationLoader instance in the HttpContext cache and retrieving it from there on subsequent use.

This means the first time the dashboard starts it will load the configuration from the file – every other time it retrieves the cached configuration. This is vitally important because it means the dashboard will not pick up any changes to the configuration! If the configuration is changed then the dashboard must be restarted!

Tying It Together – the Objection Store

While this explains how the configuration works, it doesn’t explain how to get an instance of configuration to where it is needed. The answer to this question is very easy – just add a IDashboardConfiguration, IPluginConfiguration or IRemoteServicesConfiguration argument to the constructor of the class that needs it. This argument will then be populated by Objection.

Objection is an IoC library that does all the bindings between the different components. It is smart enough to auto-detect which classes implement which interfaces, and it also has the ability to be configured. CruiseObjectSourceInitializer is the class that handles this configuration and it is used for linking the configuration interfaces to the actual implementations.

For configuration, all three interfaces are loaded into the store. CachingDashboardConfigurationLoader is loaded for IDashboardConfiguration and the other interfaces are loaded from the properties on CachingDashboardConfigurationLoader.

Finally, the good news is CruiseObjectSourceInitializer is called automatically by the system prior to any other processing. This means it is then available to every other class.

Summary

Configuration in the dashboard is exposed via three interfaces: IDashboardConfiguration, IPluginConfiguration or IRemoteServicesConfiguration. IPluginConfiguration and IRemoteServicesConfiguration use simple NetReflector-based implementations for exposing configuration, while IDashboardConfiguration is implemented by a loader class.

There are two configuration loader classes – one that does the actual load and a second that caches the results. This caching loader means any configuration changes will not be detected until the dashboard is restarted.

Finally configuration is made available by requesting it in the constructor of a class. Objection is responsible for adding the configuration implementation and can do so because it has been loaded in CruiseObjectSourceInitializer before any other processing is done.