What’s New in 1.4.4: Miscellaneous Improvements

We’ve Updated the Current Stuff Too!

As well as adding new functionality, we’ve also taken some time to improve some of the existing components. Some of these improvements are corrections (i.e. making things work like they should have originally), others are brand-new.

Areas that have been improved include: the validator, configuration, source control, the console/service and the RSS feeds.

The Validator

The validator was originally added in the 1.4.3 release, but unfortunately due to an oversight by the development team (actually, by me!) it was not included in the normal distribution. Instead the code needed to be downloaded and compiled (not good for non-developers!)

This has now been fixed and the validator will be installed as part of the normal server install. As well as the console and uninstall short-cuts, a new short-cut will be added for the validator (CCValidator). Additionally, the validator now has its own separate installer – so it can be installed without also installing the server components (or the dashboard). Hopefully this will make it easier to get the validator and use it.

As well as tidying up the install process, the validator now offers some internal validations. Previously the validator would only validate that the file could be loaded (i.e. there were no load errors), it wouldn’t perform any checks to ensure that the settings were actually valid. Now the validator will start to offer some internal validation, and we certainly plan on improving this in future releases.

Configuration

This is kind of an overflow from the validator, but we’ve also tried to make errors in the configuration easier to understand. This involved some work on NetReflector (since this is what converts the XML into the .NET objects), and mainly involved adding more meaningful messages.

In a nutshell, we’ve tried to replace any generic exceptions with specific NetReflector exceptions. These exceptions provide some more information on the error (e.g. unable to convert to specified type, base types do not allow attributes, etc.), plus they also include the erroneous XML element. This information will make it easier to track the problem (where do we start with a NullReferenceException!) and hopefully easier to pinpoint the exact cause.

However, this is a work in progress – we can’t guarantee we’ve got all the exceptions covered. So if you find any errors that don’t make sense, feel free to let us know.

Source Control

In the 1.4.3 release we finally started logging errors with source control. This certainly opened a can of worms as a lot of installations worked on the assumption that source control errors were ignored.

Based on the feedback we’ve added a couple of new options to define how source control errors should be handled.

maxSourceControlRetries defines how many source control errors are allowed before the next stage happens

stopProjectOnReachingMaxSourceControlRetries (yes, it’s a long name) controls what happens in the next stage. If this is set to true, then the project will be stopped. This allows an administrator to work out what is wrong and then fix things without overloading the source control server. Setting it to false means the project will continue running.

sourceControlErrorHandling defines the options for what happens when an error occurs. The valid options are:

• ReportEveryFailure – this is the current functionality of 1.4.3 – every source control exception will be reported.
• ReportOnRetryAmount – only report once when the maximum number of attempts has been made. Any subsequent errors will be ignored.
• ReportOnEveryRetryAmount – only report when the maximum number of attempts has been made. After this the counter is reset and the error reported again after the next number of attempts has failed.

The retry attempt counter is always reset back to zero after a successful source control operation – no matter which option is chosen.

Additionally cleanUp and revert options have been added to the Subversion source control block.

Console/Service

Both the console and the service now have the ability to hot-swap DLLs. This is similar to ASP.NET where a binary can be x-copied into the folder and the application will automatically restart.

This feature has been added to make it easier to upgrade running instances of CruiseControl.NET. Previously the server (either console or service) would need to be stopped, the binaries copied over and then the server restarted. Now it is just a simple matter of copying the new binaries in and the server will take care of the rest.

The only file that cannot be copied over is the application executable (e.g. ccnet.exe or ccservice.exe).

RSS Feeds

Previously the RSS publisher in CruiseControl.NET only reported on the last build – previous builds were overwritten. The publisher now can publish multiple builds – which allows the feed to have a history of what has happened.

By default, the last twenty builds will be in the RSS feed, but this can be configured by adding an items property to the task.

<rss items="15"/>
In Conclusion
So, as well as adding new functionality, we’ve been tweaking the existing functionality to make things easier. However these new tweaks and features will remove some of the headaches people have been having.
As always, if you find any issues or problems, please let me know and I’ll look into them.
Now, this pretty much wraps up a lot of the changes for 1.4.4 (excluding bug fixes). At the moment we’re preparing for a second RC and then we will do the final release! So, please try out RC2 when it comes out and let us know if you find any issues. Otherwise, we’re gearing up for 1.5.0 (and that will definitely be an exciting release!)
								

			

What’s New in 1.4.4: New Tasks

Gendarme, NDepend and ZIP Files

Previously in CruiseControl.NET we have shied away from adding new tool-based tasks in CruiseControl.NET. This has mainly been because these tools have third-party dependencies, a lot of which we cannot guarantee their availability.

However, this approach has made things harder for administrators. The administrator needs to modify the build script to execute the tool, and then they need to modify the config to merge the results from the tool. It places the onus on the administrator to ensure the correct files are merged.

We are now trying a new approach and started adding some tool-based tasks to CruiseControl.NET. As well as executing the tool, these new tasks will merge all the results from the tool (including non-XML files). Now an administrator just needs to add the relevant task to ccnet.config and everything else is handled (although it is still possible to do things the old way as well).

In the 1.4.4 release we have added two new tasks and a new publisher.

Gendarme Analysis

Gendarme is a tool produced by the Mono project to inspect assemblies for common problems. The full details on this tool is available at http://mono-project.com/Gendarme.

This task will run the Gendarme tool and then merge the results. The tool is highly configurable (http://confluence.public.thoughtworks.org/display/CCNET/Gendarme+Task has the full details on what is available). Additionally there are now some Gendarme reports that can be included in the dashboard – including a package to install them.

The following is a brief example of how this tool could be configured:

<gendarme>

	<executable>Tools\gendarme.exe</executable>

	<assemblies>

		<assemblyMatch expr='*.dll' />

		<assemblyMatch expr='*.exe' />

	</assemblies>

	<limit>200</limit>

	<failBuildOnFoundDefects>true</failBuildOnFoundDefects>

</gendarme>
This will run the Gendarme application (tools\Gendarme.exe) and analyse all the .dll and .exe files in the working folder. It will report a maximum of 200 defects and fail the build if any defects are found.
NDepend Analysis
I’ve already posted recently on this task (read it here), so I won’t go into too many details. As the basic details, NDepend is a third party tool to analyse a code base for complexity (their website is http://www.ndepend.com/). This new tool will run the application (again with a full set of options) and then merge the results, plus there is a dashboard package to automatically install the reports.
One of the major enhancements for this task is not in the actual task itself, but in the dashboard. As part of the reporting phase of NDepend it generates a number of helpful images. Previously, these images have not been available in the dashboard (or at least not available with a work-around.) The 1.4.4 release now allows these images to be displayed, so the NDepend reports are available in their full glory.
Since this is a general fix, it also means that images can be displayed for other reports as well. Sometime in the future I’ll write a post on how this works, so other reports can also utilise this functionality.
The following is a brief example of how this task could be configured:
<ndepend>

  <project>NDepend-Project.xml</project>

  <executable>tools\NDepend.Console.exe</executable>

  <emitXml>true</emitXml>

  <outputDir>NDepend-Reports</outputDir>

  <baseDir>project\</baseDir>

</ndepend>
This will run the NDepend application (tools\NDepend.Console.exe) on the NDepend-project.xml file (this will need to be generated using NDepend). The base directory for the tool will be the project folder under the working folder and the results will be written to NDepend-Reports.
Full details on the NDepend task are available at http://confluence.public.thoughtworks.org/display/CCNET/NDepend+Task.
Package (ZIP) Publisher
It has always been possible to generate ZIP packages of files in CruiseControl.NET – just not directly from CruiseControl.NET itself (normally it’s been done via a NAnt, MSBuild or other build script task.) The 1.4.4 release now includes a built-in ZIP package generator.
This publisher will compress the specified files into a single ZIP file. The options include the level of compression, the files to include (including wild-cards), and whether to include the directory structure or not. This will generate the ZIP file, copy it to the artefacts folder and add it to a list of generated packages for the project (this list will be utilised more in future releases.)
One of the additional feature for this publisher is the ability to generate a manifest. A manifest is a list of all the included files, plus additional metadata. CruiseControl.NET uses manifest files in the dashboard packages to control the installation of the package, just to provide one example of how a manifest can be used.
Rather than forcing a set format for the manifest, it is possible to choose a manifest generator (or even write one of your own.) Currently there are only two choices – a custom format that just lists the files plus some build metadata, or a manifest importer. Hopefully in future we’ll expand the number of options (including the ability to generate a dashboard package manifest.)
The following is a brief example of how this publisher could be configured:
<package>

  <name>Binaries</name>

  <compression>9</compression>

  <manifest type="defaultManifestGenerator" />

  <files>

    <file>*.dll</file>

    <file>*.exe</file>

  </files>

</package>


This will compress all the .dll and .exe files from the working folder and store them in a ZIP file called binaries.zip using maximum compression. Additionally it will generate a CruiseControl.NET style manifest.
Again, full details on this new publisher is available on Confluence at http://confluence.public.thoughtworks.org/display/CCNET/Package+Publisher.
A Quick Wrap-Up
In this post I’ve briefly mentioned the two new tasks and the new publisher for the 1.4.4 release. We are working on adding additional tasks and expanding the current functionality for future releases, so if you think there are other tools we should integrate into CruiseControl.NET let me know.
								

			

What’s New in 1.4.4: Dashboard Administration Plug-in

Looking at the Competition

CruiseControl.NET is not the only Continuous Integration (CI) server available – both on the .NET scene and outside of it. One of the items that people like about some of our competitors is their ease of configuration and simplicity to learn. Of course, I still think CruiseControl.NET is one of the best CI servers around, but they obviously have a good selling point here!

To try and make things easier for people we have added a new plug-in to the dashboard – a web dashboard administration plug-in. This plug-in is designed to simplify administration of the dashboard (sorry, nothing on the main config … yet!)

This plug-in has three main areas:

• remote servers
• packages
• reload dashboard

Remote Servers

The dashboard is actually independent of the CI server – it doesn’t even need to be on the same machine (by convention a lot of people leave it on there, but it doesn’t need to be.) This is configured by the <remoteServices> element in dashboard.config, and it is possible to not only connect to a remote server, but multiple remote servers.

The new plug-in allows an administrator to add, modify or delete remote servers – all within the plug-in. As soon as a server is added, modified or deleted, the configuration is updated and the new details are available. There is no validation on the servers, but it will be obvious very quickly in the main grid if a server is incorrect!

Packages

The second area – packages – is completely new to CruiseControl.NET. Previously, adding a new report or plug-in involved a series of manual steps. These steps could have included copying one or more files and modifying dashboard.config. The problem with a set of manual instructions is the ease at which they can be screwed up (copied the wrong file to the wrong place, or misspelt a configuration element).

To remedy this we have added packages to the dashboard. A package is a ZIP file containing all the required files, plus an instruction manifest telling the dashboard what to do. There is no longer any need to have a manual set of instructions to follow.

Adding a package is a two-step process. First the package must be uploaded to the server (loading a package). This just copies the package file and updates the list of packages – it doesn’t make the new functionality available. To enable the functionality the package must be installed. This is as simple as clicking on the package and choosing install – the dashboard handles everything else!

And another great point about packages – they are reversible! You can uninstall a package (which will remove the files and configuration elements) and even unload it from the server.

Currently the following packages are available:

• Fitnesse Results
• FxCop Results
• Gendarme Results
• Modification History
• MSBuild Results
• MSTest Results
• NAnt Results
• NCover Results
• NDepend Results
• NUnit Results
• Project Configuration Display
• Project Statistics
• Queue Status Display
• Security Configuration Display
• Server Configuration Display
• Server Information Display
• Server Log Display
• Simian Results
• User List
• Web Dashboard Administration

Over time we will add more packages and also additional functionality to the way they work.

Reload Dashboard

As well as using the remote services and packages functionality, it is possible to do a configuration reload. This is needed for one simple reason – the configuration is cached!

For performance reasons, the configuration is only loaded when the application is started. Subsequent calls to the dashboard uses the cached configuration. This then causes a problem when dashboard.config is modified. Unless the web application is reloaded (i.e. by restarting the web server), the new configuration is ignored!

The reload dashboard commands allows us to get around this problem. An administrator can manually modify the config file and then click on reload dashboard to force a reload of the configuration.

Security

This new plug-in is reasonably powerful, but with great power comes great responsibility (guess the movie!) With this plug-in it also makes it incredibly easy for something to screw things up in the dashboard (either by accident or on purpose!) To limit this, it is possible to set a password on the plug-in so only authorised people can access it.

This password is set for the entire dashboard (it is independent of the security changes for 1.5.0) and is set in dashboard.config. To set a password merely add a password attribute to the <administrationPlugin> element. The value of the attribute is the password.

If this element is empty or missing, then anyone can access the plug-in.

Initial State

Since 1.4.4 is a mainly a fix release this plug-in is not enabled by default. Instead an administrator will need to manually modify dashboard.config. The <administrationPlugin> element will need to be added to the <farmPlugins> element (including setting a password if required).

Once this element has been added, then the web server will need to be restarted to load the configuration. And the new plug-in is ready to go.

So, I hope this makes things easier for any administrators out there (and any plug-in developers). Let me know if there is anything we can do to improve it.

What’s New in 1.4.4: Server Extensions

A Quick Intro

As well as a number of bug fixes, we’ve added some new functionality to 1.4.4. Most of this functionality as been around for a while, mainly in the security branch, so I thought I’d review some items. Over my next few blog posts I’ll look at some of the new items that have been added, and the possibilities they open up.

Expanding Expandability

One of the great things about CruiseControl.NET is its flexibility, and if the trunk distribution doesn’t contain what you want, it’s pretty easy to add your own extensions. The only real limitation is the allowed places for extensions – they are limited to being items in the configuration file (ccnet.config), like tasks, source control, triggers to name a few common ones.

With release 1.4.4 it is now possible to build a “global” extension to the server. These extensions have nothing to do with ccnet.config, instead they are added in the application configuration (ccnet.exe.config or ccservice.exe.config).

These extensions work at a deeper level than the older extensions. The older extensions had to work within the context of a build, they could affect the outcome of a build, but not how the actual server worked. The new extensions allow the actual functionality of the server to be modified!

Sounds exciting? I certainly think so!

Some Possibilities

These extensions now allow developers to influence how the server works, and allows developers to open new channels to the server.

As an example, currently in CruiseControl.NET there are projects and there are queues. It is possible to limit builds, but only by adding projects to a queue. If you wanted to limit the server so only five projects could build, no matter which queue, you were in trouble! Now with server extensions, we can build an extension to allow only five projects to build, no matter which queue they are in (and yes, I have built it).

A second possibility is to extend the logging. For example, an administrator might want to record all the force builds, or maybe just the number of times a project is stopped and started, or how often builds are triggered. Currently, they’d have to go through the log (assuming the correct log level is set) and manually count all the instances. Now, we could build an extension that just counts the number of force builds, stops, starts, etc. (sorry haven’t written this one yet).

The third possibility I can think of is with communications. Currently there is only one communications channel to the server – .NET Remoting (yes, there is HTTP, but this goes via the dashboard and still uses .NET Remoting to connect to the server). The new extensions now allow us to add any types of communications channels we want. I’ve built a simple one that uses WCF as the underlying channel, the only hard part was translating the objects – since I couldn’t add the WCF serialisation attributes to the existing types.

Time to Code

I’ve listed a few possibilities for the new server extensions, over time I’ll try to implement these and add a few more options to the list.

If you want to try and build an extension, I’ve started on the documentation (http://confluence.public.thoughtworks.org/display/CCNET/Server+Extensions), although I still need to work on it. In a nutshell, ThoughtWorks.CruiseControl.Remote.ICruiseServerExtension must be implemented (it only has four methods) and then the extension registered in the application config.

So have a play, let me know what you think. Also, I’m open to any ideas for extensions – just send me the idea and I’ll take a look at it.

Happy coding…