A Busy Time

I haven’t been posting much recently – or doing much in the way of development for CruiseControl.NET. The reason why is simple – my life is changing!

A month ago I accepted a new job, at one of the local universities. There I will be the technical team for the team that looks after the Learning Management System for the university. So, as you can imagine, my past four weeks have been busy tidying up everything and producing reams of documentation!

At the same time, I’m going to be starting post-graduate studies. I’ve been chasing up people at the university to get my enrolment accepted – my bachelors degree was in exercise physiology, but I’m wanting to do post-grad in computer science! The easy part was getting departmental approval – the rest of the time was jumping through the hoops in the facility’s enrolment process

So, next week I’m starting my new job and becoming a student at the same time. So if I’m not doing much on CruiseControl.NET you’ll know why

XAML Round-tripping

A Small Problem

Today I decided to try the round-tripping using XAML. After all XamlServices has both loading and saving – so this could open some potential areas in future. To actually save XAML is very easy – just call the Save() method on XamlServices!

However, it didn’t look like I expected

This was my original configuration:

<Project Name="PlainProject"
         ModificationDelay="1m">
    <Project.SourceControl>
        <scm:Subversion CleanUp="True"
                        Revert="True"
                        Timeout="1h">
        </scm:Subversion>
    </Project.SourceControl>
    <Project.Tasks>
        <tasks:Executable Timeout="1h">
        </tasks:Executable>
    </Project.Tasks>
</Project>

Reasonably concise and straight-forward (note, this is only a part of the entire config, but it illustrates my issue). I loaded it and then saved it and this is what I got:

<Project ArtifactDirectory="{x:Null}"
         WebURL="{x:Null}"
         WorkingDirectory="{x:Null}"
         ModificationDelay="60s"
         Name="PlainProject">
    <Project.ExternalLinks>
        <sco:ObservableCollection x:TypeArguments="ExternalLink" />
    </Project.ExternalLinks>
    <Project.PreBuildTasks>
        <sco:ObservableCollection x:TypeArguments="ProjectTask" />
    </Project.PreBuildTasks>
    <Project.Publishers>
        <sco:ObservableCollection x:TypeArguments="ProjectTask" />
    </Project.Publishers>
    <Project.SourceControl>
        <scm:Subversion CleanUp="True"
                        Revert="True"
                        Timeout="60m" />
    </Project.SourceControl>
    <Project.Tasks>
        <tasks:Executable Timeout="60m" />
    </Project.Tasks>
    <Project.Triggers>
        <sco:ObservableCollection x:TypeArguments="ProjectTrigger" />
    </Project.Triggers>
</Project>

Ouch!! Where did all this extra stuff come from?

The answer (from what I understand) is XamlServices is being pedantic! When it doesn’t know what to do (e.g. there is no data), it will literally output what is there – even though it is not what we are expecting. So what can we do about this?

Handling Nulls

Nulls are already a funny thing in development – even in .NET. After all, we have DBNull, nullable types, etc. And now, here is XamlServices with its approach to nulls. If a property has null in it, XamlServices doesn’t ignore it – instead it displays {x:Null}. I’m guessing that it is doing this because it doesn’t want to miss any data. Unfortunately for us, this just adds noise to our configuration.

Thankfully there is an easy way to handle this. XamlServices pays attention to a lot of the attributes in System.ComponentModel, and one of these attributes is DefaultValue. This allows us to associate a default value to property, and when XamlServices sees this it will ignore the property if it matches the default. So to remove these annoying {x:Null} attributes we just add:

[DefaultValue(null)]

And voila, the attribute disappears in the round-tripping

Handling Empty Collections

The item that surprised me was:

<Project.Publishers>
    <sco:ObservableCollection x:TypeArguments="ProjectTask" />
</Project.Publishers>

I had marked this property as DesignerSerializationVisibility(DesignerSerializationVisibility.Content), and yet I was getting this definition. I would have expected it to be empty or (even better) missing altogether. I’m guessing this is another case of XamlServices trying to cover all its bases.

Getting rid of this is a bit harder. I initially tried DefaultValue, but since this wasn’t null it didn’t work (surprise, surprise!) However I remembered ShouldSerialize…() methods that can be added to a class. These are “special” methods that can be called to see if a property should be serialised (yes, I spell it with an ‘s’, I’m not in the States.) It turns out that XamlServices also checks for this method and if it exists will call it.

So, I added the following method:

public bool ShouldSerializePublishers()
{
    return (this.Publishers != null) &&
        (this.Publishers.Count > 0);
}

And the ObservableCollection definition disappears

Some Tidier XAML

With these changes, my round-tripped XAML now looks like this:

<Project ModificationDelay="60s"
         Name="PlainProject">
    <Project.SourceControl>
        <scm:Subversion CleanUp="True"
                        Revert="True"
                        Timeout="60m" />
    </Project.SourceControl>
    <Project.Tasks>
        <tasks:Executable Timeout="60m" />
    </Project.Tasks>
</Project>

As you can see this looks almost identical to the original – so I’m a lot happier

So in conclusion, it does require a couple of changes to the code, but it is possible to get some nice round-tripping with XAML. This opens the door for being able to do some nice things with the configuration in the future.

Handling Intervals – the XAML Way

Continuing On

In my last post (read it here) I looked at using XAML for the configuration instead of NetReflector and its custom attributes. Rob Relyea from the XAML team at Microsoft saw my post and made a couple of suggestions (read his post here). One of these suggestions was using a TypeConverter to handle the conversion from a string to a value (I was using a MarkupExtension.)

Now the TypeConverter class has been around in .NET for a long time, but I have never actually used one. They are what gives the nice conversion from a value like “1cm” or “1in” to the correct number of pixels. The XAML readers use them extensively to handle converting from a string to another value type. So I thought I’d have a quick attempt at writing one.

The basis for writing a converter is to inherit from TypeConverter – nothing too hard about that – and then overriding the desired methods. To deserialise values we need to override CanConvertFrom() and ConvertFrom(), while to serialise we need to override CanConvertTo() and ConvertTo().

The two CanConvert… methods are the checks to see if the value type can be converted one way or the other. These need to be override to return true if the source/target type is a string (since this is what XAML works with.) The Convert… methods do the actual hard work of the conversion.

The final piece of the puzzle is to register the new converter so the XAML reader (or writer) will use it. This is achieved by adding a TypeConverterAttribute in the relevant place. From what I understand it can be added to either a property (XamlReader/XamlServices does not handle fields) or to a class. If used at a property level it will only be used for that property – so it is good for pre-existing types (e.g. in the BCL, etc.) or for a class that can be converted from many different values (e.g. integers, etc.) In contrast, applying it at the class level means it will be used for every occurrence of that class – unless it has been overridden at the property level.

In the end, I decided to add a new class with the converter applied to it. This saves having to apply the attribute every time we want to use the converter.

The XAML

So, my XAML for dealing with time periods changes from:

<tasks:Executable Timeout="{Time 1, Unit=Hour}" />

to:

<tasks:Executable Timeout="1h"/>

This also gives some more flexibility in that it allows for multiple units, e.g.:

<tasks:Executable Timeout="1h30m">

So in the end, it is not only more concise, it is more flexible!

The Solution

Here is how I did this. The converter class looks like this:

public class TimeIntervalConverter
    : TypeConverter
{
    public override bool CanConvertFrom(ITypeDescriptorContext context, Type sourceType)
    {
        if (sourceType == typeof(string))
        {
            return true;
        } 

        return base.CanConvertFrom(context, sourceType);
    } 

    public override bool CanConvertTo(ITypeDescriptorContext context, Type destinationType)
    {
        if (destinationType == typeof(string))
        {
            return true;
        } 

        return base.CanConvertTo(context, destinationType);
    } 

    public override object ConvertFrom(ITypeDescriptorContext context, CultureInfo culture, object value)
    {
        var stringValue = value as string;
        if (stringValue == null)
        {
            throw new ArgumentException("value must be a string", "value");
        } 

        var interval = new TimeInterval();
        // Parsing logic omitted 

        // Return the converted interval
        return interval;
    } 

    public override object ConvertTo(ITypeDescriptorContext context, CultureInfo culture, object value, Type destinationType)
    {
        var timeValue = value as TimeInterval;
        if (timeValue == null)
        {
            throw new ArgumentException("value must be a TimeValue", "value");
        } 

        if (destinationType != typeof(string))
        {
            throw new ArgumentException("destinationType must be string", "destinationType");
        } 

        timeValue = timeValue.Normalise();
        return ((timeValue.Days > 0 ? timeValue.Days.ToString() + "d " : string.Empty) +
            (timeValue.Hours > 0 ? timeValue.Hours.ToString() + "h " : string.Empty) +
            (timeValue.Minutes > 0 ? timeValue.Minutes.ToString() + "m " : string.Empty) +
            (timeValue.Seconds > 0 ? timeValue.Seconds.ToString() + "s" : string.Empty)).TrimEnd();
    }
}

I have omitted the actual logic for the parsing, since that is pretty straight-forward. The main fun is around the implementation of the TypeConverter methods. I am being paranoid and double-checking the arguments – this is probably not needed as I imagine the XamlReader will not pass in invalid parameters (especially if it is using the CanConvert… methods!)

Since I added a new class to hold the values, here it is:

[TypeConverter(typeof(TimeIntervalConverter))]
public class TimeInterval
{
    public double Days { get; set; }
    public double Hours { get; set; }
    public double Minutes { get; set; }
    public double Seconds { get; set; }

    [DesignerSerializationVisibility(DesignerSerializationVisibility.Hidden)]
    public double TotalSeconds
    {
        get
        {
            return this.Days * 86400 +
                this.Hours * 3600 +
                this.Minutes * 60 +
                this.Seconds;
        }
    }
}

This is even simpler – the TypeConverterAttribute at the top associates the converter, plus the attributes. I also added a helper property to convert to the total number of second, plus I marked it as hidden so the XamlReader won’t process it.

A Quick Thanks

Thanks to Rob Relyea for pointing this out – it’s good to see someone from Microsoft actively looking in the community to see what is happening.

From NetReflector to XAML

A Quick Intro

I always seem to have a big backlog of things to look into – ideas to play around with, concepts to try, etc. – and lately I haven’t had much time to investigate. However, with my pending change to a new job I sometime have small periods with nothing to do (or when I don’t want to start a larger task.) So it is an ideal time to “play” around with some ideas.

Warning: This is just an idea at this point in time (i.e. a proof of concept). It may make it into a future version of CruiseControl.NET. But then again, it might not!

One of the ideas I have been wanting to investigate for a while is using XAML for the configuration. Currently CruiseControl.NET uses a custom serialisation called NetReflector. As far as I am aware, we are the only people using this library, so it doesn’t get a lot of maintenance. Now it works reasonably well (especially considering it was first developed for .NET 1.0!), but there are some limitations with it. I have added some small tweaks to it over time, but it would be nice to move to something more mainstream.

XAML – A New Serialisation?

Anyone who has been working with WPF or Silverlight (or even WF) will be familiar with XAML. XAML is an extension to XML that provides a consistent format for deserialisable definitions. The .NET framework has included classes for reading and writing XAML since .NET 3.0. Unfortunately, there were two (three with Silverlight) versions of these classes with .NET 3.0 and 3.5 – one for WPF and another for WWF (later WF). The good news with .NET 4.0 is there will be a new namespace just for XAML – System.Xaml which will provide a single entry point for working with XAML (although from what I understand WPF will still have its own implementation.)

XamlReader (XamlServices in .NET 4.0) provides methods for loading a XAML definition and converting it into POCO instances. These instances can be any class – they don’t have to be WPF or WF. This means a developer can write their classes and then use XamlReader to load them, without any extra work on their part! Of course, there are attributes that can extend the functionality, but they are optional.

So, in theory it would be nice and easy to change to use XamlReader (or XamlServices when we move to .NET 4.0) and get rid of NetReflector

The Plan

To test this concept, I decided to build a quick configuration reader and some configuration classes (in the future I would like to change how CruiseControl.NET handles its configuration, but that is a post for another time). I’m not going to modify the existing classes, but instead build a separate configuration section. If this is used in future we will probably do it in a different way, but this is just to test the concept.

Current Configuration

To test the configuration I’m going to use a modified version of the configuration for CruiseControl.NET on ccnetlive (modified because its only a proof of concept, I don’t want to have to generate the entire configuration model!)

Here is the configuration I converted:

<?xml version="1.0" encoding="utf-8" ?>
<cruisecontrol>
  <project name="CCNet" queue="CCNet">
   <webURL>http://ccnetlive.thoughtworks.com/ccnet/server/local/project/CCNet/ViewLatestBuildReport.aspx</webURL>
    <modificationDelaySeconds>30</modificationDelaySeconds>
    <workingDirectory>e:\sourcecontrols\sourceforge\ccnet</workingDirectory>
    <artifactDirectory>e:\download-area\CCNet-Builds\1.5.0</artifactDirectory>
    <maxSourceControlRetries>3</maxSourceControlRetries>
    <triggers>
      <intervalTrigger seconds="300"/>
    </triggers>
    <sourcecontrol type="svn">
      <executable>E:\tools\svn.bat</executable>
      <revert>true</revert>
      <cleanUp>true</cleanUp>
      <trunkUrl>https://ccnet.svn.sourceforge.net/svnroot/ccnet/trunk</trunkUrl>
      <timeout>1800000</timeout>
    </sourcecontrol>
    <tasks>
      <nant>
        <executable>Tools\NAnt\NAnt.exe</executable>
        <baseDirectory>e:\sourcecontrols\sourceforge\ccnet</baseDirectory>
        <buildFile>ccnet.build</buildFile>
        <buildArgs>-listener:CCNetListener,CCNetListener</buildArgs>
        <targetList>
          <target>all</target>
        </targetList>
        <buildTimeoutSeconds>1000</buildTimeoutSeconds>
      </nant>
      <buildpublisher>
        <sourceDir>e:\sourcecontrols\sourceforge\ccnet\Publish</sourceDir>
        <publishDir>e:\download-area\CCNet-Builds\1.5.0</publishDir>
        <useLabelSubDirectory>true</useLabelSubDirectory>
      </buildpublisher>
    </tasks>
    <publishers>
      <merge>
        <files>
          <file>BuildMetrics\*-result.xml</file>
          <file>BuildMetrics\NDependOut\*.xml</file>
        </files>
      </merge>
      <xmllogger />
      <artifactcleanup cleanUpMethod="KeepLastXBuilds"
                       cleanUpValue="75" />
      <artifactcleanup cleanUpMethod="KeepLastXSubDirs"
                       cleanUpValue="75" />
      <rss />
    </publishers>
    <externalLinks>
      <externalLink name="CruiseControl.NET Home Page"
                    url="http://ccnet.thoughtworks.com/" />
    </externalLinks>
  </project>
</cruisecontrol>

New Configuration

After some playing around (and a bit of research) I came up with the following configuration:

<?xml version="1.0" encoding="utf-8" ?>
<CruiseControl xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
               xmlns="http://www.thoughtworks.com/ccnet/2/0"
               xmlns:tasks="http://www.thoughtworks.com/ccnet/2/0/tasks"
               xmlns:scm="http://www.thoughtworks.com/ccnet/2/0/sourcecontrol">
    <Project Name="CCNet" Queue="CCNet"
             ArtifactDirectory="e:\download-area\CCNet-Builds\1.5.0"
             WorkingDirectory="e:\sourcecontrols\sourceforge\ccnet"
            WebURL="http://ccnetlive.thoughtworks.com/ccnet/server/local/project/CCNet/ViewLatestBuildReport.aspx"
             ModificationDelay="{Time 5, Unit=Minutes}">
        <Project.Triggers>
            <Interval Period="{Time 5}"/>
        </Project.Triggers>
        <Project.SourceControl>
            <scm:Subversion Executable="E:\tools\svn.bat"
                            Timeout="{Time 30, Unit=Minutes}"
                            TrunkUrl="https://ccnet.svn.sourceforge.net/svnroot/ccnet/trunk"
                            Project.MaxRetries="3"
                            Revert="True"
                            CleanUp="True">
            </scm:Subversion>
        </Project.SourceControl>
        <Project.Tasks>
            <tasks:Nant BuildFile="ccnet.build"
                        Timeout="1000"
                        Executable="Tools\NAnt\NAnt.exe"
                        BuildArgs="-listener:CCNetListener,CCNetListener"
                        BaseDirectory="e:\sourcecontrols\sourceforge\ccnet">
                <tasks:Nant.Targets>
                    <tasks:Target>{Parameter buildType}</tasks:Target>
                </tasks:Nant.Targets>
            </tasks:Nant>
            <tasks:BuildPublisher SourceDir="e:\sourcecontrols\sourceforge\ccnet\Publish"
                                  PublishDir="e:\download-area\CCNet-Builds\1.5.0"
                                  UseLabelSubDirectory="True" />
        </Project.Tasks>
        <Project.Publishers>
            <tasks:Merge>
                <tasks:MergeFile>BuildMetrics\*-result.xml</tasks:MergeFile>
                <tasks:MergeFile>BuildMetrics\NDependOut\*.xml</tasks:MergeFile>
            </tasks:Merge>
            <tasks:XmlLogger />
            <tasks:ArtifactCleanup CleanUpMethod="KeepLastXBuilds"
                                   Value="75" />
            <tasks:ArtifactCleanup CleanUpMethod="KeepLastXSubDirs"
                                   Value="75" />
        </Project.Publishers>
        <Project.ExternalLinks>
            <ExternalLink Name="CruiseControl.NET Home Page"
                          Url="http://ccnet.thoughtworks.com/" />
        </Project.ExternalLinks>
    </Project>
</CruiseControl>

This is a fully working example – I have put together a loader that will handle it (actually, I’m just calling XamlReader, it does all the work for me!)

Namespaces

First thing to note, there are multiple namespaces. I have added namespace mappings so they give nice URL namespaces, instead of the CLR reference assemblies. There are also multiple namespaces that are needed – the default namespace (http://schemas.microsoft.com/winfx/2006/xaml), plus a namespace for tasks (http://www.thoughtworks.com/ccnet/2/0/tasks) and source control (http://www.thoughtworks.com/ccnet/2/0/sourcecontrol). This is because certain items in the old way needed a meaningless suffix just to identify it from another item with a similar name. The prime example is null items – there is a nullTask, a nullSourceControl, etc. Now they are tasks:Null and scm:Null – a more consistent naming style.

The namespaces also means people can add their own tasks with pre-existing names. Don’t like the way tasks:Null works? You can add your own version – your:Null. It has the same basic name, so people know it does the similar functionality, but since it has its own namespace it will go to your version.

No More Types

The type attribute has been removed from SourceControl (and all other similar examples). Instead the item is defined by adding it as the content, e.g. <scm:Subversion>.

In the old configuration, the type attribute is a nuisance. It is handled in a different way from all the other configuration settings (from an internal code view). This change moves it more in line with the rest of the configuration (plus it is needed for XAML!)

Times and Parameters

Times were another item that are not handled well in the existing configuration. In some places they are an integer value (seconds), other places they are a custom time (that allows specifying the unit).

The XAML version uses a much simpler approach – I have added a mark-up extension to handle the conversion. For example, on the Subversion block the timeout block has “{Time 30, Unit=Minutes}”. This will set the timeout to 30 minutes, or 500 seconds.

This will happily work anywhere there is a time period. Under the hood the property is an integer (will probably need to change to a double). I added a new MarkupExtension called Time that converts from the above syntax to a number of seconds.

Parameters will also work in the same way, but there is one issue I haven’t resolved (yet). Parameters are dynamic values – they change per build. XAML loads a static instance, and it doesn’t change unless re-loaded. This means the {Parameter} setting will be loaded when the configuration is loaded – but we need it to change for every build. Gives me something to look into

In Conclusion

This will require a significant change to the configuration files, but it will simplify the internal workings of configuration a lot! But it offers some nice changes to the syntax, which will simplify things in the long run.

At this point the main stopping point is the requirement for .NET 3.0 or later – CruiseControl.NET still targets .NET 2.0. Perhaps when we do a .NET 4.0 version of CruiseControl.NET we can look at making these changes.

Let me know what you think…