Archive

Posts Tagged ‘Configuration’

Using Ninject and XAML

27 December, 2010 2 comments

Since we’re trying to make vNext more testable we need to have a nice IoC implementation that makes it easy to wire everything together and still leverage the full power of XAML. The good news is this is really easy to do. Simon Ferquel implemented an IoC-enabled Xaml parser that used Unity as the IoC container. This was easy enough to modify to use Ninject instead.

Basically there are three classes that we need to provide implementations of:

  • XamlSchemaContext
  • XamlType
  • XamlTypeInvoker

The first class, XamlSchemaContext, provides the entry point to implementing an IoC container. This provides a context for any XAML operations including retrieving types to be instantiated. The full implementation of the class is as follows:

public class NinjectXamlSchemaContext
    : XamlSchemaContext
{
    private readonly IKernel kernel;
	
	public NinjectXamlSchemaContext(IKernel kernel)    
	{
		this.kernel = kernel;   
	}     
	
	protected override XamlType GetXamlType(string xamlNamespace,         
		string name, params XamlType[] typeArguments)    
	{        
		var retval = base.GetXamlType(xamlNamespace, name, typeArguments);        
		return retval == null ?             
			null :             
			new NinjectXamlType(this.kernel, retval.UnderlyingType, this);    
	}
}

The important part is the GetXamlType() override. This allows us to return a custom XAML type that uses Ninject instead of activation via reflection. I also added a constructor that takes in the Ninject kernel to allow storing the kernel externally but this could be moved internally if there is no need to pass the kernel around.

The second type to implement is XamlType. This provides a set of methods for hooking into the initialisation process. This implementation is:

public class NinjectXamlType    
	: XamlType
{    
	private readonly NinjectXamlTypeInvoker invoker;     
	
	public NinjectXamlType(IKernel kernel, Type type, XamlSchemaContext context)        
		: base(type, context)    
	{        
		this.invoker = new NinjectXamlTypeInvoker(kernel, type, this);    
	}     
	
	protected override XamlTypeInvoker LookupInvoker()    
	{        
		return this.invoker;    
	}     
	
	protected override bool LookupIsConstructible()    
	{        
		return true;    
	}
}

The main override is LookupInvoker(). This returns our custom invoker for new types. By itself XamlType doesn’t seem to do much it mainly appears to act as a coordinator for various look-ups, hence overriding the invoker method.

The final class is XamlTypeInvoker, which is the workhorse for our example. This type actually generates the various object instances. Thus our implementation is:

public class NinjectXamlTypeInvoker    
	: XamlTypeInvoker
{    
	private static readonly Logger logger = LogManager.GetCurrentClassLogger();    
	private readonly IKernel kernel;    
	private readonly Type typeToResolve;     
	
	public NinjectXamlTypeInvoker(IKernel kernel, Type typeToResolve, XamlType xamlType)        
		: base(xamlType)    
	{        
		this.kernel = kernel;        
		this.typeToResolve = typeToResolve;    
	}     
	
	public override object CreateInstance(object[] arguments)    
	{        
		logger.Trace("Generating an instance of '{0}'", this.typeToResolve.FullName);        
		return this.kernel.Get(this.typeToResolve);    
	}
}

Again there is one method we are overriding – CreateInstance(). In our case all we are doing is going to the kernel which creates the instance for us and wires everything up. I’ve also added so logging so we can hook in later to see what is actually happening.

The last piece of the puzzle is how to call these classes. To make this easier I have added a wrapper class:

public class ConfigurationService
{    
	private static readonly Logger logger = LogManager.GetCurrentClassLogger();    
	private readonly IKernel kernel;    
	private readonly XamlSchemaContext context;     
	
	public ConfigurationService()    
	{        
		this.kernel = new StandardKernel(new CoreModule());        
		this.context = new NinjectXamlSchemaContext(this.kernel);    
	}     
	
	public Server Load(Stream stream)    
	{        
		logger.Debug("Loading configuration from stream");        
		var reader = new XamlXmlReader(stream, this.context);        
		var server = XamlServices.Load(reader) as Server;        
		return server;    
	}
}

At the moment there is only one thing we are loading – the server configuration – but later this may change. To use this class all we need to do is create a new instance, open a stream containing the configuration and call Load().

To use dependency injection with a configuration class we just need to reference one of the standard interfaces and put an Inject attribute on it:

[DesignerSerializationVisibility(DesignerSerializationVisibility.Hidden)]
[EditorBrowsable(EditorBrowsableState.Never)]
[Inject]
public IFileSystem FileSystem { get; set; }

I also add DesignerSerializationVisibility and EditorBrowsable to make sure it does not appear in any outputted XAML and the designer.

Finally the class that wires together the various interfaces and their implementations is CoreModule. This is just a standard Ninject kernel implementation so I won’t go over it here.

Hopefully this covers all the details around how we can use Ninject together with Xaml.

CruiseControl.NET – vNext – Task Property Standards

24 December, 2010 1 comment

In my last post I mentioned that there are 211 different property names for tasks (and I haven’t gotten to the source control blocks, triggers and labellers yet!) As part of the changes I want to consolidate the different property names in CruiseControl.NET so it is easy to use.

In this post I’m going to describe some standards I think we should have for property names. Later when I look at source control blocks, triggers and labellers we’ll see how they also apply.

First off there will be a number of “common” properties that apply to every task. I covered these in a previous post so I won’t do more than just mention them here. These common properties are:

  • Name
  • Description
  • Conditions
  • FailureActions.

In addition to these properties any tasks that execute external applications will have the following common properties:

  • BaseDir: the directory where the application will be executed.
  • Environment: any additional environment variables to be passed to the application.
  • Executable: the name of the application to execute.
  • Priority: the priority level for the application.
  • Timeout: the period before the application is assumed to have timed-out. After this period the application will be killed.

Tasks that can contain other tasks will have the following common property:

  • Tasks: the child tasks

Beyond these common properties the following general rules will apply:

  1. Use the shortest form of the property that makes sense: e.g. use SolutionFile instead of BuildSolutionFile, ServerUri instead of Server, etc.
  2. If two or more tasks use the same or similar property then they should have the same name. This rule takes precedence over 1. so if a longer name is needed to make properties unique then this name would be used in all tasks. E.g. if Output is used in one task to refer to a file and OutputDir and OutputFile is used in another then OutputFile would be used in both tasks.
  3. Use Dir whenever a directory is referred to, and File whenever a file is referred to. Again this rule takes precedence over 1. E.g. SolutionFile or OutputDir. Do not use Path or Directory.
  4. Use a singular name for a single property and a plural name for an array: e.g. InputFile and InputFiles. The underlying classes for a plural property should be an array or similar these should not be used with a singular name.
  5. Sub-classes can be used to group together common properties that used in multiple tasks. E.g. a user class for holding user name/password, etc.
  6. Time-outs should be defined as a Timespan rather than an integer – this allows more flexibility in defining times.

We will probably add some more rules as we find other anti-patterns but we’ll try not to add too many rules! And as I mentioned at the beginning of this post these same rules would also apply to the other elements in a project configuration.

Finally the other common property type I am thinking of is a verbosity level. Many executable tasks have something like quiet or no logo properties which tends to be a boolean value. Some of these same tasks also have another property to increase the output level (e.g. verbose or detailed output). Therefore this type would have three values: Quiet, Normal, Detailed. If the task does not support the output level then it would be ignored.

CruiseControl.NET – vNext – Task Stats

22 December, 2010 1 comment

Before we continue with how the configuration might work in CruiseControl.NET vNext, here are a few numbers about tasks.

In the 1.6 release there are 37 different tasks (publishers are the same as tasks so they are included in this number.) Additionally there are three base classes to support them, one required interface and four optional interfaces. Six tasks are container that contain other tasks and 14 execute external applications. The majority of these tasks will be ported to vNext but some will no longer be needed (two maybe three at the moment.)

Together these 37 tasks expose 211 properties but there are a number of duplications with slightly different names. For example there is baseDir and baseDirectory – obviously the same name. Others are more different but still refer to the same thing: timeout and buildTimeoutSeconds is a example of this category. There are a few other properties that might be duplicates as their names are similar – such as server, serverName and serverUri – but we’d need to check further. Four is the least amount of properties on a task, while 44 is the most – overall there is an average of 11 properties per task. In addition there are a number of classes that encapsulate other properties for tasks such as e-mail groups, subjects and users – I haven’t included these in the count.

For vNext I am thinking of changing the names so the task names have a consistent verb-noun form (there will be some exception but not many.) For example the email task is currently called email but this will be changed to SendEmail (the meaning is a bit more obvious.) I’m also thinking of using common property names where possible but only if the properties mean the same thing. Both of these should help reduce the learning curve.

To round off, here is the current list of tasks and what I am thinking of renaming them to:

Old Name New Name
antsPerformance RunAntsPerformance
artifactcleanup CleanUpArtifacts
buildpublisher PublishFiles
checkHttpStatus CheckHttpStatus
codeItRight RunCodeItRight
commentTask Comment
conditional Select
conditionalPublisher Omitted
cruiseServerControl ControlServer
devenv BuildVisualStudio
DupFinderTask RunDupFinder
email SendEmail
exec RunExternal
fake BuildFake
FinalBuilder BuildFinalBuilder
forcebuild ForceBuild
ftp Ftp
gendarme RunGendarme
merge MergeFiles
modificationHistory PuhlishModifications
modificationReader ReadModifications
modificationWriter WriteModifications
msbuild BuildMSBuild
nant BuildNAnt
ncoverProfile RunNCoverProfile
ncoverReport RunNCoverReport
ndepend RunNDepend
nullTask Null
nunit RunNUnit
package PublishPackage
parallel Parallel
powershell RunPowershell
rake BuildRake
rss PublishRss
sequential Sequence
synchronised Synchronise
xmllogger Omitted

Let me know what you think.

CruiseControl.NET – vNext – Task Configuration

20 December, 2010 Leave a comment

Another area that will see some changes for vNext is the configuration of tasks. Currently tasks need only implement ITask which has a single method defined. There is no standard properties which means each and every task can define its own properties. Which has in turn lead to different names for the same property.

With vNext I am planning on having an abstract base class for all tasks and getting rid of the interface. This means we can add some standard properties and standard functionality for all tasks. Some of the properties are currently in the (optional) base task but two are new and deserve some mention.

But first here is an example of what the task properties may look like:

<Server Version="2.0" 
        xmlns="urn:cruisecontrol:server" 
        xmlns:tasks="urn:cruisecontrol:server:tasks"
        xmlns:conditions="urn:cruisecontrol:server:tasks:conditions"
        xmlns:failure="urn:cruisecontrol:server:tasks:failure">
  <Project Name="SampleProject">
    <tasks:Comment Name="TestComment">
      <tasks:Comment.Text>A Test Comment</tasks:Comment.Text>
      <tasks:Comment.Description>A demonstration task to show common properties</tasks:Comment.Description>
      <tasks:Comment.Conditions>
        <conditions:HasModifications />
      </tasks:Comment.Conditions>
      <tasks:Comment.FailureActions>
        <failure:RetryTask />
      </tasks:Comment.FailureActions>
    </tasks:Comment>
  </Project>
</Server>

Only one of these properties is specific to the comment task – Text – the rest are all standard properties. Name and Description are straight-forward. These will be used for identifying and labelling the task in the various logs, status reports, etc.

In 1.6 we added a conditional task which allows a block of tasks to run if a condition (or set of conditions) is met. While this was implemented very nicely it did increase the amount of structure that was needed for conditions (it was the best way to implement it without breaking things.)  In vNext I’m thinking of making the conditions block available on every task – hence the tasks:Comment.Conditions element. There will be a number of predefined conditions and people can build their if they want. This means any task can have conditions required before it will execute. This also helps with the removal of the pre-build and publishers sections as these blocks can be simulated using conditions instead.

The other new section is FailureActions. With the current version of CC.NET we have only one failure option – fail the build. With vNext people will have the choice of what to do if a task fails. By default failing tasks will also fail the build but by adding this section people can now do other things. In the above example the action is to retry (I’ll be adding some parameters around how the retry works later.) There could also be actions like ignore the failure, send an e-mail, set an internal parameter, run another task, etc. The entire purpose of this section is to allow people to decide what happens if a task fails instead of always assuming the build should fail. This also helps for various scenarios with the next task flow where a source control repository may be down but this shouldn’t effect the build.

On a final note, there is no dynamic values section. This section in 1.5 onwards was a hack to get around limitations in how tasks (and other blocks) worked to add in parameters. Under the hood it uses some nasty reflection and a couple of other tricks to set the values. Additionally using the short form involved another set of nasty hacks to convert from a user-friendly form to a machine understandable form. Instead we’ll be adding support for parameters from the ground up. I’ll cover how this will work in another post soon.

So as always, send me any comments on what you think of the new configuration. As we haven’t done anything yet this is your time to help mould the future of CruiseControl.NET.

A Small Enhancement

5 November, 2010 1 comment

One of the things about CruiseControl.NET is the configuration is all XML-based. While this is great for flexibility it is also great for shooting yourself in the foot – especially if you do not know what you are doing! During my time on the project I’ve tried to add items that make it easier to configure CC.NET (e.g. CCValidator, the XSD, etc.)

One area of the configuration that is a little weird is the definition of queues. Queues are normally defined by adding a queue attribute (or element) to a project element:

<project name="project1" queue="queue1">…</project>

If we want to change the way a queue works we had a queue element to the configuration:

<queue name="queue1">…</queue>

But here’s the catch: the two are completely separate. Yes there are some checks in the configuration loading to ensure there are no empty queues, but it is still very easy to screw up when configuring queues. One common area that gets me is if a queue name changes – I have to go through and manually check every project, plus see if there is a queue definition – not good! And what happens if a new project is added with a type-o in the queue name, e.g.:

<project name="project2" queue="que1">…</project>

Easy enough to do – but now there are two completely separate queues!

So the small enhancement I added was the ability to include projects inside a queue definition:

<queue name="queue1">
  <projects>
    <project name="project1">…</project>
    <project name="project2">…</project>
  </projects>
</queue>

Everything else still works the same way, plus it is still possible to define queues the old way (internally it converts from the new way to the old way).

Hopefully this will make things a little bit easier for the configuration.

Categories: CruiseControl.Net Tags:

A XAML Hiccup

10 March, 2010 1 comment

One of the neat things about XAML is the ability to use attached properties. These are properties that do not belong to an object, but instead to its parent object. The classic example used is the dock location for a control within a dock panel – this is associated with the control, but actually belongs to the parent.

It turns out that this “feature” is WPF specific (although it also works for Silverlight, but the two are related.) Now this may not seem like an issue, but it causes an hiccup in the XAML serialisation.

To illustrate, here is a XAML fragment that I have been working with:

<Queue Name="TestQueue">
    <Project Name="QueueProject"
         Queue.Priority="0">
        <Project.ExternalLinks>
            <ExternalLink Name="Somewhere"
                      Url="http://somewhere" />
        </Project.ExternalLinks>
    </Project>
</Queue>

This fragment contains an attached property on the project within the queue – Queue.Priority. XamlServices.Load() handles this without any problems – this property can be retrieved and it has the correct value.

Unfortunately XmlServices.Save() completely ignores it!! Here is what the XAML looks like after serialisation:

<Queue Name="TestQueue">
  <Project Name="QueueProject">
    <Project.ExternalLinks>
      <ExternalLink Name="Somewhere"
                    Url="http://somewhere" />
    </Project.ExternalLinks>
  </Project>
</Queue>

Now, all is not lost, it is still possible to serialise the attached property – we just have to use System.Windows.Markup.XamlWriter.Save() instead. That gives the following XAML:

<Queue Name="TestQueue">
  <Queue.Elements>
    <Project Name="QueueProject"
             Queue.Priority="0">
      <Project.ExternalLinks>
        <ExternalLink Name="Somewhere"
                      Url="http://somewhere" />
      </Project.ExternalLinks>
    </Project>
  </Queue.Elements>
</Queue>

But now we have another problem – the content properties have re-appeared!

So now I’m stuck – XamlServices.Save() misses out important data, while XamlWriter.Save() adds unnecessary elements (I haven’t seen if it has any of the other bloatware elements yet). In this scenario I have to go with XamlWriter.Save() as the attached properties are critical!

Perhaps this is just an issue with the RC and it will be fixed in the release – after all XamlServices.Load() handles the attached properties. I certainly hope so…

XAML Round-tripping

24 February, 2010 Leave a comment

A Small Problem

Today I decided to try the round-tripping using XAML. After all XamlServices has both loading and saving – so this could open some potential areas in future. To actually save XAML is very easy – just call the Save() method on XamlServices!

However, it didn’t look like I expected :-(

This was my original configuration:

<Project Name="PlainProject"
         ModificationDelay="1m">
    <Project.SourceControl>
        <scm:Subversion CleanUp="True"
                        Revert="True"
                        Timeout="1h">
        </scm:Subversion>
    </Project.SourceControl>
    <Project.Tasks>
        <tasks:Executable Timeout="1h">
        </tasks:Executable>
    </Project.Tasks>
</Project>

Reasonably concise and straight-forward (note, this is only a part of the entire config, but it illustrates my issue). I loaded it and then saved it and this is what I got:

<Project ArtifactDirectory="{x:Null}"
         WebURL="{x:Null}"
         WorkingDirectory="{x:Null}"
         ModificationDelay="60s"
         Name="PlainProject">
    <Project.ExternalLinks>
        <sco:ObservableCollection x:TypeArguments="ExternalLink" />
    </Project.ExternalLinks>
    <Project.PreBuildTasks>
        <sco:ObservableCollection x:TypeArguments="ProjectTask" />
    </Project.PreBuildTasks>
    <Project.Publishers>
        <sco:ObservableCollection x:TypeArguments="ProjectTask" />
    </Project.Publishers>
    <Project.SourceControl>
        <scm:Subversion CleanUp="True"
                        Revert="True"
                        Timeout="60m" />
    </Project.SourceControl>
    <Project.Tasks>
        <tasks:Executable Timeout="60m" />
    </Project.Tasks>
    <Project.Triggers>
        <sco:ObservableCollection x:TypeArguments="ProjectTrigger" />
    </Project.Triggers>
</Project>

Ouch!! Where did all this extra stuff come from?

The answer (from what I understand) is XamlServices is being pedantic! When it doesn’t know what to do (e.g. there is no data), it will literally output what is there – even though it is not what we are expecting. So what can we do about this?

Handling Nulls

Nulls are already a funny thing in development – even in .NET. After all, we have DBNull, nullable types, etc. And now, here is XamlServices with its approach to nulls. If a property has null in it, XamlServices doesn’t ignore it – instead it displays {x:Null}. I’m guessing that it is doing this because it doesn’t want to miss any data. Unfortunately for us, this just adds noise to our configuration.

Thankfully there is an easy way to handle this. XamlServices pays attention to a lot of the attributes in System.ComponentModel, and one of these attributes is DefaultValue. This allows us to associate a default value to property, and when XamlServices sees this it will ignore the property if it matches the default. So to remove these annoying {x:Null} attributes we just add:

[DefaultValue(null)]

And voila, the attribute disappears in the round-tripping :-)

Handling Empty Collections

The item that surprised me was:

<Project.Publishers>
    <sco:ObservableCollection x:TypeArguments="ProjectTask" />
</Project.Publishers>

I had marked this property as DesignerSerializationVisibility(DesignerSerializationVisibility.Content), and yet I was getting this definition. I would have expected it to be empty or (even better) missing altogether. I’m guessing this is another case of XamlServices trying to cover all its bases.

Getting rid of this is a bit harder. I initially tried DefaultValue, but since this wasn’t null it didn’t work (surprise, surprise!) However I remembered ShouldSerialize…() methods that can be added to a class. These are “special” methods that can be called to see if a property should be serialised (yes, I spell it with an ‘s’, I’m not in the States.) It turns out that XamlServices also checks for this method and if it exists will call it.

So, I added the following method:

public bool ShouldSerializePublishers()
{
    return (this.Publishers != null) &&
        (this.Publishers.Count > 0);
}

And the ObservableCollection definition disappears :-)

Some Tidier XAML

With these changes, my round-tripped XAML now looks like this:

<Project ModificationDelay="60s"
         Name="PlainProject">
    <Project.SourceControl>
        <scm:Subversion CleanUp="True"
                        Revert="True"
                        Timeout="60m" />
    </Project.SourceControl>
    <Project.Tasks>
        <tasks:Executable Timeout="60m" />
    </Project.Tasks>
</Project>

As you can see this looks almost identical to the original – so I’m a lot happier :-)

So in conclusion, it does require a couple of changes to the code, but it is possible to get some nice round-tripping with XAML. This opens the door for being able to do some nice things with the configuration in the future.

Handling Intervals – the XAML Way

22 February, 2010 1 comment

Continuing On

In my last post (read it here) I looked at using XAML for the configuration instead of NetReflector and its custom attributes. Rob Relyea from the XAML team at Microsoft saw my post and made a couple of suggestions (read his post here). One of these suggestions was using a TypeConverter to handle the conversion from a string to a value (I was using a MarkupExtension.)

Now the TypeConverter class has been around in .NET for a long time, but I have never actually used one. They are what gives the nice conversion from a value like “1cm” or “1in” to the correct number of pixels. The XAML readers use them extensively to handle converting from a string to another value type. So I thought I’d have a quick attempt at writing one.

The basis for writing a converter is to inherit from TypeConverter – nothing too hard about that – and then overriding the desired methods. To deserialise values we need to override CanConvertFrom() and ConvertFrom(), while to serialise we need to override CanConvertTo() and ConvertTo().

The two CanConvert… methods are the checks to see if the value type can be converted one way or the other. These need to be override to return true if the source/target type is a string (since this is what XAML works with.) The Convert… methods do the actual hard work of the conversion.

The final piece of the puzzle is to register the new converter so the XAML reader (or writer) will use it. This is achieved by adding a TypeConverterAttribute in the relevant place. From what I understand it can be added to either a property (XamlReader/XamlServices does not handle fields) or to a class. If used at a property level it will only be used for that property – so it is good for pre-existing types (e.g. in the BCL, etc.) or for a class that can be converted from many different values (e.g. integers, etc.) In contrast, applying it at the class level means it will be used for every occurrence of that class – unless it has been overridden at the property level.

In the end, I decided to add a new class with the converter applied to it. This saves having to apply the attribute every time we want to use the converter.

The XAML

So, my XAML for dealing with time periods changes from:

<tasks:Executable Timeout="{Time 1, Unit=Hour}" />

to:

<tasks:Executable Timeout="1h"/>

This also gives some more flexibility in that it allows for multiple units, e.g.:

<tasks:Executable Timeout="1h30m">

So in the end, it is not only more concise, it is more flexible!

The Solution

Here is how I did this. The converter class looks like this:

public class TimeIntervalConverter
    : TypeConverter
{
    public override bool CanConvertFrom(ITypeDescriptorContext context, Type sourceType)
    {
        if (sourceType == typeof(string))
        {
            return true;
        } 

        return base.CanConvertFrom(context, sourceType);
    } 

    public override bool CanConvertTo(ITypeDescriptorContext context, Type destinationType)
    {
        if (destinationType == typeof(string))
        {
            return true;
        } 

        return base.CanConvertTo(context, destinationType);
    } 

    public override object ConvertFrom(ITypeDescriptorContext context, CultureInfo culture, object value)
    {
        var stringValue = value as string;
        if (stringValue == null)
        {
            throw new ArgumentException("value must be a string", "value");
        } 

        var interval = new TimeInterval();
        // Parsing logic omitted 

        // Return the converted interval
        return interval;
    } 

    public override object ConvertTo(ITypeDescriptorContext context, CultureInfo culture, object value, Type destinationType)
    {
        var timeValue = value as TimeInterval;
        if (timeValue == null)
        {
            throw new ArgumentException("value must be a TimeValue", "value");
        } 

        if (destinationType != typeof(string))
        {
            throw new ArgumentException("destinationType must be string", "destinationType");
        } 

        timeValue = timeValue.Normalise();
        return ((timeValue.Days > 0 ? timeValue.Days.ToString() + "d " : string.Empty) +
            (timeValue.Hours > 0 ? timeValue.Hours.ToString() + "h " : string.Empty) +
            (timeValue.Minutes > 0 ? timeValue.Minutes.ToString() + "m " : string.Empty) +
            (timeValue.Seconds > 0 ? timeValue.Seconds.ToString() + "s" : string.Empty)).TrimEnd();
    }
}

I have omitted the actual logic for the parsing, since that is pretty straight-forward. The main fun is around the implementation of the TypeConverter methods. I am being paranoid and double-checking the arguments – this is probably not needed as I imagine the XamlReader will not pass in invalid parameters (especially if it is using the CanConvert… methods!)

Since I added a new class to hold the values, here it is:

[TypeConverter(typeof(TimeIntervalConverter))]
public class TimeInterval
{
    public double Days { get; set; }
    public double Hours { get; set; }
    public double Minutes { get; set; }
    public double Seconds { get; set; }

    [DesignerSerializationVisibility(DesignerSerializationVisibility.Hidden)]
    public double TotalSeconds
    {
        get
        {
            return this.Days * 86400 +
                this.Hours * 3600 +
                this.Minutes * 60 +
                this.Seconds;
        }
    }
}

This is even simpler – the TypeConverterAttribute at the top associates the converter, plus the attributes. I also added a helper property to convert to the total number of second, plus I marked it as hidden so the XamlReader won’t process it.

A Quick Thanks

Thanks to Rob Relyea for pointing this out – it’s good to see someone from Microsoft actively looking in the community to see what is happening.

CCValidator Improvements

I Need a Break!

For the past couple of months I’ve been reviewing documentation, fixing bugs, etc. – all in preparation for an RC1 release for 1.5. However, at times I need a break from the “hard slog” of software developer to play with some “fun” stuff.

And of course, I still have plenty of ideas to implement ;)

Configuration Hierarchies

One of the main issues that a lot of people have with CC.NET is its configuration. CC.NET is designed to be the ultimate in configuration flexibility, but this comes with a price – it is often more difficult to configure and diagnose problems. To help with this we added CCValidator to 1.4, and so far it sounds like it has been really helpful.

However for 1.5, the validator has been pretty much untouched (other than a few bug fixes), so I thought I’d add some funky new functionality to it. First on top of my list – alternate visualisations of the configuration.

For my first attempt, I decided to use the tried and true treeview to display the configuration, and so was born the “Hierarchy” view. Like its name suggests it uses a hierarchy to display the configuration.

How It Works

When CCValidator starts, you will see a new tab in the right-hand panel:

image

When a configuration file has been loaded, this tab gets populated:

image

At the root level there is a list of all the queues in the configuration. Expanding a queue node displays all the projects in that queue:

image

The number next to each queue is the number of projects in the queue – makes it easy to see which queues are loaded and which aren’t.

Clicking on an item (a project in the above example) displays the properties for the item. Some items also have additional sub-items, e.g. the project in the image above. This allows the user to quickly and easily drill down through the configuration.

A Work in Progress

Before everyone starts rushing off to JIRA to log faults with this new feature, let me tell you it is a work in progress! There are several known limitations:

  1. Not every node has drill-down functionality – I am still in the process of discovering what needs drill-downs and what doesn’t
  2. Not all the properties are displayed – this is because CC.NET uses a lot of fields internally instead of properties
  3. The property names are the internal names, not the configuration names
  4. The properties are editable, but nothing is saved – this is deliberate, later I will change them to read-only
  5. Some of the properties look really ugly – this is because I am using the raw data views, maybe in future I will fix this

Some of these issues I will fix, but others I will leave as they are for the moment. The reason why is very simple – I am trying not to modify any of the code in Core or Remote, only in Validator. So any changes that affect other projects will be dropped for the moment (this also means the code is a little hacky, in future I plan to remove these hacks and make the code of CC.NET more validator friendly!)

So, take a look at the new hierarchy and let me know what you think – any feedback is welcome :-)

Categories: CruiseControl.Net Tags:

Dynamic Parameters and Configuration: Inner Workings

15 December, 2009 1 comment

What’s So Hard About Them?

It appears that one of the more popular improvements for the 1.5 release is dynamic parameters (and their associated elements dynamic values). A dynamic parameter allows an administrator to define a project definition with some “placeholders”. When the project runs, it populates these “placeholders” from either user-specified values or some defaults.

Now, the concept is very, very simple – just replace some item with a value. Unfortunately because of the way configuration has been implemented in CruiseControl.NET, this is not that simple a task :-( So let’s take a look at how configuration works in CruiseControl.NET and how it has been “hacked” to allow for dynamic parameters.

From Text to Data

At the moment, all the configuration for CruiseControl.NET is loaded from a XML file (basically a plain text file). This gets converted from XML (which is not very useful to the system) to POCOs (Plain Old Class Objects). A developer new to CruiseControl.NET would think that this involves a bit of magic (I certainly did!)

The way this conversion works is via a third-part library called NetReflector. This uses attributes on the classes to convert the XML into POCOs. For example, here is a piece of the executable task:

[ReflectorType("exec")]
public class ExecutableTask
    : BaseExecutableTask
{
    [ReflectorProperty("executable", Required = true)]
    public string Executable = string.Empty; 

    [ReflectorProperty("baseDirectory", Required = false)]
    public string ConfiguredBaseDirectory = string.Empty; 

    [ReflectorProperty("buildArgs", Required = false)]
    public string BuildArgs = string.Empty; 

    [ReflectorArray("environment", Required = false)]
    public EnvironmentVariable[] EnvironmentVariables = new EnvironmentVariable[0];

    [ReflectorProperty("buildTimeoutSeconds", Required = false)]
    public int BuildTimeoutSeconds = DEFAULT_BUILD_TIMEOUT;

    // Other code omitted
}

The ReflectorType attribute at the start of the class defines <exec> as an item that can appear in the configuration. The ReflectorProperty attributes then define the allowed values on <exec>. So far, so good.

The first three properties defined are all strings – so these get converted into strings within the POCO. The fourth property is an array of EnvironmentVariables – so what happens here? This is where NetReflector does its stuff and automatically generates POCOs from the XML configuration. The same happens with the last property, which is an int, NetReflectors handles the conversion automatically.

This process is all done at start-up. CruiseControl.NET starts up, loads the configuration, runs it through NetReflector and the resulting POCOs are then fixed until the configuration is reload. In theory the values from the configuration are supposed to be read-only after the configuration has been loaded, but it is possible that the classes can change their configuration values :-(

What’s The Big Deal?

At this point, you may be wondering what the big deal is?

When people want dynamic parameters, they also want a nice easy way to use them. For example, if the project had parameters for baseDirectory and time-out, people want to use the following type of configuration:

<nant>
  <baseDirectory>$[baseDirectory]</baseDirectory>
  <buildTimeoutSeconds>$[timeout]</buildTimeoutSeconds>
</nant>

Now baseDirectory is no problem, because the underlying data type is a string. But how do we handle timeout? The underlying data type is an int – so NetReflector tries to convert this and fails with an exception! Nasty :-(

And unfortunately, due to the way NetReflector works, it is not an easy change to handle this.

Evolution of Dynamic Values

When I first started on this feature I added dynamicValues as another configuration element. So the above example would be written as:

<nant>
  <baseDirectory>C:\Somewhere</baseDirectory>
  <buildTimeoutSeconds>600</buildTimeoutSeconds>
  <dynamicValues>
    <directValue>
      <parameter>baseDirectory</parameter>
      <property>baseDirectory</property>
      <default>C:\Somewhere</default>
    </directValue>
    <directValue>
      <parameter>timeout</parameter>
      <property>buildTimeoutSeconds</property>
      <default>600</default>
    </directValue>
  </dynamicValues>
</nant>

Now this worked (more on this below), but as you can see the parameters now take up more space then the actual task definition! When the feature was first released this is the first thing they complained about.

So, I went back to the drawing board.

Now the application liked the dynamicValues element because it allowed it to change the settings when the project was run. Even though the underlying value is changed, because the dynamicValues element is static, the application can reset them because it always knows what to set. People didn’t like this because it was extra work, plus it was harder to figure out exactly what needed to be where (plus if you misspelt the property name, things didn’t work!)

In contrast people liked the short-hand version – it was similar to other programs (e.g. NAnt, MSBuild, etc.), it reduced the possibility of errors and most importantly – it was less typing. The application hated it because of NetReflector (see above) and even for string values it caused problems (the value was being overwritten.)

Thus we had two solutions – one that people liked but was technically impossible and the other that worked but was disliked. So the next step in the evolution of this feature was to add a converter from one to the other.

Configuration Pre-processing

CruiseControl.NET already has a pre-processor, but after looking at it, I went with a different approach (it was too complex to try and add it to the existing processor). The new pre-processor is specific to dynamic values and it involved hacking NetReflector a little. The change was very simple – NetReflector will check to see if a class has a pre-processor method on it, if it does it passes the XML for that class into that method to be converted.

The CruiseControl.NET side involved converting the shorthand parameters into the full XML definition. So the user would enter the first (short) version, the pre-processer would convert it and the application would see the second (long, ugly) version. This provides a win-win situation for both the application and the users :-)

Now, I could go into the technical details on how it does this, but if you really want to see how, take a look at the code. All I’ll say for now is it scans all the elements and attributes for any short-hand parameters and then generates a matching parameter for it in the configuration. It is a little nasty in the conversion step, because it needs to handle sub-properties and arrays, but it appears to be working.

And Finally, How are They Used?

The pre-processing is all to set up the dynamicValues element. This then gets converted into POCOs so the CruiseControl.NET can use them. The actual usage is at run-time. When a project runs a task (or a labeller or source control block) that has parameters it will pass the parameters into that task (or other item). In order for this to work, the item needs to implement IParamatisedItem, which is the interface that says “hey, I can use parameters”.

To help with using parameters there is a static utility class called DynamicValueUtility. This uses reflection to find the property and set its value. The tricky part was matching the configuration property name to the actual property name (since one is in the code and the other is set via an attribute), but it is all working, including handling arrays. The actual classes then call this utility class to perform the actual work.

To make things even simpler, there are a few base classes (TaskBase, LabellerBase and SourceControlBase) that implement this functionality by default. All a class needs to do is derive from one of these classes and it automatically get dynamic parameters functionality.

That’s All Folks

This has been a quick look at dynamic parameters and how they came to pass in CruiseControl.NET. Hopefully in a future version we’ll make them work even better, but considering the current limitations, this provides a working implementation :-)

Follow

Get every new post delivered to your Inbox.