Here are the cmdlets that are current exposed:

Most of these were in the previous version – but there is one major change – New-CCConnection has replaced Get-CCServer. I’ll come back to why for this later on.

So with the changes the commands now look like this:

To work with projects we now directly use Get-CCProject rather than using Get-ChildItem. This has been simplified to take in either the server name or connection and directly return the projects. This can also be filtered by putting in a name parameter.

In the screenshot above I’m directly working against the name of the server (in this case localhost.) To allow this I’ve made some changes to the name resolution. If connecting to localhost or 127.0.0.1 it will use .NET Remoting (TCP connections) – all other names will default to HTTP. It is still possible to connect via .NET Remoting – just have to put in the full URI for the server (including the protocol.)

I’ve also expanded the connections out so it is possible to use a backwards compatible connection (for connecting to pre-1.5 version servers) and allow encrypted connections. These can be turned on by adding the relevant flag to the command (note – they are mutually exclusive – if both are used an error will be thrown!)

Now to simplify connecting to a server I have added a cmdlet called New-CCConnection. This will take in the parameters for a connection and return a connection instance (if the connection is valid.) This connection can then be used in any of the commands. Now, while this may seam to add overhead (beyond reducing the number of keystrokes per command) the beauty of the command comes when we want to use security. For example if we have a secured server:

In this screenshot I started two connections – the first time it was an unsecured connection, the second it was secured. Both connections can view the projects (this is the way security has been configured) but only the second connection can force a build.

And for a final fun example, here is a screen shot showing how it is possible to work with multiple servers at once:

I don’t actually have two servers running – hence all the projects are duplicated – but it shows how we can set up two different connections and pipe them into the Get-CCProject cmdlet to retrieve all the projects. Note there are no queues for the connection that uses backwards compatibility as this feature was added in v1.5.

So hopefully this is more useful now. For my next post I’ll try and add some new functionality to make the cmdlets more useful

I’m currently playing around with PowerShell in my job at the moment. Basically we never have enough time, so I’m trying to figure out how to automate various tasks via PowerShell. And I’ll admit, so far I like it! It is easy to use, has a lot of power and flexibility and there are limitless possibilities for what can be done. But this post isn’t about PowerShell in general – instead I’ve been playing around with building a PowerShell provider for CruiseControl.NET.

“What? But why?” you might say. A while back I built a command-line tool that allows people to interact with a CruiseControl.NET server. But it is very limited and is fairly complex for what it actually does (in terms of the code for it.) As I was playing around with PowerShell I started thinking a lot of the syntax in PowerShell would allow me to build a much better command-line tool, especially when I started seeing the extensibility options for PowerShell.

So I have started putting together a PowerShell provider for CruiseControl.NET. This allows an administrator to connect to a CruiseControl.NET server and administer it. As I’m still playing around with things the code is production quality yet, so use it at your own peril! There is no installer – you will need to grab the code off SourceForge and build it, then copy the contains of the PowerShell\Bin\Debug folder into your modules directory (see help about_modules in PowerShell.)

However if you are interested in seeing what can be done, take a look at the following screen shots:

Here I am:

• Importing the module
• Seeing the new drive that has been mapped
• Getting the child items in the various folders
• Starting all the projects
• And forcing a build for all the projects

I like how I can work with multiple projects at the same time – in my previous tool we would have needed to run the tool once for each project!

I have implemented the cmdlets in the provider to use the pipeline, so we can use a lot of the functionality within PowerShell. For example if we only wanted to get the projects for a certain queue:

This is something that wasn’t even thought of in the command-line tool! And here it is, without any extra work on my behalf

At the moment I have only implemented four cmdlets:

• Get-Log
• Start-Project
• Stop-Project
• Start-Build

Plus the provider infrastructure (i.e. you can use built-in cmdlets like Get-Item, Get-ChildItem, Set-Location, New-PSDrive, etc.) Currently I have mapped the local server to local:. It is possible to connect to other server using New-PSDrive as long as you know the address for it (this is the same address that CCTray uses.) I have implemented the provider as a hierarchy. The top level is the server itself, with two sub-folders: Queues and Projects.

My next step is to add builds in, but I think this is going to need some changes on the server side to work.

Hopefully this will be helpful to someone

In vNext I want to make adding a new method as simple as setting an attribute on a method I’ve already talked a bit about the design in my posts on Universal Naming, Three External Methods and Invokable Methods, with this in place I was ready to test whether it would work or not.

To do this I put together a simple command-line application to work as an interactive “console”. This has a few simple commands that the user selects for performing the various operations. It’s not much to look at but it does allow some simple testing:

The main commands are:

• query (? for short): this lists the available commands for the current URN
• invoke command (.command for short): invokes the command on the remote server
• up name (+name for short): adds the name to the current URN
• down (- for short): removes the last part of the current URN
• quit: exits the console

This provides the basics for querying, invoking and navigating the remote server. In the above screenshot I’ve shown an example of how this works.

Note: this console has a large number of limitations at the moment – it is only a proof of concept for testing!!

In the above screenshot there are only two commands available on a project – Start() and Stop() – not enough to do much with a project. It would be nice to add a command to retrieve the current status. So I opened the code for Project and added the following method:

public virtual Messages.SingleValue GetStatus(Messages.Blank request)
{
    var response = new Messages.SingleValue(this.State.ToString());
    return response;
}

The method is very simple – all it does is return the current state as a string. In order to expose a method in the API a method needs to have a single input argument and a return value. In the Common library I’ve added some basic messages in the Messages namespace to simplify this process.

Note: at the moment the console only handles messages that take in a blank message – I did warn you there are severe limitations

The next step is to expose this as an action in the API. This is as simple as adding a RemoteAction attribute:

[RemoteAction]
[Description("Retrieves the current status of the project.")]
public virtual Messages.SingleValue GetStatus(Messages.Blank request)
{
    var response = new Messages.SingleValue(this.State.ToString());
    return response;
}

I also added a description attribute to provide some help on action.

Note: yet another limitation in the console – no way to see this help yet – the server is actually returning the information

Now I can query the server and see the new method. When I call the method it will return the current state:

Nice and simple to add a new command to the API Of course, there is a lot more work to do on the client side but at least the workload has been reduced on the server development!

Now, the next question is what if we want a different result set than one of the standard messages? Again this is easy enough to implement. First add a new class for the return response:

public class ProjectStatus
{
    public string Status { get; set; }
    public DateTime LastBuildDate { get; set; }
}

And then use the class in the method:

[RemoteAction]
[Description("Retrieves the current status of the project.")]
public virtual Messages.ProjectStatus GetStatus(Messages.Blank request)
{
    var response = new Messages.ProjectStatus
                        {
                            Status = this.State.ToString(),
                            LastBuildDate = this.PersistedState.LastIntegration.FinishTime
                        };
    return response;
}

Now when we call the command we get the custom message returned instead:

So still nice and easy

For those who want to know the technical details:

• The messages are encoded as XAML, hence why the cyan text in the screen shots is XML. XAML handles the serialisation and deserialisation nicely without any extra work on our part.
• The querying and invoking is done via reflection. The ActionInvoker class in the code is responsible for this.
• The communication is all via WCF at the moment – and there are two possible channels (HTTP basic and NET.TCP). However both are extendable and over time these will be expanded.
• There is a simple class in the Common library called ServerConnection which implements the low-level client functionality. In time I hope to build some higher level APIs to make it really easy to use.

Here is an example configuration that demonstrates the problem:

<cruisecontrol xmlns="http://thoughtworks.org/ccnet/1/6" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <internalSecurity>
    <users>
      <passwordUser name="Bob">
        <display>Bob the Builder</display>
        <password>Bob1</password>
      </passwordUser>
    </users>
    <permissions>
      <rolePermission name="Builders">
        <forceBuild>Allow</forceBuild>
        <defaultRight>Deny</defaultRight>
        <users>
          <userName>
            <name>Bob</name>
          </userName>
        </users>
      </rolePermission>
    </permissions>
  </internalSecurity>
  <project name="SecurityTest">
    <security xsi:type="defaultProjectSecurity">
      <defaultRight>Deny</defaultRight>
      <permissions>
        <rolePermission name="Builders" ref="Builders" />
      </permissions>
    </security>
  </project>
</cruisecontrol>

Here there is an user called bob the Builder who has forceBuild permission – all other rights are denied. The problem is there is another right called viewProject which controls the visibility of projects in the dashboard. Since all other permissions are denied this permission also is denied, so the project does not appear in the dashboard!

Now if we are happy only allowing Bob the Builder to see project then the solution is easy – add a viewProject right and set it to Allow:

<rolePermission name="Builders">
  <forceBuild>Allow</forceBuild>
  <viewProject>Allow</viewProject>
  <defaultRight>Deny</defaultRight>
  <users>
    <userName>
      <name>Bob</name>
    </userName>
  </users>
</rolePermission>

Now when Bob the Builder is logged on he can see the project. But what if we want to allow everyone to see projects?

To allow this we need to add a guest account with the view permission as well:

<cruisecontrol xmlns="http://thoughtworks.org/ccnet/1/6" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <internalSecurity>
    <users>
      <passwordUser name="Bob">
        <display>Bob the Builder</display>
        <password>Bob1</password>
      </passwordUser>
      <simpleUser name="Guest" />
    </users>
    <permissions>
      <rolePermission name="Builders">
        <forceBuild>Allow</forceBuild>
        <viewProject>Allow</viewProject>
        <defaultRight>Deny</defaultRight>
        <users>
          <userName>
            <name>Bob</name>
          </userName>
        </users>
      </rolePermission>
    </permissions>
  </internalSecurity>
  <project name="SecurityTest">
    <tasks>
      <commentTask>
        <message>Ran successfully!</message>
      </commentTask>
    </tasks>
    <security xsi:type="defaultProjectSecurity">
      <defaultRight>Deny</defaultRight>
      <guest>Guest</guest>
      <permissions>
        <userPermission user="Guest">
          <defaultRight>Deny</defaultRight>
          <viewProject>Allow</viewProject>
        </userPermission>
        <rolePermission name="Builders" ref="Builders" />
      </permissions>
    </security>
  </project>
</cruisecontrol>

This example adds a new account called Guest, sets it as the guest account for the project and gives the account view permission. Now everyone can see the project in the dashboard.

Note: the Builders role still needs the viewProject right – otherwise when Bob logs in he will no longer be able to see the project!!

Download Metrics Package

If I get time I’ll try and add some jQuery smarts to it sometime, but it does show the basics. Feel free to improve it

The first issue is about interrupting tasks. While we are making some changes to tasks to allow for breaking the flow of a task these changes are more for tasks that are containers of other tasks. However another scenario was raised recently where we might want to break the internal flow of a task. By this I mean we want to stop a task from running during its normal process. This was raised in response to the parallel task – where there are two tasks that take a long time. If one task fails then there is no point in continuing the other task – instead it should be cancelled so the entire build is failed more quickly.

This however raises the issue of forced vs. co-operative cancellation. The preferred approach is co-operative cancellation. Both the task and its parent share some mechanism for cancellation. When the parent cancels the task it stops at the first convenient opportunity. This means it can ensure that everything is reset to a proper tidy state and there is no corruption of any data. The other approach – forced cancellation – is where the parent just kills the task. This means the cancellation is immediate – no need to wait around – but it also means that data can be corrupted. In other words we have absolutely no idea of what has happened!

Now we could expand the mechanism that is already in place for vNext to handle returning non-tasks (perhaps null) and then let the parent container handle the cancellation. But the current mechanism is specifically for running tasks so it would mean making the process more complex and it still wouldn’t allow for the task to perform any tidying up. An alternate is to add a new mechanism for cancelling the task and then getting the task itself to monitor this mechanism (similar to how cancellation works in .NET 4.0.)

Something to think about – I’m open for any suggestions on what people think will be the best way.

The second issue is around additional control for the meta-structures (queues, gates, etc.) Currently there are four meta-structures we are planning on adding:

• Queues: Allow only one (or more) active integrations at a time – take the next integration either LIFO or FIFO
• Round-robin: As a queue but using round-robin scheduling
• Gate: Waits for all active integrations to complete before allowing the next item to continue – this has a variation that will trigger all the children when the gate is hit
• Pipeline: Only allow the children to trigger within the order they are within the pipeline – one variation will wait for external triggers, the other will trigger each item in sequence

The additional control that was requested is for a “clean-up” integration to start when a queue has finished (i.e. there are no active integrations.) There are two ways I can think of doing this at the moment. One is we add a new control structure (e.g. GatedQueue) that remains active until the queue is clear. This could then be combined with a pipeline to trigger the “clean-up” integration. The second is we add start-up/clean-up properties to the queue that contains projects that must be run before/after the queue. At the moment I prefer the first approach as it is a bit tidier.

Does anyone else have any suggestions or preferences?

Simple and easy to use

Since working with CruiseControl.NET is only a small part of Patrick’s job he does not want to have to spend a lot of time learning the system. The interface should be intuitive to use, allow him fix any mistakes without them becoming serious problems and only require a minimum of work. He should also be able to setup “templates” for different projects that can be quickly applied.

Configure the entire system

Patrick should be able to configure everything in the system from a single console. He should not need to swap between different programs to set things up. This includes setting up, configuring and upgrading multiple different build servers. The “everything” that he needs to configure includes the different build projects, users and their security access, servers, etc.

View complete diagnostic information on the entire system

When Patrick is trouble shooting he should be able to see exactly what has happened if anything fails. When an integration fails he needs to know which project failed, which step in the integration and why it failed. Since he is only called in for fixing the problems that developers can’t (since he growls at them if they bug him) he needs to see more than just want the developers see. This information includes details about the servers (memory, CPU usage, available disk space), any other processes that were running (and might have caused interference), etc. Of course the information should be in a format that makes it easy for him to spot the cause of the problem quickly!

Automated monitoring

Given that Patrick prefers not to be bothered by developers he would much rather be notified of problems by the system directly rather than the developers. This means he can fix things before they come to him, which in turn makes the developers even more impressed with his abilities. As such he wants the system to notify him when certain parameters are exceeded. As he already monitors other systems and servers for Rainbow Interactions he wants the notifications for CruiseControl.NET to appear in his monitoring tool rather than needing a separate tool.

Comparing Patrick’s goals with Michelle’s we see that they both have a different focus. Patrick prefers an in-and-out approach – get in, do his tasks and get out – as quickly as possible. He also does not want to become an expert in the system so it needs to be as easy to use as possible. In contrast Michelle is more of a regular user. She is happy to spend some time to become familiar with the system if it will make things easier for her in the long run. They also both have some common features: it should work within their current tools, it should notify them when things go wrong and it should provide them access to everything they need in a single location (although for Michelle this will be two different locations.)

Hopefully this is starting to sound like an interesting challenge

Next step, time to put together some context scenarios to provide some generic detail on when and how they will approach their different tasks. Stay tuned…

For both of the personas we need an experience goal (what sort of experience they expect to have when using the system) and some end goals (what they are trying to achieve.) These are not technical goals instead they need to be aimed them at a higher level – what is the user trying to achieve. This gives us some more flexibility around how we meet the goals (i.e. the goal does not describe how it can be achieved.) In this post I will cover the goals for Michelle:

Flows with her working style

Michelle has two primary ways to access the build information: from within Visual Studio and over the web. She should be able to see the same information and make the same changes from both Visual Studio and the web. All build information and artefacts need to be accessible.

Displays all the information to diagnose problems with a build

When a build goes wrong Michelle needs to be able to quickly and easily see where things went wrong and why. To start with she needs to see that there is something wrong with a build. From here she should be able to drill down to find the exact place that the build failed and why. She would also like to see who added the code that failed the build and when.

Control a build project

As the lead developer for Here and Now Shipping’s system Michelle needs to have full control over how the build project works. The main action she needs to do is to start a new build integration, but she also needs to occasionally abort builds, start and stop the process, etc. A task she performs on a much lesser basis is modifying the configuration for a build.

Be notified when something goes wrong

If something goes wrong Michelle needs to be notified as soon as possible. She does not want to have to wait until she manually checks to find out something is wrong. The notification needs to have sufficient detail for her to know where to start looking for the problem.

As you can see these are all very high-level. They describe what the user (in this case Michelle) wants to do in the system without providing the implementation details. The next step in the process is to put together some scenarios that show when Michelle would use the system.

So stay tuned, more information is coming…

Note: these are not true personas as they are not based on research. Instead they are based on my understanding of a typical user; so they will not be as effective as true personas.

Patrick

37 year old male

Patrick is the systems administrator in a team of 12 at a small software development house called Rainbow Interactions. His job involves implementing and improving processes within the team. He directly reports to the CEO of Rainbow Interactions and often works directly with their customers. As such he has a very busy schedule and likes his time to be as streamlined as possible.

Outside of work Patrick is married with three children. As he spends a lot of time at work during the week he refuses to work during the week-ends so he can spend time with his children. He enjoys walking in the wilderness and so will often go away during the week-ends; which also means that work cannot contact him with any “emergencies”.

Patrick is involved in the initial setup of every project and then helps the other developers as required. He is also involved in cleaning up after each project has completed. His work varies from day to day depending on where in the project cycle the team is at. At the beginning of the project he makes time for setting up everything required for the project, including continuous integration (CI). During the project he will help the developers trouble shoot problems and even occasionally write code. Finally at the end of the project he archives all the work for the client. He might also go to the client’s site to help transfer the work over if necessary.

As part of their process for working on client projects they always set up a new CI process. Each process has a common core with the basic functionality (e.g. retrieve latest changes, compile the code, etc.) but they can all be customised depending on the clients’ needs. While Patrick sees this as an important part of his job he doesn’t have much time that he can spend on setting up CI processes. Instead he just comes in, copies the default template and then adds in the modifications as required. As different developers can be working on different projects, and sometimes the clients also have access, he likes to lock down the processes so only the relevant people can have access.

Patrick only helps with trouble shooting if there is a problem that the developers cannot solve themselves. His first question when they come to him for help is to ask what they have already done. If they have not tried anything (e.g. don’t know where to start) or they have missed something then he will advise them where to look. If it is a problem outside of their ability or capacities then he will get involved. These are normally problems on the CI servers (maybe bad configuration, maybe server issues, etc.) The first place he always looks is the server logs and from here he will trace the problem. As such he likes to see as much information as possible (which might include elevating the logging level to get additional information.)

To archive a project all he does is select the project and the destination and then starts the process. This copies everything to the destination and removes the process from the server. Later on he can restore the process or he can install it on another CI server if needed.

Michelle

26 year old female

Michelle is a developer at Rainbow Interactions. She has been working on a project for their most important client – Here and How Shipping – for the past five years and has recently become the lead developer for their system. Since Here and Now Shipping has been with Rainbow Interactions for several years now they have a well-defined CI process. However from time to time things change and the process needs some improvements (for example they learnt about code coverage recently and wanted that included.)

Michelle’s day-to-day job is very simple – she takes the list of requirements she gets given from the business analyst at Here and Now Shipping and works with her time to implement them as quickly as possible. Most days this involves writing code, sometimes testing, sometimes bug fixing. No matter what she is doing though she is always running short of time and will often stay late at work, with the occasional week-end day as well.

She just expects the CI process to work – when she (or any of the other developers) checks in code for Here and Now Shipping’s system it should automatically build and then notify her of any problems. If the build goes fine she doesn’t care (although sometimes she might look at it to just touch base” but it breaks she wants to know as soon as possible. When the build does break she wants as much information on why it has broken as possible so she can either fix things or chase down the person who broke it. The last thing she wants to do is bother Patrick as he is often grumpy if developers bother him without a “good” reason.

Her primary development environment is Visual Studio, but she also accesses build information remotely via the internet. She likes to see all the information possible on the build (number of changes, which unit tests were run and how long it took, the amount of coverage, any coding rule exceptions, etc.) She also needs to start and stop the process, trigger builds and modify the configuration if necessary. Basically she needs to do everything with the CI process for Here and Now Shipping. But conversely she doesn’t care about any of the other clients Rainbow Interactions works with.

Of these two personas I’d put Michelle as the primary persona as she is the main user of the system. However Patrick would have a different set of interaction goals so he would probably be another primary, although not as important as Michelle.

In my next post I’ll look at some interaction goals these two personas have and maybe some scenarios. With these in place we can start to look at how we can design the interaction process for our users.

Stay tuned…

There are some areas in CruiseControl.NET (CC.Net) that tend to turn people off. I often see people mentioning they use alternate products because of “blah”. Now these are not necessarily UI problems but a good UI can go a long way to helping people enjoy a product despite other problems (just look at Apple products as a prime example!) So any love and attention we put into the UI will pay off a 100-fold.

The following diagram shows the current components and their interactions in CC.NET:

The server is a black box to the user. Yes there is a console version that will display the log but the user has no way of interacting within the server beyond just seeing what is happening.

Instead the user needs to use an external component to interact with the server. For example the web dashboard is a web-based application that allows a user to see what is happening on the server and to make some limited changes (e.g. force/abort a build, etc.) The grey lines show the direct communications with the server – which is via .NET Remoting.

CCTray is the second type of interface application. It is a Windows-based client that also allows viewing the server status plus some changes. Unlike the dashboard it can use HTTP-based communications but this is an indirect path – it needs to go via a dashboard to actually connect to a server. And CCTray tends to be a lot more limited in the functionality it offers.

There is a third path of interaction – via the ccnet.config. This is the path that the validator takes. It does not directly interact with the server at all, instead it reads the same configuration. And of course people directly modify ccnet.config as the ultimate way of controlling the server.

There are also a few more applications that can be used to interact with the server: cradiator (BVC), FastForward.NET, CCCmd, etc. These all tend to fit in the same space as CCTray in that they use either HTTP or .NET remoting to connect to the server (although some only use HTTP.)

Each of the various UI components has their own limitations. The dashboard is reasonably easy to configure and extend (if you know how it fits together) but it is global in scope. There is no way to configure it to have custom views for each user. In contrast CCTray is easy to personalise but the options for extending it are very limited (especially as there is no plug-in framework for it.) Plus going via the dashboard for HTTP means it can be slow. Finally (and this is one of the main complaints I see) there is no way to easily modify the configuration. The config file is XML – if you want to configure CC.Net you have to use XML. End of story!

So going forward I’m going to look at how we can apply some interaction design principals to CC.Net to improve the user experience. If there is anything that annoys you about the current UI or any suggestions you have let me know and I will see if (and how) they can be resolved or added.

When an integration is started a new TaskExecutionContext is created using StartNew() on ITaskExecutionFactory. This creates a new folder for the integration, starts an XmlWriter for the build log and adds the basic project details to the log.

The next call in the sequence is to StartChild() on TaskExecutionContext. This creates a new child instance of the context that is specific to a task. This will output the task details (e.g. name and type) to the log. An instance is created for each task by the Project – this then gets passed into the task.

Once a task has finished the Complete() method is called. This writes the final details of the task to the log and tidies up. The same complete method is called whether the context is for a task or for an integration. The only difference is the integration level call will close the log at the same time.

There are currently four methods that can be called from within a task:

• AddEntryToBuildLog(): adds a comment directly to the log
• ImportFile(): copies/moves a file to within the folder for the integration and adds an index entry to the log
• StartOutputStream(): creates a new file within the integration folder and adds an index entry to the log
• AddModifications(): adds any modifications from a source control block to the context

There are also a number of properties that are available. These are:

• CurrentStatus: the current status of a task. This can be set by a task but if not set then calling Complete() will set it to Success.
• IsCompleted: a read-only property flag indicting whether a context has been completed. Calling any methods on a completed context will throw an exception.
• Request: the original integration request details (read-only).
• Project: the project the integration is for (read-only).
• Parent: the parent context. If the context is the project level context this will be null. Otherwise this will point to the context that created the current context (i.e. the context that StartChild() was called on)(read-only).
• ModificationSets: the current modifications within the integration (read-only).

One point to note about ModificationSets is there is only a single instance for an integration. This is why AddModifications() is required – it handles any locking required so they are no thread conflicts when adding or retrieving modifications.

So despite the changes to how the context works, from a task authoring perspective it should be even simpler to use.

One of the major differences between TaskExecutionContext and IIntegrationResult is there will be a single TaskExecutionContext per task. In the current versions there is only a single IIntegrationResult for an integration – it is up to the tasks themselves to correctly manage the properties on the instance. As you can imagine this can potentially led to issues where the properties are treated differently between the different tasks. So in vNext each task will have its own TaskExecutionContext instance. The controller (typically the project) is responsible for starting these instances and completing them once the task has completed.

This important difference also affects how the contexts are started. The initial context (at the project level) is started via a factory class (TaskExecutionFactory). This will initialise all the required settings for the instance. Subsequence contexts are then started via the StartNew() method on the current context. This generates a new context that is associated with a parent – which means the log entries will be written to the same log.

This is important because of a second major change – log entries are now written directly to a file instead of being stored in memory. This is to get around the nasty OutOfMemory problem that we are having with the current versions. As such this significantly changes the way that the build log is being written. In the current versions the log is written by (and only by) an xmlLogger task. If this task is omitted or fails to execute then there will be no log. As you can imagine this can be slightly disconcerting for new users (I remember it was for me!) Now because the log is emitted straight to disk it will always be written – which makes the xmlLogger task obsolete.

With this in mind, the format of the file has changed significantly. I am still working on the format and re-adding all of the goodness that’s in the current versions, but here is an example of what the file is shaping up as:

<?xml version="1.0" encoding="utf-8"?>
<project name="TestProject">
  <start>2011-01-09T15:28:43</start>
  <task type="Comment">
    <start>2011-01-09T15:28:44</start>
    <entry time="2011-01-09T15:28:44">A Test Comment</entry>
    <finish>2011-01-09T15:28:44</finish>
    <status>Success</status>
  </task>
  <task type="MergeFiles" Name="NUnitFiles">
    <start>2011-01-09T15:28:44</start>
    <file time="2011-01-09T15:28:44">testproject.xml</file>
    <file time="2011-01-09T15:28:52">otherproject.xml</file>
    <finish>2011-01-09T15:28:53</finish>
    <status>Success</status>
  </task>
  <finish>2011-01-09T15:28:53</finish>
  <status>Success</status>
</project>

This now contains a wealth of information on the actual tasks that were executed. Details for each task appears within the tasks themselves (this will include references to imported files, embedded data, etc.) We can quickly and easily see:

• The name (if any) and type of a task
• The start and finish times
• The final status
• What data was added during the task

I’ll also flesh out the project data to include most of the data that is in the current build log.

Finally the other significant change I have had is external data is no longer directly embedded within the log. This was something that worked when we played around with fixing the out-of-memory issue (unfortunately it never got implemented) so we will include it in vNext. Instead of directly embedding the data within the file we add a reference to the file instead.

The advantages of this approach is our build logs will be a lot smaller as it now contains the build summary instead of all the details. It also gets rid of the problems around XML encoding –currently if the source file was invalid XML (or not XML at all) it would be put in a CDATA section. And since the files are linked to a task it is easy to see which task generated the file – especially if there are duplicates of the task type within the project.

The main disadvantage to this is the client will need to first parse the build log to get the names of the relevant files and then get the relevant files from the server. This could be troublesome when generating the build summary but given that the build summary is static we could look at some way of either caching this on the clients or pre-generating the summary on the server.

So this is a look at some of the differences between the current and new build logs, in my next post I’ll look at some of the nuts and bolts of how the log is generated.

To achieve this there are two parts that need to be done.

First to pass arguments and return values a sub-class of BaseMessage is needed. There will be a few default sub-classes that will hopefully provide most of the required functionality but it will also be possible to add new sub-classes as needed. These classes will allow passing around the information between the client and the server plus a few extra bits and pieces of plumbing (e.g. security information.)

The second part is to implement the method and add a RemoteActionAttribute to its definition. This method needs to take in a sub-class of BaseMessage as the argument and return a sub-class (it can be the same but it can also be different as required.)

And that’s it – hopefully nice and simple

There is one additional step – add an optional DescriptionAttribute. This will be used by the framework as a human-readable description that gets passed around.

For example for a ForceBuild action we could have the following definition:

[RemoteAction]
[Description("Trigger a build remotely.")]
public BuildMessage ForceBuild(ProjectMessage request)
{
    // Implementation omitted
}

Actions can be exposed on any item within the configuration (e.g. server, queue, project, tasks, triggers, etc.) The invoker will use the universal name to find the correct object and then either query or invoke the actions. The List method will check for all methods that have the RemoteActionAttribute and return the name and description (if present.) The Query method will add the input and output message types. And finally the Invoke method actually calls the method passing in the arguments and returning the result. Under the hood this uses Reflection to do all the dirty work – but that is the subject of another post.

So up to this posts I’ve covered the basics of the communications. The next question (and my next post) is how does this get exposed to the client? I’m looking at using WCF for the default implementation but it will be generic enough to cover additional protocols as required. So stay tuned and in my next post I’ll cover the implementation of a communications channel on the client.

To simplify things in vNext I’d like to only expose three methods – these methods would then expose any additional functionality we want without having to modify the actual communications infrastructure. We already have a version of this with the messaging infrastructure that was added in 1.5 but I’d like to take this a step further and also handle some scenarios that the messaging does not handle.

So in a nutshell the three methods I am proposing are:

• List
• Query
• Invoke

These three methods provides the basics of a communications framework where any item in a server can expose functionality. I’ll come back to this in a minute but first let’s go over these three methods.

The List method will list the possible actions that are possible for an item. The method has only one parameter – a universal name of an item. The communications framework will look up the item based on its name, retrieve its type and then get the possible actions that are available for the current user. For example a project type item might allow GetStatus, ForceBuild, AbortBuild, etc. actions while a queue type item might have GetPendingRequests, CancelRequest, etc. This method will return a list of actions – each action will have a name and a description.

The Query method will provide the details on an action. Like the List method it requires a universal name and it also requires the name of an action. If the action is available and allowed it will return the description and the input and output parameters. If the action is either unavailable or not allowed for the current user it will return nothing.

Finally the Invoke method is what is used to call an action. It requires the universal name of an item, the name of an action and its required input parameters. It will invoke the action and then return whatever output parameters are generated. If the action is not available or not allowed then an error will be generated and returned.

These three methods provide a way for external clients to not only run actions on the server but to also determine whether an action is available. For example force build in 1.5 or later versions can be locked down with security but the dashboard has no way of knowing this. The only way a user can find out if they are allowed to force build a project is by clicking on ForceBuild which will generate an error in return – not very nice!

With the List and Query methods the client can check to see if a user has access to the action, maybe as part of the UI generation or as a sanity check before invoking the action. The description can also be used to return help information to the user as required. It will also be possible in future to generate dynamic interfaces where the list of available actions is dynamically generated based on what the server returns, the user selects the action and is prompted for the input arguments and then performs the invoke.

The other part to having these three methods is any item on the server can expose methods – not like the current state where it is primarily projects that have methods with a few additional methods floating around. Using the universal names we can list the possible actions on any item and invoke them. For example there are some items within a project that store state: e.g. the Subversion source control block can store the last revision number. We could add an action to this block to retrieve this state and return it to the user and we could add another action to allow the user to update the state. By having it on the level of the item only the item needs to know about the actions (we don’t need to keep bloating Project with actions for child items.)

Now I have covered two pieces of the communications framework I’m wanting to implement – universal names and the public methods. The third part is how to tie these together – how can we expose actions on various items and have them picked up by the communication structure. As you can imagine this is the heart of the new structure so I will cover it in my next couple of posts.

Stay tuned…

In addition CruiseControl.NET only supports one protocol for talking to the server – .NET Remoting. There is a web-based protocol but this requires an installation of the dashboard which acts as an intermediate. And as the bug reports show this has caused some other problems. There is an WCF extension that can be used but because extensions were tacked on later not many people know about them – especially as they are configured via the app.config instead of ccnet.config!

For vNext I’d like to take this another step further and abstract away more of the communications infrastructure. This will allow for additional communications protocols natively on the server. Plus we can handle from the start some of the issues that have been raised in the current web-protocol.

These changes will involve a number of different components which I’m planning on covering in the next few posts. In this post I’m going to start on a simple that allows a lot of functionality: universal naming.

Currently we have a bit of a mixed bag with the current versions – we can access some items remotely; others are more hidden. For example it is very easy to access and work with projects – the ever present project name servers as a very useful identifier. It is possible to get some information about queues and tasks but normally this is accessed via a higher level item (e.g. the server snapshot or via a project.) If I wanted to find out how an MSBuild task on a project is configured I would have to get the whole project’s configuration and then extract just the task I wanted – not very obvious.

What I would like to propose is a universal naming standard that allows almost any item within the server to be addressed. This will make it different from how things currently work but it will allow much more control over what is accessed.

The base of this naming is to use non-standard URNs. These will be non-standard in they will not use an officially defined namespace id but otherwise will work just like a standard URN. The namespace will be ccnet – so all identifiers will begin “urn:ccnet:”. As per RFC 2141 “urn” is case-insensitive and “ccnet” will be also.

The generic form of a ccnet urn is:

urn:ccnet:server:path-to-item:item

server and item will always be required. server is either the unique name of a server or its address. For example for the server ccnetlive located at http://ccnetlive.thoughtworks.com server would be either http://ccnetlive.thoughtworks.com or ccnetlive. If ccnetlive is used then the mapping to the URL will need to be stored somewhere else (more on this in a later post.)

item is the unique name of an item. Which item depends on the path that has been entered. To help with resolving an item identifier I think we need a few rules around names within the configuration:

1. All item names within a level of configuration must be unique
2. All projects must have unique names that cannot be the same as another root-level item
3. All project items must have either a single instance or a unique name

Rule #1 basically means that at each level the name will identify a unique item. For example a level might be the root level of the server configuration or it might be the child items within a queue, tasks within a project, etc.

Rule #2 means we have an easy way to identify requests for projects (more on this in a minute.) Ensuring that project names do not exist at the root (unless the project is at the root) removes any confusion between referring to a project or a root level item.

Finally rule #3 means we can refer to individual items within a project. The single instance part means if an item is not named then there cannot be any other instances of that item type. For example if there is an unnamed MSBuild task then there cannot be any other MSBuild tasks within that project or project item.

Returning back to the URNs if we had a project called ccnet-trunk on ccnetlive the basic URN would be:

urn:ccnet:ccnetlive:ccnet-trunk

The final part of the URN is the most interesting part – a path to an item. Coming back to our earlier example of an MSBuild task within a project we could reference this task by using the project name as the path:

urn:ccnet:ccnetlive:ccnet-trunk:msbuild

In this example the path is “ccnet-trunk”. The power of this comes when we start looking beyond just the simple project/task structures. For example if we wanted to look at a queue within a queue we could have something like:

urn:ccnet:server:outerqueue:innerqueue

Here the item is innerqueue and the path to it is outerqueue. This means only projects need unique names – other structural items can have duplicate names as long as they are not duplicated under an item – we could have another item called innerqueue that exists under an item called roundrobin for example.

We could also use this syntax for looking at items that belong to other items. If we had an MSBuild task within a Sequence we could use:

urn:ccnet:ccnetlive:ccnet-trunk:sequence:msbuild

This syntax would allow us to access any part of a server’s configuration easily and consistently.

There is one additional area that needs addressing and that is for individual builds. For example we might want to retrieve the status of a build of a project. Now this by itself has a couple of issues: first the build is more of a filter rather than an actual part of the address and second there is currently no easy way to address every build. Projects do have labellers that generate unique labels for a build, but these are only guaranteed unique for successful builds (i.e. the labels can be re-used from failed builds.) To get around this the current version uses a combination of a date/time stamp, the build status (passed/failed) and the label for uniquely identifying a build!

So to address the first issue I think we should use a different symbol for identifying the build, and I am proposing “->”, this would be appended to the end of the URN. For the second issue I think we should use either the date/time stamp or the build label for the name of a build.

For example if there was a build for the above project that started at 13:02:14 on 12-Dec-2010 we could use the following URN for the build:

urn:ccnet:ccnetlive:ccnet-trunk->20101212130214

The date/time stamp should only go to the nearest second (otherwise it gets too long) and we should enforce that a project cannot start building more than once a second.

Alternately if the build was successful and generated a final build label of 2.0.1 then we could use:

urn:ccnet:ccnetlive:ccnet-trunk->2.0.1

This would also be usable with tasks and other project items:

urn:ccnet:ccnetlive:ccnet-trunk:msbuild->2.0.1

Using the alternate symbol (->) instead makes it easier for address resolution. If we used colons for both the build and the item name when we would need to do a look-up to decide whether it is a build name or otherwise.

So this post has covered a naming standard we can use for addressing different parts of a server. As a preview there are two additional parts I’m planning on covering for communications:

• The Three External Methods: looking at how we can condense our external surface to use three methods (List, Query and Invoke)
• Invokable Methods: looking at how we can expose any method on any item so it can be invoked by an external client

So if you don’t know why I want to add universal naming to vNext stay tuned and (hopefully) things will become clearer.

And as always, I’m happy for feedback, so let me know what you think.

There’s actually two parts to this question:

1. Are the logs being generated correctly?
2. Are we putting the right stuff in the logs?

Question #1 is really outside our scope – considering the number of different ways the logging can be configured we’d have to duplicate all the tests in NLog itself. So rather than doing this we’re just going to assume that the tests in NLog will pick up any errors with log generation.

That leaves the second question – how can we know we are putting the right stuff into the logs? The short answer is we can’t – but then there is no way that we can guarantee that any code we write is 100% correct!

Instead what I’m interested in doing is checking that we are logging some expected stuff. In other words how can we intercept the logging to ensure that we are logging the expected messages?

To answer this question we need a little bit of background. Logging in NLog can be configured either via a configuration file or via code. Now the recommended way is the config file so it can be changed without forcing a re-compile. But for us the way that we will use is via code.

NLog also has the concept of Targets: where the output is written to. There is a wide variety of different targets for different scenarios (console, file, e-mail, database, etc.) Of interest in unit testing is the Memory target. This will write the log messages to an array in memory.

So all we need is some way to wire up the logging to a memory target and then check the messages that are being written. It would also be nice if the unit tests reset the logging after they’ve finished just in case there has been some configuration in the unit test config file.

To make this easy I have added a helper class to our unit tests that simplifies this whole process. This is a disposable class with a static method for initialising the whole process. To use this class looks something like:

using (var intercept = LogHelper.InterceptLogging(typeof(TypeBeingTested)))
{
    // Tests go here
    var expected = new[]
                        {
                            // Expected messages go here
                            "LogLevel|TypeBeingTested|Message"
                        };
    CollectionAssert.AreEqual(expected, intercept.Messages);
}

Since the class implements IDisposable it can be used in a using statement. This will ensure that the configuration is reset after the tests have finished.

The static method takes in an array of types. These types are the loggers that should be intercepted – after all there can be other messages occurring that we don’t want to intercept.

To actually check the results we need to compare the Messages property on the class with the expected messages. This is just a facade property that exposes the messages from the underlying memory target. As in the example above the messages consist of three parts: loglevel (e.g. debug, info, error, etc.), the loggger (this will be one of the types in the array) and the actual message.

Nice and simple, without adding too much overhead to the testing process.

Let me know if anyone is interested in the actual code behind LogHelper and I will post it. Otherwise it is in the Subversion repository for CruiseControl.NET.

The current triggers can be roughly divided into two groups – those that “fire” integrations and those that modify integrations. When I say “fire” I roughly mean that the trigger is reasonable for initiating an integration request. The triggers in this category are:

• CronTrigger
• IntervalTrigger
• ProjectTrigger *
• ScheduleTrigger
• UrlTrigger *

An asterisk next to the trigger means it contains an inner trigger. The trigger itself first checks the inner trigger; if this trigger fires then it does any additional checks which may result in it  being fired.

The second category are those triggers that modify the integration request from another trigger. These all contain other triggers:

• FilterTrigger *
• MultipleTrigger
• ParameterTrigger *
• RollUpTrigger *

The one unusual trigger is MultipleTrigger. This actually contains multiple other triggers (it is the only one that can contain multiple triggers.) Technically it does not really modify an integration but it does contain some logic around whether the integration requires just one or all of the triggers to fire.

The two methods that need to be implemented are Fire() and IntegrationCompleted(). Fire() is called when the trigger needs to be checked. If the trigger has been tripped then this should return an IntegrationRequest; otherwise it should return null. IntegrationCompleted() is called after an integration has been completed – this allows the trigger to reset itself. There is one thing to note about this arrangement – although IntegrationCompleted() allows the trigger to reset this is only for resetting internal state. A new IntegrationRequest is actually generated every time Fire() is called (assuming the trigger has been tripped.)

The final part of the current triggers is the NextBuild property. This returns a DateTime specifying when the next build will be either checked or triggered. It is not documented anywhere (at least that I can see) but if the trigger does not have a next build date/time it should return DateTime.MaxValue.

So how will triggers change in vNext?

First they will have the standard Validate() method. This will be called when the Project itself is being validated (i.e. after it is loaded but before it is started.)

Second they will have Initialise() and CleanUp() methods. The Initialise() method will be called when the project starts – this will allow the trigger to reload any previous state or initialise itself to an initial state. CleanUp() is called when the Project has stopped – allowing the trigger to perform any clean up work.

The Fire() method has been renamed to Check() and it has become a sealed method. Internally this calls an abstract method called OnCheck() which should perform the actual check. The reason for this change is the trigger will store the request once it has been tripped – subsequent calls to Check() will return the same request until the trigger has been reset.

IntegrationCompleted() has been renamed to Reset(). This will clear the previous request and then call OnReset() to allow any custom state resetting.

These changes mean that triggers now perform more like a ManualResetEvent – once tripped the trigger will remain tripped until reset. This should resolve some of the issues that have been identified with how MultipleTrigger works with some of the other triggers (e.g. ProjectTrigger and UrlTrigger.)

The final change is NextBuild has been renamed NextTime and changed to a nullable DateTime. If there is no next time then this should return null.

Otherwise triggers will continue to be used in the same way – they will be called on a regular basis (~2Hz.) When a trigger has been tripped it will start an integration – it is up to the tasks to define what happens after a trigger has been tripped.

Internally they will be roughly the same. Like tasks they will inherit from a new base class – SourceControlBlock. This class has one standard property (Name) and the following methods that can be overridden:

• Validate()
• Initialise()
• GetModifications()
• Label()
• GetSource()
• CleanUp()

Validate(), Initialise() and CleanUp() are the same as the task-based versions – they will act the same way and be called at the same time. The main difference is these are directly overridable (in the task the OnMethod() is overridden instead.)

The other three methods need to be implemented by a source control block as they provide the actual functionality. Each of these methods takes in a MethodParameters instance (e.g. GetSourceParameters, etc.) This allows us to easily add new parameters later on without breaking other implementations (something we currently have a problem with.)

To use the source control blocks there is a base class called SourceControlTask. This contains the Use property and a helper method to get the source control block (and in time it may have other helper methods.) Using this makes implementing the source control tasks very easy. For example the current implementation of GetSource is:

protected override IEnumerable<Task> OnRun(TaskExecutionContext context)
{
	var block = this.GetSourceControlBlock();
	var parameters = new GetSourceParameters();
	block.GetSource(parameters);
	return null;
}

Note: this needs to be expanded to correctly pass the parameters – I haven’t gotten that far yet.

Other than these changes source control blocks will continue to work in the same way as they do currently. They can have their own parameters that are configurable and their different implementations depending on the actual source control provider.

This is in contrast to how validation currently works within 1.5/1.6. These versions have a helper interface that allows both warnings and errors to be handled. A warning will merely be displayed in the log – it doesn’t stop the server – while an error will prevent the server from running. Additionally the validator handles both errors and warnings and displays them for the relevant items.

The good news is I plan to continue this in vNext, especially as the validator is a very useful tool. To handle this I’ve added a new interface to the code called IValidationLog. This works in a similar way to the current versions. I’ve also changed the signature of the Validate() method to use an instance of this interface:

void Validate(IValidationLog validationLog)

Like the interface in the current versions that this interface is based on this interface has a method for adding errors and another for adding warnings. As an example of how this works here is the implementation of Validate() in ServerItem:

public virtual void Validate(IValidationLog validationLog)
{
    // Everything must have a name
    if (string.IsNullOrEmpty(this.Name))
    {
        validationLog.AddError("The {0} has no name specified.", this.ItemType);
    }
}

As name is a required property for any server items this will add an error.

Hopefully this clears up how validation will work a bit.

I’m just wondering what the benefit of returning an array(enumerable) is when calling OnRun. It does not seem/sound right. When a function is named OnRun, I would expect no return value, making it void, or maybe returning a result. So returning ‘other’ tasks feels unnatural.

Since it doesn’t sound right it means that I haven’t explained the why clearly enough

Basically the why can be summed up in one word: “Control”!

But before I go more into the why, first let’s cover a bit of background. In CruiseControl.NET there are two general categories of tasks – those that execute and return and those that execute other tasks (containers.) The majority of tasks are execute and return type tasks (e.g. MSBuild, NAnt, etc.) They do their work and that’s it – nice and simple. However from 1.5 onwards there is a new category: those that execute other tasks. These include the parallel, sequential and conditional tasks (plus a few others.) The change I am proposing is primarily for this second category.

In the current versions of CruiseControl.NET (1.5 and 1.6) the parent of the task is responsible for executing any children. If the parent is the project then the project executes the tasks; if the parent is one of the container tasks then it is responsible for executing the task. For these container tasks the project (or any other parents) has no idea of what is happening. As far as it is concerned the task is just a standard execute and return task.

Now this can cause a problem: what if the project wants to do something else while the tasks of a container are running? At the moment the only option it has is to wait until all the tasks within the container have finished (actually there is another option – abort the task – but it is not very nice!) One scenario that I want to implement where this is a problem is pausing a build (more on this below), another scenario is looking at distributing tasks across multiple servers, another is allowing different actions for a task if it has failed, etc.

Ok, now back to the why. For the execute and return tasks this change has absolutely no impact (well, maybe one, it forces them to add a return null to the end of the method.) So if all we had was this category I wouldn’t suggest this change. But for the second category this change makes a huge difference! Having the container task return the child tasks to the project to execute them means the project knows exactly what is happening. If the project needs to be paused it can just wait until whichever task is currently running has finished and then not start the next one until the project is resumed. If the project is distributing tasks to other servers again it knows which task is currently running and can distribute the child tasks to different servers if desired. In addition the code for executing a task is centralised – no need to duplicate the code across different tasks (like we currently have.) So adding the failure options (and the conditions) is very easy – it only needs to be added to one place.

So hopefully this helps explain why I want to make the change. As stated above it makes no difference to the execute and return tasks – it is primarily for benefit of the container tasks.

The second question Ruben had was:

Also what is the benefit of ‘pausing’ a task?

He then mentioned some pain areas we could have with people manually pausing a project (we won’t really pause tasks – the execute and return tasks will not be effected.) And on the whole I tend to agree with his comments – if a project is paused then people are likely to forget to resume it and this will cause problems.

However I’ll admit – I wasn’t thinking of allowing people to manually pause projects (although some people might actually want this.) My idea for pausing projects comes from two areas:

1. Somebody wants another more important project to “interrupt” a current build and then the current build resumes
2. One or more tasks in a project are sent to another server to execute and the build administrator does not want this to prevent other builds

For the first scenario a user would send a priority force build request to the server (assuming they have sufficient permissions). The project would receive this request and ask to start integrating. If the request is not immediately granted then this means another project is building (and they are both in the same queue or similar.) The forced project would then send a pause to the building project and re-ask to start integrating. When the current building project pauses the forced project would integrate and when finished it would tell the original project to resume. So in this scenario the pause and resume would both be automatic without the user having to do anything (or forget anything.)

The second scenario is a bit more interesting. Again there would be two or more projects in the same queue (or similar structure.) If the first project has remote tasks it would send off these tasks and pause itself. This would allow the second project to integrate at any time. When a remote task on the first project is returned it would send a pause request to the second project and wait. Again the second project would pause and the other project can continue. If the first project finishes then it would resume the second project. Or if another a remote task is encountered it would again pause allowing the second project to continue.

Now I should mention at the moment these are merely some ideas I have around pausing a project – at this point I have no idea on how to implement them (especially the communications between projects) – but at least if we know about them we can plan on implementing them.

Finally as I was writing this I thought of a third scenario where pausing might be useful – a task might fail because a resource (typically remote) is not available. Rather than failing the entire build the project might just pause for a specified period and then retry the task.

So hopefully this clears up some of my intentions a bit. The overall goal of these changes is to make the codebase simpler for future work so if this sounds too complicated we can drop it

And I’ll continue working my way through my e-mail backlog and answer any other questions, stay tuned…

void Run(IIntegrationResult result)

Or if implementing from TaskBase:

bool Execute(IIntegrationResult result)

Both of these use the IIntegrationResult for the current integration, which means we often had to do some checks around it before we updated the status otherwise the incorrect status could be set! In addition the actual implementation is a black box. As far as the caller is concerned once Run() is called (Execute() is a protected method that is called internally by Run()) the task is on its own. It does not know about any child tasks just that the task started and (hopefully) finished.

Instead I would like to change the signature to:

IEnumerable<Task> OnRun(TaskExecutionContext context)

There are two main differences here. One every task will have its own TaskExecutionContext and it will return an IEnumerable<Task>.

The first change is to reduce side effects between the different tasks. The final project state will be set externally – based on the context results from the different tasks. People can use the FailureOptions on a task to define how the final state is set. The context will also provide a number of other helper methods e.g. for writing build log entries, storing files, etc.

The second change is more significant – returning an enumerable. This allows the controlling logic (i.e. the project) to know exactly what is happening during a build. If a task has sub-tasks then the project will also know about them making it easier for the controller to stay in control! Using an IEnumerable means we can use the C# syntax to make it easy to develop tasks – specifically the yield keyword.

If a task does not have any sub-tasks it merely does its work and returns null. For example in the Comment task:

protected override IEnumerable<Task> OnRun(TaskExecutionContext context)
{
	logger.Info("Adding comment to the build log");
	context.AddEntryToBuildLog(this.Text);
	return null;
}

The comment task adds an entry to the build log and then signals that it is complete by returning null.

In contrast this is the logic for the Sequence task (which runs a sequence of tasks in order):

protected override IEnumerable<Task> OnRun(TaskExecutionContext context)
{
	logger.Info("Running sequence '{0}' with {1} task(s)", this.NameOrType, this.Tasks.Count);
	var count = 0;
	foreach (var task in this.Tasks)
	{
		logger.Info("Running child task {0} in '{1}'", count++, this.NameOrType);
		yield return task;
	}
}

The implementation is primarily logic around logging and returning the tasks. yield return is used to return each task in order to the controller. The controller will then decide whether or not to run the task (e.g. it will call CanRun(), etc.) When there are no more tasks the method merely returns which signals the end of the enumeration.

The good news is the C# compiler handles everything necessary to implement the enumeration under the hood, making it nice and easy for us

On a side note this is the approach used in Microsoft Robotics Studio so it is nothing new.

One of the main reasons for this change is in future I want to add the ability to “pause” an integration. To try and pause an integration in the current version is a major hassle (I have tried without success to implement it.) The major stopping point is there is no easy way for a project to halt execution of tasks and their sub-tasks as the control logic is embedded within the tasks themselves. This change will move the control logic back to the project allowing the project to decide when (and maybe even where) tasks are run.

As always let me know what you think – especially for an important change like this one.

Here is the state flow for a task:

When a task is first created (normally from XAML) its state is set to loaded. All this means is the task has been created but nothing else has happened.

Shortly after the task has been loaded its parent will validate it by calling the Validate(). Validation happens after all the configuration has been loaded so every item knows its place in the object graph and can reference other parts of the configuration as needed. When validation has finished the state is set to Validated. At this point the task can safely be used. If there is a catastrophic validation failure then the project will not be startable.

When an integration (build) starts every task is initialised by calling the Initialise() method. This call happens irrespective of any conditions – at this point the project does not know what will or will no run so it initialises everything. At this point tasks can initialise any resources they need and set their initial properties for an integration. After initialisation has finished the state is set to Pending.

After everything has been initialised the project can actually run the integration. For an integration it merely iterates through all the tasks in its list. The first action performed on a task is calling CanRun(). This sets the state to CheckingConditions and performs any logic to detect whether the task can run.

If this check is successful Run() is called. At the start of this method the state is set to Executing. Then the actual implementation of the task is called and finally the task is set to Completed. If the check fails then Skip() is called and the state is set to Skipped. This way it is easy to see whether a task was executed or not.

Once the task has completed or skipped then the next task is checked and so on.

Finally at the end of an integration CleanUp() is called. This will check the state of the task to ensure that it is in a valid end state. If the state is Pending then it will set the task to Skipped (the logic assumes that someone else forgot to set the state correctly.) If the state is CheckingConditions or Executing then it will set the state to Terminated (it assumes that an exception had occurred and the step did not complete.)

And that’s it for the state cycle for a task. The next time an integration starts it will it go back and initialise all the tasks again.

As I mentioned above there are a number of extension points in the state process. These are:

• OnValidate()
• OnInitialise()
• OnCanRun()
• OnRun()
• OnSkip()
• OnCleanUp()

All of these are optional except OnRun(). This must be implemented to provide the implementation logic for the task, but I will write on this next time.

Hopefully this post gives an overview of how the new task will work and shows how it both communicates it state and allows extensions.

Basically there are three classes that we need to provide implementations of:

• XamlSchemaContext
• XamlType
• XamlTypeInvoker

The first class, XamlSchemaContext, provides the entry point to implementing an IoC container. This provides a context for any XAML operations including retrieving types to be instantiated. The full implementation of the class is as follows:

public class NinjectXamlSchemaContext
    : XamlSchemaContext
{
    private readonly IKernel kernel;

	public NinjectXamlSchemaContext(IKernel kernel)
	{
		this.kernel = kernel;
	}     

	protected override XamlType GetXamlType(string xamlNamespace,
		string name, params XamlType[] typeArguments)
	{
		var retval = base.GetXamlType(xamlNamespace, name, typeArguments);
		return retval == null ?
			null :
			new NinjectXamlType(this.kernel, retval.UnderlyingType, this);
	}
}

The important part is the GetXamlType() override. This allows us to return a custom XAML type that uses Ninject instead of activation via reflection. I also added a constructor that takes in the Ninject kernel to allow storing the kernel externally but this could be moved internally if there is no need to pass the kernel around.

The second type to implement is XamlType. This provides a set of methods for hooking into the initialisation process. This implementation is:

public class NinjectXamlType
	: XamlType
{
	private readonly NinjectXamlTypeInvoker invoker;     

	public NinjectXamlType(IKernel kernel, Type type, XamlSchemaContext context)
		: base(type, context)
	{
		this.invoker = new NinjectXamlTypeInvoker(kernel, type, this);
	}     

	protected override XamlTypeInvoker LookupInvoker()
	{
		return this.invoker;
	}     

	protected override bool LookupIsConstructible()
	{
		return true;
	}
}

The main override is LookupInvoker(). This returns our custom invoker for new types. By itself XamlType doesn’t seem to do much it mainly appears to act as a coordinator for various look-ups, hence overriding the invoker method.

The final class is XamlTypeInvoker, which is the workhorse for our example. This type actually generates the various object instances. Thus our implementation is:

public class NinjectXamlTypeInvoker
	: XamlTypeInvoker
{
	private static readonly Logger logger = LogManager.GetCurrentClassLogger();
	private readonly IKernel kernel;
	private readonly Type typeToResolve;     

	public NinjectXamlTypeInvoker(IKernel kernel, Type typeToResolve, XamlType xamlType)
		: base(xamlType)
	{
		this.kernel = kernel;
		this.typeToResolve = typeToResolve;
	}     

	public override object CreateInstance(object[] arguments)
	{
		logger.Trace("Generating an instance of '{0}'", this.typeToResolve.FullName);
		return this.kernel.Get(this.typeToResolve);
	}
}

Again there is one method we are overriding – CreateInstance(). In our case all we are doing is going to the kernel which creates the instance for us and wires everything up. I’ve also added so logging so we can hook in later to see what is actually happening.

The last piece of the puzzle is how to call these classes. To make this easier I have added a wrapper class:

public class ConfigurationService
{
	private static readonly Logger logger = LogManager.GetCurrentClassLogger();
	private readonly IKernel kernel;
	private readonly XamlSchemaContext context;     

	public ConfigurationService()
	{
		this.kernel = new StandardKernel(new CoreModule());
		this.context = new NinjectXamlSchemaContext(this.kernel);
	}     

	public Server Load(Stream stream)
	{
		logger.Debug("Loading configuration from stream");
		var reader = new XamlXmlReader(stream, this.context);
		var server = XamlServices.Load(reader) as Server;
		return server;
	}
}

At the moment there is only one thing we are loading – the server configuration – but later this may change. To use this class all we need to do is create a new instance, open a stream containing the configuration and call Load().

To use dependency injection with a configuration class we just need to reference one of the standard interfaces and put an Inject attribute on it:

[DesignerSerializationVisibility(DesignerSerializationVisibility.Hidden)]
[EditorBrowsable(EditorBrowsableState.Never)]
[Inject]
public IFileSystem FileSystem { get; set; }

I also add DesignerSerializationVisibility and EditorBrowsable to make sure it does not appear in any outputted XAML and the designer.

Finally the class that wires together the various interfaces and their implementations is CoreModule. This is just a standard Ninject kernel implementation so I won’t go over it here.

Hopefully this covers all the details around how we can use Ninject together with Xaml.

Note: This post is a technical post – it you don’t care about the inner workings of vNext then it can be ignored.

First I believe that we should be doing TDD for vNext. Everything that is checked in should have working unit tests that prove that it works the way it should. This has two main benefits for us:

1. We have a better idea of what we are trying to build
2. We have a better safety net for when things change

There are additional reasons why TDD is good but for me these are the main two – especially the second one! I know things are going to change in the future (just look at how much they have changed for the current version) so having the safety net makes it so much easier for us.

Based on this decision I think we should be using an IoC container for wiring everything up. In the current version there is no consistent way of getting dependent class instances. For example for getting the file system interface we have three or four different ways of doing it. As you can imagine this makes things very confusing! So for vNext everything will use IoC for resolution. This also helps with TDD as we can abstract the various external dependencies behind interfaces and then mock them as required.

Based on these two patterns there are a number of choices we have for third party libraries to assist.

For TDD we will use NUnit. This is what CruiseControl.NET currently uses so people are familiar with it and it is one of the most popular unit testing frameworks, plus it is open source. To help with mocking we will use Moq. We’ve tried both NMock and Rhino.Mocks in the current version but both of them have their challenging areas. I’ve used Moq extensively in my jobs and I’ve found it easy to use. While it does have some challenging areas on the whole I’ve found it easier to use than the other mocking frameworks. And again it is open source (we try to use open source where possible as it’s not always possible for people to get licenses to commercial products.)

For IoC we’ll use Ninject. There is no particular reason why – I asked people for their preferences a while ago and didn’t get a consistent answer. I’ve used Ninject before and liked it, plus who can dislike a ninja-themed product

Another important part of the system is logging – this provides a picture of what is happening inside the system – especially on systems that don’t have source code. Currently we use log4net but this is not maintained anymore. So instead we’ll change to NLog. Jaroslaw has done an excellent job of making NLog simple and easy to use and configure. It pretty much has the same functionality as log4net plus more.

Finally I’m going to change the XML parser we are using. As mentioned earlier we are using NetReflector but this is pretty much dead now.  So instead we’ll go with XAML and the .NET classes for it. System.Xaml provides a lot of the base functionality we need for parsing XML and generating an object tree. In addition it provides a number of extension points for going beyond the base functionality. In some future posts I’ll describe how we can extend the base functionality to tie in with IoC, logging, etc.

In this post I’m going to describe some standards I think we should have for property names. Later when I look at source control blocks, triggers and labellers we’ll see how they also apply.

First off there will be a number of “common” properties that apply to every task. I covered these in a previous post so I won’t do more than just mention them here. These common properties are:

• Name
• Description
• Conditions
• FailureActions.

In addition to these properties any tasks that execute external applications will have the following common properties:

• BaseDir: the directory where the application will be executed.
• Environment: any additional environment variables to be passed to the application.
• Executable: the name of the application to execute.
• Priority: the priority level for the application.
• Timeout: the period before the application is assumed to have timed-out. After this period the application will be killed.

Tasks that can contain other tasks will have the following common property:

• Tasks: the child tasks

Beyond these common properties the following general rules will apply:

1. Use the shortest form of the property that makes sense: e.g. use SolutionFile instead of BuildSolutionFile, ServerUri instead of Server, etc.
2. If two or more tasks use the same or similar property then they should have the same name. This rule takes precedence over 1. so if a longer name is needed to make properties unique then this name would be used in all tasks. E.g. if Output is used in one task to refer to a file and OutputDir and OutputFile is used in another then OutputFile would be used in both tasks.
3. Use Dir whenever a directory is referred to, and File whenever a file is referred to. Again this rule takes precedence over 1. E.g. SolutionFile or OutputDir. Do not use Path or Directory.
4. Use a singular name for a single property and a plural name for an array: e.g. InputFile and InputFiles. The underlying classes for a plural property should be an array or similar these should not be used with a singular name.
5. Sub-classes can be used to group together common properties that used in multiple tasks. E.g. a user class for holding user name/password, etc.
6. Time-outs should be defined as a Timespan rather than an integer – this allows more flexibility in defining times.

We will probably add some more rules as we find other anti-patterns but we’ll try not to add too many rules! And as I mentioned at the beginning of this post these same rules would also apply to the other elements in a project configuration.

Finally the other common property type I am thinking of is a verbosity level. Many executable tasks have something like quiet or no logo properties which tends to be a boolean value. Some of these same tasks also have another property to increase the output level (e.g. verbose or detailed output). Therefore this type would have three values: Quiet, Normal, Detailed. If the task does not support the output level then it would be ignored.