An Overview of the WCF Extensions

Introduction

Recently I documented adding a WCF extension to CruiseControl.Net. Due to mono-requirements I had to do it in a way that I wouldn’t force an upgrade of the CruiseControl.Net source code to .Net 3.0 or 3.5. Here is the complete set of posts:

• CruiseControl.Net and WCF
• WCF Server Extension
• Cruise Control Contract
• Transporting Data via Data Contracts
• WCF Router
• Extending CCTray
• Configuring a CCTray Custom Protocol
• WCF Custom Protocol

This will be my last post in this series as I’m preparing to move onto some more work (adding some security extensions to CruiseControl.Net). However I thought I ought to add this final post just to wrap up what I have done.

Before WCF

Before I started adding the extensions CruiseControl.Net consisted of three main components:

• Server (whether as a service or running as the console app)
• Dash board
• CCTray (the client application)

These communicated in the following ways:

Initial CC.Net communcations

As you see .Net Remoting was the only way to communicate with the server, thus the client either had to use .Net Remoting or go via the dashboard.

Extension Points

The first change was to add some extension points to both the server and CCTray. This would allow me to add the WCF code without converting the rest of the application. With the extension points the system now looked like this:

Extension points to CruiseControl.Net

I didn’t add any extension points to the dashboard as I’m planning on going directly from the server (the worker of the system) to CCTray.

WCF Extensions

WCF Extensions

Now that I had some extension points for communications, I added the actual WCF components. These consisted of a server-side WCF extension, plus a custom transport protocol for CCTray.

The server side extension used the new hooks in CruiseServer to start and stop/abort a WCF ServiceHost. I added a service contract, ICruiseControlContract, plus its implementation and a number of data contracts to allow data extract. The only thing that a server administrator now needs to do is configure the server to use WCF – which uses the standard WCF configuration for the following service/contract combination:

• Service: ThoughtWorks.CruiseControl.WcfExtension.CruiseControlImplementation
• Contract: ThoughtWorks.CruiseControl.WcfExtension.ICruiseControlContract

CCTray is a custom transport extension. I added transport extensions so other people can also build their own transport mechanisms if desired. The main points to a transport extension is they must provide implementations of ICruiseProjectManager and ICruiseServerManager. These implementations do most of the actual work in CCTray, and they are returned from factory methods in ITransportExtension. For the WCF extension I implemented these classes and passed on all the calls to the client code generated by svcutil.

There is no need to modify the app.config for CCTray as the bindings and endpoints are generated in the code from user input – making it even simplier for end-users to set it up on their machines.

Routing Messages

The final piece of the puzzle is a router to sit in IIS:

WCF Extensions plus Router

This is required for installations that already have IIS listening on port 80 and want to use HTTP transport within WCF. Without the router we would have to use a different port to listen for requests.

The router merely needs to be added to IIS (either in an existing .Net 3.0 application or in its own application) and then its web.config modified to point to the listening WCF extension (which can listen on any port or protocol it likes).

What’s Next?

That, in a nutshell, is the work I did to allow WCF to be used in CruiseControl.Net. The extension points are general and can be used for other, non-WCF extensions.

Now I haven’t quite finished the work on the WCF extensions – they are missing one important component. Currently I am not passing any user information around! This is a major issue as ForceBuild(), AbortBuild() and FixBuild() all need this information. However I haven’t figured out an easy, secure way of passing this information, so I’m going to leave it for the moment.

At the moment I’m working on fixing a few issues that have been raised in Jira to try and get a better understanding of the code. Once I have finished these I’m going to look into adding a security layer to CruiseControl.Net that is easy to configure, flexible to expand and yet powerful enough to handle any security scenarios.

WCF Custom Protocol

The Story So Far…

This is just a quick recap of what we’ve done so far. If you’ve been following along you can probably skip down to the next heading.

This series has been based on the task of adding Windows Communications Foundation (WCF) to CruiseControl.Net. However there was one condition that made it a bit harder – the additions must not force an upgrade to .Net 3.0/3.5 or add code that can’t be compiled on Mono (e.g. for Linux users).

To get around this condition I first extended the CruiseControl.Net server to allow developers to write their own custom extensions that hook into the server process. With this in place I then developed a custom extension that added a WCF service host to the server, designed a contract of what would be provided and implemented the interface.

The next step was to add a router to the dashboard to automatically forward requests onto the server.

With these steps in place we were ready to tackle the client – CCTray. The next two posts covered adding the ability to add custom protocols to CCTray and to be able to configure them. This post now handles the final step – implementing a custom protocol to handle using WCF.

A Simple Task

In order to add a custom transport protocol, there are three interfaces that we need to implement:

• ITransportExtension
• ICruiseServerManager
• ICruiseProjectManager

ITransportExtension is the new interface we added to allow custom protocols, while ICruiseServerManager and ICruiseProjectManager already existed in CCTray. We need to implement these as they provide the glue between our transport and CCTray’s inner workings. They are created by calling RetrieveProjectManager() and RetrieveServerManager() on the ITransportExtension interface.

I started a new project called CCTrayExtensions and implemented the interfaces as follows:

• WcfExtension implements ITransportExtension
• WcfServerManager implements ICruiseServerManager
• WcfProjectManager implements ICruiseProjectManager

The last two classes open a WCF client and call the appropriate method on the client. Yes, all they are is pass-through classes to send the actual requests onto the server. They use auto-generated code – all I did is call svcutil on the WCF endpoint.

WcfExtension handles instantiating WcfServerManager and WcfProjectManager, plus it also displays a configuration dialog and converts between Settings (the property on the interface) and the internal settings (e.g. the transport binding to use). The configuration dialog is a class called WcfConfigurationWindow - this looks like the picture below:

WCF Extension Configuration Dialog

The binding is stored in an enumeration – TransportBinding. As endpoint is stored in the Url property the only value Settings needs to store is this binding, so all the extension does is convert the binding to a string and back.

The final important task of WcfExtension is to open and close the clients. WcfExtension handles the open so it can retrieve the binding and endpoint and correctly configure the client. Close is just to make sure everything is tidied up correctly – plus later on I plan on looking at caching the client so we don’t need to open a new client everytime (which can become expensive for some bindings).

And that’s all there is to the extension – most of the hard work has already been done in adding the custom transport protocol to CCTray.

And In Conclusion

This concludes adding WCF to CruiseControl.Net – both on the server side and the client side. This involves a set of changes to the core libraries, to allow extensions, plus some WCF-specific extensions, so we don’t break our Mono compilation requirement. I’ll add the code to JIRA under two patches – extension-enabling code, plus the actual WCF extensions.

However, I still have a few things to tidy up, so I won’t stop this series just yet. Here are some of the other items I’m thinking of covering:

• Unit testing the code (yes I did write unit tests, but for these posts I wanted to focus on the actual functionality)
• Modifying the installers – it would be nice to get the installers to also install the WCF extensions for those who want it
• Additional WCF functionality – security, logging, dual-channel communications
• How to write a server extension, and also how to write a custom transport – these will be step-by-step guides (if anyone wants to try their own extensions)
• And hopefully an overview of the whole series

As you can see, I still have plenty of ideas on what to write! However, now that I’ve finished documating most of my work to date it takes to get back into some development, so the documentation will probably slow down (I’ve been focusing on the documentation over the past week so I don’t forget anything.)

Hope you enjoy what I’ve done so far. And if anyone has any questions on what I’ve done, or suggestions on improvements, please let me know.

WCF Router

A Small Detour

Before I move on to the client side extensions for WCF, I realised I have to make a small detour. In my previous posts I talked about how I added extensions to CruiseServer and then built a WCF extension. This worked beautifully – until I tried to deploy the server to a machine where I already had IIS running. What was the problem? The port for HTTP (port 80) was already being monitored by IIS. Therefore all requests were being processed by IIS instead of my WCF service!

Now I could turn off IIS on that machine, but then I’d lose all my nice web sites (including the dashboard for CruiseControl.Net). I could map either IIS or the WCF extension to use a different port, but the entire aim of this exercise is to use port 80 so we can get through our firewall. Therefore I needed to come up with a different solution.

Introducing the Router

Thankfully WCF came to the rescue here. WCF can be hosted natively in IIS (or at least natively via an ISAPI handler), therefore all I needed to do was add a .svc file to the web site and use that instead. However I didn’t want to redo all my work with adding WCF into the service (besides which, as far as I understand I’d then have to use remoting between the .svc file and the CruiseControl.Net server). So I had a look around to see what I could do to pass on messages.

It turns out that this is a very common scenario – something called routing (see the articles by Michele Leroux Bustamante part I and part II). So using the example code I quickly whipped up a WCF router to be part of CruiseControl.Net.

Again this is a separate project (see this post for the reasons why). This is a very simple project – it only contains one interface (IRouterService), one class (RouterService), an .svc file and the web.config. The interface defines the contract or in this case merely tells WCF that we’re going to handle every single message ourselves. The class is the implementation of the contract. Here we merely receive the message, generate a client to the “real” service and then pass on the message.

The web.config needs a bit more work than just the service or the client requires. Instead it needs both a service definition (to handle incoming messages) plus a client definition (so it knows where to send messages to). Visual Studio added the service definition for me automatically and I just copied and pasted the client definition from what svcutil generated for me. I needed to modify it a bit (had to change the contract attribute to use our interface) and then it worked beautifully!

To finish things off all we need to do is compile the project and then add the web.config, .svc file and DLL to the web site. When I get around to finding out how the installer for CruiseControl.Net works I can see if it’s possible to do it in there.

It’s Not All Bad News

Now that I’ve started on a router it does open up a few new possibilities:

• I could possibility use the router to handle security (i.e. integrated security in IIS and pass on security headers to the actual service).
• The router could also be used in the same way as the dashboard – receive messages from a client and route them to multiple CruiseControl.Net servers.
• I could even use it for switching protocols (i.e. accept incoming requests over HTTP and use named pipes or TCP to connect with the server)

It’ll take a bit more investigation, but the list goes on…

Where to Next?

Now that I have the CruiseControl.Net server exposed as a WCF service and I can access it over http, we need to look at the client-side. In my next few posts I’ll look at what I did to allow custom transport mechanisms in CCTray, then my WCF transport and hopefully finish things off with an overview of the entire mechanism.

Transporting Data via Data Contracts

Catching Up

In my last post (here) I showed how I implemented the web service for CruiseControl.Net using WCF. However I stated that I couldn’t transfer data across the wire using the standard CruiseControl.Net data entities (e.g. CruiseServerSnapshot, ProjectStatus, etc.) In this post we’ll look at what I did to get around this issue.

Data Contracts

WCF automatically serialises/deserialises any basic types (e.g. integers, reals, strings, booleans, etc.) However it doesn’t handle complex types (e.g. classes) by default. I had thought that if a class was marked serializable then it would handle it, but I was wrong.

Instead to allow passing complex types WCF provides a set of attributes for marking up classes. The two basic attributes are DataContract and DataMember. DataContract tells WCF that the class can be passed over the wire, while DataMember tell WCF which properties/fields to include. Unlike XML serialisation, WCF uses an opt-in model – in other words we have to tell WCF what to include. Like service contracts we can also include a namespace – which I have done. This is the same namespace as the service (http://ccnet.thoughtworks.com/1/5/wcf).

Now I could have modified the entities in the Remote project to add the WCF attributes, but that would break one of the requirements for my code and force a migration to .Net 3.0. So, instead I added some new entities to the WcfExtension project.

What to Send

I decided that I was going to change the structure a bit for the data that I send using WCF. Mainly it’s because I wanted to use the generic List<> rather than writing my own collections (yes, I’m a lazy coder). The following shows the entities that I decided to implement:

WCF Data Objects for CruiseControl.Net

These map to the standard CruiseControl.Net entities in the following way:

• Snapshot maps to CruiseServerSnapshot.
• Project maps to ProjectStatus
• Queue maps to QueueSnapshot.
• QueueRequest maps to QueuedRequestSnapshot.

Each of the WCF entities has a constructor that accepts the standard CruiseControl.Net entity. The WCF entities have the same properties as the standard entities.

With this in place, all I had to do was modify the service contract to use the new entities. Then I modified the methods that returned data (RetrieveSnapshot(), RetrieveProject() and RetrieveProjects()) to use the new entities. And then the WcfExtension is already to roll!

Coming Soon: The Client

Now that I’ve finished the server-side of the WCF extension it’s time to start looking at the client-side. Initially for testing I had written my own client – but this had only a subset of the full CCTray functionality. Obviously I don’t want to redo all the work that’s already done, so it’s time to start looking at modifying CCTray to handle WCF protocols.

Cruise Control Contract

Previously

In my previous two posts I covered the basics of what I am planning to do (here) and my initial work around adding a WCF extension to CruiseControl.Net (here). In this post I’ve delve into ICruiseControlContract and its implementation – CruiseControlImplementation.

Why a Contract?

The purpose of a web service is to allow two systems to communicate. And just like we have a contract on how we communicate (most of us use a contract called English) so web services need one. In technical terms a contract lists the methods it exposes to the external world, what information it expects in order to perform those methods and what data it will return. If you’re thinking this sounds very much like an interface, you right on! Contracts in WCF are basically a sub-set of the interface functionality (properties and events aren’t part of a contract). In addition there are various attributes we can apply to the contract to affect the definition of the contract (namespaces are a piece of metadata from an attribute).

While the contract attributes can be applied just as easier to a class, it is recommended practise that we use an interface as a contract. So we’ll stick with this practise and define an interface called ICruiseControlContract. The purpose of this interface is list all the methods we’ll expose for working with CruiseServer.

The very first thing I did was add an attribute to tell WCF that this interface is a contract. At the same time I told it the namespace that this contract would use:

[ServiceContract(Namespace = "http://ccnet.thoughtworks.com/1/5/wcf")]

I’ve given our contract a multi-part namespace. Most of this should be obvious, but we’ll break it down:

• ccnet.thoughtworks.com: this is the base domain to use, in our case we’re on ccnet which belongs to thoughtworks.com.
• 1/5: this is the current development version of CruiseControl.Net. As I’m hoping to have the work finished by the next release I’m using 1.5 as the version number.
• wcf: this is the extension name – in our case WCF. I might change this later on to something more meaningful, but I can’t think of anything at the moment.

Designing the Contract

Now that we have started on the contract the question arises as to what should we put in it? In my initial development I added the following methods:

• RetrieveVersion()
• RetrieveSnapshot()
• ForceBuild()

I added these for testing purposes – mainly could I could find them easily in CruiseServer. The diagram on the right has the full list of method that I expose. Basically I’ve marked all of the methods in this interface as web methods, since that’s the sole purpose of this interface. The additional methods I added were:

• AbortBuild()
• CancelPendingRequest()
• FixBuild()
• RetrieveProject()
• RetrieveProjects()
• StartProject()
• StopProject()

I got this full list by mainly looking at CCTray (ICruiseProjectManager and ICruiseServerManager). This should be the full functionality to handle all of CCTray’s requests. RetrieveProject() was added as an extra to allow me to view a single project – rather than getting the full list (which could be a bit of an overkill, especially when we serialise/de-serialise).

The final note on the contract, in order to expose these methods I needed to add an OperationContract attribute to each of the methods. This is just how WCF works – it uses an opt-in model for the contract. The attribute I added to each method was:

[OperationContract]

I haven’t bothered setting any metadata yet. Perhaps if we need it later on I’ll add it in. I should mention that these methods will use the same namespace as the interface (i.e. “http://ccnet.thoughtworks.com/1/5/wcf”).

Now that we have our contract, let’s move onto making this contract work.

The Implementation

Finally, we’re ready to actually make the web service do something. The diagram on the right shows my CruiseControlImplementation class. You’ll notice it implements all the methods from ICruiseControlContract. These methods are very simple in their implementation.

If the method receives any parameters (e.g. ForceBuild(), RetrieveProject(), etc.) then the parameters are validated. The validation is first checking they are not null (if we expect a value). Then for project name parameters we check that the project actually exists. Once the parameters are validated the method call gets passed onto the CruiseServer instance (which is being held in _cruiseServer). Any outputs from the instance are passed back to the client.

To help with the validation I added a couple of private helper methods:

• CheckProjectExists()
• ValidateProjectNameInput()

These do exactly what their names are.

The final method is a helper method for getting the name of the current user: RetrieveCurrentUserName(). At the moment this isn’t working, but when I get around to adding security to the web service I’ll get it up and running.

Next Step

Now that the web service is hopefully finished it was time to test it. I built a quick test client that just called the initial three methods and tried to send and retrieve data. However it failed – badly The problem was CruiseControl.Net uses some nice custom classes for passing around its data. Since these were marked as Serializable I had hoped they would work as is. The problem is they didn’t. So for my next, and hopefully last post, on the service side of WCF I’ll talk about the data contracts I added.

WCF Server Extension

Previously

In my last blog post (here) I talked about adding some extension points to CruiseServer in CruiseControl.Net. These extension points form the basis for adding custom extensions to the server application. Any extension to CruiseServer must implement ICruiseServerExtension.

There are currently four methods that need to be implemented in this interface (I’m thinking about adding some more later on to be triggered when a build starts, finishes, etc.) These methods are:

1. Initialise() – prepare the extension for use
2. Start() – the cruise control server has started
3. Stop() – the cruise control server has stopped
4. Abort() – the cruise control server has been aborted

These four methods will form the basis for any extension that wants to tie into the CruiseControl.Net server. As stated previously the CruiseServer instance is passed to the extension in the initialise method, so our extension classes can call any method that CruiseServer exposes.

Introducing WCF

The entire aim of these extension points is to allow us to add WCF functionality to CruiseControl.Net without forcing a migration to .Net 3.0 or later for the entire solution. One of the neat points about .Net is that an assembly built in one version of .Net can use an assembly built in another version. This is what we are counting on to make this functionality work.

To start with I’ve added a new library to the solution that uses .Net 3.5 (so we can also use some of the nice Linq functionality). In this library I’ve added a class that implements ICruiseServerExtension called WcfServerExtension. This class also implements IDisposable to ensure that everything gets cleaned up. Our four basic methods are implemented (Initialise, Start, Stop and Abort) plus Dispose from the IDisposable interface.

Also in this class are two private fields: _cruiseServer and _wcfServiceHost. _cruiseServer is a reference to the CruiseServer that initialised the extension. We hold onto it just in case we need it later. _wcfServiceHost is literally what the field name says – it is a WCF service host. This host gets instantiated in the Initialise() method call. When it is instantiated it is passed a singleton instance of CruiseControlImplementation (more on this later). And that’s it to the Initialise() method. This service host instance is then either stopped, started or aborted in the various methods required by ICruiseServerExtension. The Dispose() method just calls the Abort() method.

Finally IsRunning is just a helper property we can check to see whether the service host is open (has been started) or not.

And with that WCF makes it incredibly easy to host a web service in our application.

The Contract – Exposing Ourselves to the World

WCF uses the concept of contracts to tell other systems what operations are exposed. In our case I have added a contract called ICruiseControlContract which is an interface (we could have exposed it as a class but it’s best practise to use an interface. This interface tells the world what methods we are exposing in our web service (I’ll write on this interface later in a future post).

CruiseControlImplementation is an implementation of ICruiseControlContract. Basically all it does is receive the calls from another system, validate the input parameters (since we shouldn’t trust any external systems) and then passes the call onto the CruiseServer instance. All nice and simple.

This approach means we don’t have to worry about which version of .Net the rest of the system uses, and we don’t have to know the nuts and bolts of how CruiseServer works. In my next post we’ll take a look at ICruiseControlContract and CruiseControlImplementation. Once we have covered these two items we’ve just about finished the server side of adding WCF to CruiseControl.Net. After that is how do we modify the client (CCTray) to use the WCF services that we’ve exposed.

CruiseControl.Net and WCF

CruiseControl.Net

I’ve been using CruiseControl.Net for years. As a Continuous Integration (CI) server it is without peer in the open source community (heck I even think it’s better than some of the non-open source versions). I like it so much that when I changed to my new job a year back that first thing I did was set up a CruiseControl.Net server.

However as much as I like the application there are a few things that bug me. One is the dependence on .Net Remoting for communications between the client (CCTray) and the server. I know it’s possible to use an HTTP transport but I’ve never had any success in getting it to work as nicely as the Remoting transport.

Until my new job I was able to live with it. But at my new job we use a Virtual Private Network (VPN) for remote access, and sometimes it’s nice to work from home. The down side is the VPN only allows certain ports – everything else gets blocked. And surprise, surprise the port that remoting uses is blocked. I could swap to port 80 but then I won’t be able to use the web sites on our build machine (it also hosts a number of other development tools like our source control and bug tracking applications), plus I think the web dashboard is critical for investigating why the build failed.

So it’s time to delve into the world of Open Source and get involved in improving an application. Initially I’m planning on adding WCF to CruiseControl.Net, but I also have a few other ideas to add in (more on them later).

Welcome to Open Source

I read through the instructions on getting involved in development and one of the things it recommended was sending out development ideas to the group of developers involved. So I joined the mailing groups and asked of their opinion on WCF. Overall the feedback was that it sounds liked a good idea – with one caveat. This was it needs to work in Linux under Mono – which doesn’t implement WCF (yes I know there is the Olive project, but it doesn’t seem like much is happening with it). So any development on adding WCF to CruiseControl.Net mustn’t break compatibility with .Net 2.0 (which added some challenges considering WCF is part of .Net 3.0 and later).

To get around this what I’ve been doing is making the WCF components as extension components to CruiseControl.Net and CCTray. The first part is actually adding the extension points to both the server and CCTray (neither of them has the deep integration points that I’m after).

The Nuts and Bolts

My initial work has involved adding some extension points to the server and testing whether it is possible to expose some functionality as a WCF service. As I’m wanting the WCF service to start and stop when the server does I’ve added an interface called ICruiseServerExtension. This exposes four methods:

1. Initialise()
2. Start()
3. Stop()
4. Abort()

Start(), Stop() and Abort() are called by the server when it starts, stops or aborts (simple enough). Initialise() is called when the extension is first loaded. To do this I have added a new method to CruiseServer called InitialiseExtensions(), plus added an extra parameter to its constructor. The extra parameter is simply a List<ICruiseServerExtension> of all the extensions to load (more on how this list is generated later). If there are extensions in the parameter then InitialiseExtensions() is called. This performs the actual loading of the extension object then calls initialise and passes in the CruiseServer instance plus the configuration for the extension.

The list of extensions to load is stored in the app.config file for the server. When CruiseServerFactory performs the creation of a new local ICruiseServer it also loaded the definitions from the config file. I added ServerConfigurationHandler to handle the parsing of the config and return the list of extensions.

That pretty much explains the new extensions to the server side of CruiseControl.Net to allow WCF without forcing everyone to use .Net 3.0 or later. In my next post I’ll talk about the actual WCF extension and how it works.