Documentation Published

I have been working on writing some documentation on the various configuration options. I’ve now completed all the configuration elements to date, and I aim to update this as I add more configuration options.

The documentation can be found here.

When the security work is finally moved into the main trunk then I’ll also look at migrating this documentation to the wiki. However the format is slightly different as the editor doesn’t allow me to add tables easily (yes, I could write them in the HTML but I don’t want to.)

Securing CruiseControl.Net – Web Dashboard (Part III: Authentication)

The Story to Date

Previously I’ve written on my modifications to add security to CruiseControl.Net. While this has been a multi-part set of changes, I’m nearly at the end. Initially my changes involved modifying the server components (Core, Console and Service) to have security enabled. This was followed by modified CCTray for .Net remoting. In my last post I started modifying the dashboard, but only to allow HTTP access from CCTray. In this post I’ll finally look at adding security to the web dashboard.

But first, a quick refresher on how security works in Core. In core there is now an ISecurityManager manager associated with the ICruiseServer instance. This is the interface that actually handles security – thus removing the need for ICruiseServer to know anything about security. Instead, all ICruiseServer has is a set of session management methods (login and logout), plus the secured methods have been extended to require a session token.

Thus to add authentication to the dashboard I need to receive a set of user credentials and pass them onto the server. This is where it gets tricky as there can be a number of different types of credentials – for example I implemented a simple user name, and a user name/password set for credentials. In CCTray the user selected which credentials to send and the server then validated them. The dashboard doesn’t have this option as it is setup remotely by an administrator. Plus it is stateless, and it can monitor mulitple different servers!

To keep things simple, my initial implementation of security will have the following limitations:

• It must be configured by the administrator
• Only one type of security can be configured
• Only one server can be monitored
• The session token will be stored in the query parameters

Later on I hope to work around these limitations, but this will have to do for a start.

A New Type of Plugin

Like the server, the dashboard uses plugins to allow new or modified functionality. The configuration is exposed via IPluginConfiguration, which is returned from an IDashboardConfiguration. The actual loading of the configuration is handled by DashboardConfigurationLoader, which uses Exortech.NetReflector to dynamically load the configuration.

IPluginConfiguration contains definitions for FarmPlugins, ServerPlugins and ProjectPlugins, which return arrays of IPlugin, plus a defintion for BuildPlugins which returns an array of IBuildPlugin. I’m going to modify this to add a new definition – SecurityPlugins. This will return an array of a new interface – ISecurityPlugin. This is an extension of IPlugin that contains an extra method stating whether it is valid for a server.

IPluginConfiguration is implemented by NetReflectorPluginConfiguration. This contains the Exortech.NetReflector decorated properties so the library can dynamically load the configuration. Therefore I modified this class to include a new property for the security plug-ins. This is an optional reflector property.

First Security Plugin

For testing purposes I’m going to add an implementation of the simple user name authentication. This plugin is called UserNameSecurityPlugin, and it implements ISecurityPlugin and ICruiseAction. ICruiseAction is required since the login will be action-based (otherwise it won’t fit in with the rest of the dashboard).

The action is very simple – it just generates a view with a field for the user to enter a user name. On postback this field will be sent to the server and the response displayed.

Adding the Login Link

The next step is to add this plugin to the UI somewhere. As I stated in my plan, I want to add this to the header next to the Documentation link. This link is defined in SiteTemplate.vm, which is an NVelocity template. So, I modified this template to insert the login link.

The values in SiteTemplate.vm are set from SiteTemplateActionDecorator, so this is where I will need to set the security link. Now, the way the main template works is it has a number of sub-templates, which are built via builders. To maintain this pattern, I’ve added a LoginViewBuilder which will take in the values and generate an HTML fragment containing the login link (or any other login functionality, e.g. logout).

The builder simply checks the configuration for any security plug-ins. If one is found, and the request is for a server, then it generates an HTML fragment containing a link to the first named action. Later I can expand this for handling multiple login actions.

Handling a Login Request

Now that I have the login button added, the next step is to handle a login request. If I try to run the code now I get an objection error saying the object can’t be found. This is because the request controller is trying to find an action that matches my login action. To resolve this I modified CruiseObjectSourceInitializer to load all the security plugins and store them in the object store. Additionally I added all the same decorations as for a server plugin (since I’m not sure which ones I will need).

Now when I run the code I get my nice little template displayed And since I’ve already implemented the action I can test my login (it works nicely).

Now I need to store the session token somewhere!

Storing the Session

My plan is to store the session token in the query portion of the URL. Now, while this is simple in concept, there is a huge number of URLs that can be generated for a page! What I need is a single place that I can modify so that all URLs are affected. And thankfully this is provided by IUrlBuilder, but it will take a bit of work to achieve this.

First, IUrlBuilder is part of core, not dashboard. So any changes need to be generic enough to not break the server. Because of this I can’t have one class for storing and loading the session. So I’ve split it into two – ISessionStorer and ISessionRetriever. ISessionStorer just has a property for storing the session token and a method for converting this into a query string parameter. ISessionRetriever is the same, but in reverse (it searches the query string for the session token).

Next I added an implementation of these classes and modified CruiseObjectSourceInitializer to store these as instances (so that objection will automatically load them). And that’s where my code came unstuck

The problem took me a while to work out, but it was very simple in the end. When I implemented my plugin and action I made them the same class. Now the problem is plugins are cached, whereas the rest of the entities (including my ISessionStorer) are not. Therefore when my new action was called, it was using the cached plugin, with the cached ISessionStorer. Everywhere else the new instance of ISessionStorer was being used!

So, time to do some more refactoring. First I added a new property to my ISecurityPlugin - an ISessionStorer property. Secondly I split UserNameSecurityPlugin into UserNameSecurityPlugin and UserNameSecurityAction. When the actions are being retrieved from the plugin it creates a new instance of an UserNameSecurityAction. Prior to this I set the ISessionStorer property on the plugin and now my new action is using the same session storer as everywhere else. The action now does all the actual execute functionality on its own.

Now when I login all the links now have the session id in them

Logging Out

It doesn’t make much sense to have a login button when the session has already been generated, so I went back and modified the LoginViewBuilder to return “Login” if there is no session and “Logout” otherwise.

All this required was modifying the builder to return a logout action when there is a session. This is a default action that just clears the session token from the query string and sends the user to a view telling them they are logged out.

Authentication Completed

That is now the end of the authentication in the dashboard. I’ve built a framework of plugins and actions to handle logging in to the server and storing the session token in the query string. As it’s taken me a while to figure this out, I’m going to finish here for the moment.

In my next post I’m going to look at making the session storer and retrievers dynamic, so I can build alternate implementations. Then I’ll take a look at authorisation in the dashboard. Once that’s done then I’ve finished the basics of adding security to CruiseControl.Net!

Yay

User Experience – Part 1

 

Oh My!

Now that I’m nearing complete on the security changes I thought I’d finally install my new secure version of CruiseControl.Net at work. This series of posts will cover the fun and games I’m having with trying to get it work in a user-friendly manner!

The Server

Installation was nice and easy – I just copied over the binaries. When I brought everything back up it appeared to be working fine – CCTray displayed the details correctly, and I could still do force builds.

The next step was to configure the server to use some security. Since I’ve done a lot on security since the server modifications I had to go back through my old posts to find out how to configure security – bad point #1.

After I had dug up the security settings I then proceeded to add some security settings. First off, I added the session security section and defined some password users, a catch-all simple user and a couple of general roles (admin and general). I picked one of the projects were I’m the only member and added the security to it. Then I tried saving the file – and got errors! My posting on security settings was slightly incorrect – I haven’t gotten around to implementing the defaultRight property yet – bad point #2.

CCTray

Once this was all sorted out and the server happily running again, it was time to try integration with CCTray. I copied over the new CCTray binaries and fired it up – looked good to start with. Then I went to configure the security and got the following window:

 

Choosing a security provider in CCTray

Yes, how do I know which provider I want to use? Bad point #3.

Once I’d found the right provider (via trial and error) and got it configured, it then worked fine. I changed the security around, attempted a few force builds and got roughly the results I expected. The only down side was CCTray doesn’t tell me when a request has been denied – bad point #4.

So, I left CCTray running for a while and then tried another force build. After waiting for a while, I realised that nothing was happening! Looking in the error logs I discovered that the session had expired and so the user couldn’t be authenticated! Bad point #5.

Enough Points!

I did play around for a while longer (only in CCTray, dashboard isn’t ready yet), and didn’t find any other points (yet). So I decided to call it a day on my testing and start planning my resolutions. Here are the five points I discovered:

1. Security settings are undocumented
2. Security settings are not fully implemented (defaultRight)
3. Can’t tell which provider is being selected
4. No warnings when a force build can’t be started due to security
5. Sessions time-out without any warnings

And here’s how I’m planning on resolving these issues:

1. In the short term I’ll write a couple of articles on the security configuration, what is allowed, what is required and what is missing. Medium-term I’ll get this documentation added to the wiki. Long-term I’m thinking of building a security editor that will have a nice UI for setting security (perhaps both Windows and web-based).
2. Finish implementing the security settings – this includes defaultRight, plus doing authentication for starting/stopping projects.
3. Add some attributes to the providers to provide friendly “names” to the security providers (would also help for the transport extensions).
4. Find out why warnings aren’t being displayed (I’m sure I put some in there).
5. Add the ability to re-login if the session has timed-out.

Now I should mention that this is only the beginning of my testing, and so far it’s just the issues that I don’t like. I also have a few other ideas on how I’d like the security to work in order to be more user-friendly which I’ll implement in time.

On a side note, I probably won’t fix these issues in the order above, as I’m still working on the dashboard changes, plus some are easier than others. (yes, I will pick the easier ones first!)

So, that’s it for now, if there’re any issues you’ve found please let me know and I’ll add it to my “bad points” list.

Securing CruiseControl.Net – Web Dashboard (Part II: CCTray Integration)

The Plan

The plan for CCTray integration is very simple – add a new action to handle XML security requests and modify the existing force/abort build actions to use session tokens. Initially I was going to modify all the methods to take in sessions, but since I’ve only modified force/abort build there’s no point in modifying any others.

Authentication

Authentication is something that is new to the dashboard – it’s never had to allow for it before (in any form). Therefore to handle this I’ve added a new action – XmlServerSecurityAction. This will be a generic action for handling any non-project based security requests. Currently the only actions I’ve implemented are:

• Login
• Logout

All parameters to this action must be passed in as form post data. The two required fields are action and server. Action is which action to perform (login or logout), while server is the machine to apply the action to. Other fields may be required depending on the action.

The Execute() merely checks the action and then calls a specific method for the action. If the action is unknown it will return a failure response.

Login

The login action expects a set of credentials in an XML format. These credentials are deserialised and passed onto the Login() method of ICruiseManager via IFarmService. I had to modify ServerAggregatingCruiseManagerWrapper and IFarmService to include this method (plus Logout() below), but this is a simple matter of passing on the arguments.

If the login is successful then a success response is generated containing the session token. Otherwise a failure response is returned saying that the login failed.

Logout

The logout action is very similar, but instead of credentials it expects a session token. Again this is passed onto the Logout() method of ICruiseManager.

If the login is successful then an empty success response. Otherwise a failure response is returned saying that the login failed.

Responses

All the responses are in XML. A responses looks like the following:

<security result=""></security>

The result will be either success or failure, depending on what happened for the action.

If the result is a failure then the security tag contains a message telling what sort of failure. Otherwise the tag will contain any relevant data. For example, a successful login result returns the following:

<security result=""><session>token</session></security>

It is up to the client to parse the result and decide what to do.

Registration

The final part of authentication was to modify CruiseObjectSourceInitializer to always register an XmlServerSecurityAction. This was just added at the bottom of the method, along with the other CCTray actions.

Authorisation

Handling authorisation is even simpler than authentication. The only tricky part was finding out where to implement it. VelocityProjectGridAction is the action that CCTray calls to do a force or abort build request. Within this class this a simple check to find out which action, and then control is passed onto the relevant method in ICruiseManager (again via IFarmService). Again IFarmService and ServerAggregatingCruiseManagerWrapper had to be modified to allow the security pass-through.

My only modification to VelocityProjectGridAction was to check for a session token and pass the result onto the relevant method.

CCTray

To start with in CCTray I needed to modify IWebRetriever and WebRetriever. These classes have a POST send method, but it didn’t handle the response. So I modified the Post() to retrieve the response and return it as a string. If it’s not needed then the caller can ignore it.

I then modified WebDashboardUrl to return the correct URL for my new security action.

To allow for authentication I needed to implement my prior stub methods in HttpCruiseServerManager. These methods are Login() and Logout(). The pattern is very similar to what I implemented for .Net remoting, except I send a HTTP post and process the response. The sequence of steps for logging in is:

1. Check for authentication, if no authentication then exit the method
2. Initialise the authentication type and pass in the security settings
3. Generate and send a POST request containing the serialised credentials.
4. Process the response, if successful store the session token

Logout is even simpler, just send a POST request containing the session and then clear the session token.

To help with the POSTs I added a helper method called SendSecurityAction(). This works out the URI, adds the parameters as form fields to the POST and then calls Post() on WebRetriever. Afterwards it loads the response into an XmlElement and returns it. The caling methods then parse this element and extract the items they need.

Finally I modified HttpCruiseProjectManager to pass the session token for the force and abort methods. This was a simple matter of adding the session token to the form fields – all the rest is already handled.

CCTray Integration Completed

And that finishes the integration for CCTray. CCTray now handles security for both .Net remoting and HTTP requests. Of course, this will only work with the latest version of the dashboard – I imagine earlier versions will fail with an error. In this case the resolution is to not set security!

So, this finishes the easy part of the dashboard modifications. Coming up next is adding authentication to the UI of the dashboard!

Security now in Subversion

Update on Development

David has now given me a branch in Subversion to put all the work I’m doing on security in Subversion. If you’re feeling adventerous you can now download my latest code and try it out.

This morning I checked in all my work-to-date, which includes authentication and authorisation on the server, plus the modifications to allow CCTray to use security via remoting.

I am currently working on the dashboard side of things and hope to have it finished soon. But since I now have access to Subversion I’ll check in my code at the end of each post.

The location of the security code is https://ccnet.svn.sourceforge.net/svnroot/ccnet/branches/security. You’ll need a Subversion client (e.g. TortoiseSVN) to access it.

Writing a Dashboard Plug-in

Extending the Dashboard

Adding new functionality to the dashboard is very simple. The current structure provides a clean, easy interface to add new views. Of course, it depends on just what is being displayed, but the actual building of a plugin is simple!

One area is that not included in the dashboard is the ability to see the queues on a server. So, to test adding a new plugin I decided to add this functionality.

The Plugin

The first question when adding a plugin is at what level will it be displayed: farm, server, project or build. Build plugins implement IBuildPlugin, all others implement IPlugin. Additionally, most plugins also implement ICruiseAction. IBuildPlugin and IPlugin provide the registration information for the plugin. This consists of two properties – NamedActions and LinkDescription, which are used internally when adding a plugin to the side bar. ICruiseAction is the actual processing that is done by a plugin. These can be seperate classes, or they can be a single class.

Since queues is at a server level, I’m implementing IPlugin. Since the actual processing will be very simple, I’m also implementing ICruiseAction. My description is “View Queue Status”, and I’m only exposing one action – “ViewServerQueues”, which points to the same class.

Since I’m wanting to get the status from the server and display it using HTML, I’ve added two arguments to the constructor for the class. One is an IFarmService, which will handle all access to the server, and the other is an IVelocityViewGenerator, to generate the HTML. These will both be populated by ObjectStore when the action is instantiated.

In the Execute() method I retrieve the snapshot from the server. The server is part of the incoming ICruiseRequest, so I don’t need any complex processing to find this. There is already a method on IFarmService for retrieving the server snapshot, so I just called it with the server name. After that it’s merely a matter of populating an hashtable with the queues from the snapshot and finally calling the IVelocityViewGenerator to generate a template with the hashtable.

The final part of the plugin is to decorate it with the correct Exortech.NetReflector attributes. Since this doesn’t have any configuration options, all that is needed is a ReflectorType attribute to tell the dashboard which class to instantiate.

Simple!

The Template

The template just contains the HTML+NVelocity markup for displaying the queues. To keep things simple I use a table. The table has a header row, and then iterates through all the queues, outputing the queue name and then the project requests (one per table row).

The NVelocity engine automatically handles lists and objects very nicely. For example to iterate through an array, all that is needed is the following statement:

#foreach ($queue in $queues)

$queues is the value I added the hashtable in the plugin. $queue is the name that each individual item will be referenced by. The #foreach command tells NVelocity to iterate through the array.

To use an object (and the properties on an object) is the same as using properties in C#. NVelocity uses dot-notation, so to display the property is just the name of the object, plus the property name (this is the same name as the .Net class property). For example to show the queue name, the following statement is used:

$queue.QueueName

And that’s all there is to the template.

Registration

The final part of a new plugin is registering it so it appears in the dashboard. This is done by modifying dashboard.config. This file contains a couple of sections, but the main one of interest is <plugins>. Within this section are different sections for each area of the dashboard (farmPlugins, serverPlugins, projectPlugins and buildPlugins). All I need to do is find the area I want, and then add my plugin in the correct position (yes, position is important – it determines the position in the display).

And that’s it, the dashboard will now automatically add the plugin in the correct position.

Conclusion

And that’s all there is to adding a plugin. Compile the code and view the dashboard and the queues status is now there This took me less than an hour to put together.

So now that it’s so simple to add new functionality, the only limiting factor is what we want!

I’ve attached the code here if anyone wants it. I’ll also raise a patch and get it added to the trunk.

Securing CruiseControl.Net – Web Dashboard (Part I: The Plan)

An Introduction

Previously I’ve written a series of posts on adding security to CruiseControl.Net. In these posts I talked about what I had done to secure the server and CCTray. This included authentication (identifying the user) and authorisation (what the user can do). However I had left the web dashboard untouched as I didn’t understand it.

Since then I’ve done some investigation on how the dashboard works and what I could do to add some security (and posted on the learning). Now it’s time to combine the two!

Rather than just jumping in and doing it, this post is about my plan on how I’m going to do it (actually it’s more about my ideas on how to do it, but very similar).

Returning To CCTray

At the end of my first security posts I had security working with CCTray – as long as .Net remoting was used. While the server was locked down, it was not possible to pass authentication and authorisation details through via the dashboard. My first step will be to remedy this failing!

The actions for handling XML requests are a custom set of XML-based actions. These are always added to the ObjectStore (i.e. they are hard-coded in CruiseObjectSourceInitializer). Some new security XML-based actions will need to be added. Once these actions are in place I can then modify HttpCruiseServerManager and HttpCruiseProjectManager in CCTrayLib to make use of them.

Currently I’m planning on adding a single security action that will handle login/logoff. I’m also going to modify XmlServerReportAction to handle sessions, and then track down which action or actions handle force and aborting builds and stopping and starting projects. Since the .Net remoting changes are already in place, this should be a simple matter of just passing through the parameters and returning the responses.

Adding Authentication to the Dashboard

Authentication is a bit harder. First off, there are potentially mutliple authentication methods. Secondly the dashboard can monitor multiple different servers – which may have different authentication methods and will have their own sessions.

To keep things simply I’m initially only planning on modifying the dashboard so the security is for a single server, but I hope to do this in an extensible way. I’m going to add a login/logout button to the header – next to the documentation link. To decide which version to display I’m going to added a decorator to the plugins that will check for a session token.

Session tokens will be the hard part, but I’m going to build a framework for these. It will be a plug-in that is defined in dashboard.config. It will need to hook into the URL generator (hence why I covered it) and also the new decorator I’m adding. The idea is the decorator will be called as part of the request chain, this will call the session plugin to retrieve the actual session id and set it so it can be used downstream. Any URLs that are generated can then use this session token.

Finally, the login view will also need to be extensible (to handle the different authorisation options). Initially this will need to be set in the configuration, so the users will be forced to use an authentication method. However in the long term I’m aiming to make this more flexible so the users can choose which authentication method to use (like CCTray). So this will be another type of plugin, but with the ability to interact with the session plugin.

Authorising Actions

With authentication locked down, authorisation is fairly simple. This is a matter of modifying some actions to pass the session id onto the server. Initially I’ll need to modify the actions that handle forcing and aborting builds, plus stopping and starting a project.

Since the session tokens are already parsed and set, all I should need to do is retrieve the token and pass it onto the .Net remoting calls.

Wrapping Up

When I’ve finished all this I’ll be just about done (and ready for some testing). But to help make things easier I’m planning on writing at least one post on using the new security settings. If there is anything you’d like me to cover, please let me know and I’ll add it in.

Anyway, enough planning, let’s start implementing

Building with NVelocity

Finishing the Dashboard

In my previous articles I’ve covered a lot of the “core” functionality of the web dashboard – how it works and its pieces. As the end goal of the dashboard is to render HTML content, I’ve covered the various classes that contribute towards this goal – responses, templates and actions. But so far I haven’t covered the actual HTML generator itself – IVelocityViewGenerator. This post will look into the workings of this interface, both the classes that implement it and the process for actually generating the full HTML page that is sent to the client.

Wrapping the Generator

The actual HTML generator is the NVelocity engine – but this functionality is encapsulated in a series of classes and interfaces. Every action that generates HTML uses the IVelocityViewGenerator interface. This interface has a single method – GenerateView(). This method takes in a template name and a hashtable of context items and generates an HtmlFragmentResponse containing the HTML. The actual template only contains a fraction of the page – the generator is responsible for adding everything else.

The only implementation of IVelocityViewGenerator is VelocityViewGeneratorWithTransformer. This class is merely a wrapper around IVelocityTransformer, which is implemented by LazilyInitialisingVelocityTransformer. This class contains the reference to a VelocityEngine - the heart of the NVelocity library. As it’s name implies, it only loads the engine when required, and when it does it loads all the required settings (the location of the templates, logging and resource manager classes).

And that’s all the view generator actually does – it just hides the view engine. So, how is the actual page generated?

Back to Objection

Remember all the way back to my first post on the web dashboard? I talked about types like ObjectStore and CruiseObjectSourceInitializer - these actually do a bit more work behind the scenes. As part of the initialisation of an object in the store they do something called decorating them. These “decorations” are actually wrapper classes that contain the actual actions, and do various pieces of work around the actions. It can be thought of as building a chain of operations to perform for an action.

The following shows the chain of operations for a farm action (this is typical for any generic action):

Farm Chain of Operations

In the case of building the page HTML, there is a “decoration” called SiteTemplateActionDecorator. Like all actions, this implements the IAction interface, plus it contains a references to the wrapped action. When the RequestController instantiates an instance of an action, it actually gets a chain of actions including the SiteTemplateActionDecorator (assuming it is performing an action that generates HTML).

Building a Full Page

When the Execute() method on SiteTemplateActionDecorator is called it does a number of things:

1. Call Execute() on the wrapped action
2. Generates HTML fragments for the breadcrumbs, side bar, header and footers and stores them in an hashtable
3. Calls IVelocityViewGenerator on a site template to generate the final page

The breadcrumbs, side bar, header and footers are all small templates that contain just the required details. There are “builder” classes around these templates that decide which template should be rendered (e.g. farm, server, project or build), plus builds the hashtables with the context details. Most of these builders are simple – they just see what items are in the ICruiseRequest instance and then set the template and context items. They are a bit more involved than just an action as they need to handle all cases (e.g. the different levels within the dashboard).

And That’s All!

That pretty much wraps up this series of post on the web dashboard and how it works. While the dashboard is fairly complex, this complexity allows for a wide range of extensible functionality. This ability to extend the dashboard made it harder to figure out how it worked, but time, persistence and liberal use of the debugger in VS2008 helped to crack it open

If all that is needed is a new plug-in, just implement IPlugin and IAction (I’ll write up something on doing this at some point). If more is needed, then hopefully these posts will help. Now I know I haven’t covered every single detail of the dashboard, but it is enough to give me an understanding of how it works for my next project – adding security to the dashboard!

Stay tuned…

Requests, Actions and Templates

Previously

I’ve finally bitten the bullet and started pulling apart the dashboard in order to understand it. In my last post on the dashboard (here) I started looking at how it does its job. I looked at the HttpHandler, the RequestController, IActionFactory and the IResponses. While this was a general overview of the processing, there are several areas uncovered! E.g. how does the dashboard know which server/project is being used? What happens to the data in the ASP.Net request? How is NVelocity used? What are templates? And so on …

In this post I’ll look at what happens at the ASP.Net request and how the URL gets translated into project/server information. Then I’ll pull part a couple of actions and their associated templates. By this point I should know enough of the inner workings of dashboard to start modifying it!

Requests

Rather than using the standard Request class in ASP.Net, the web dashboard has its own version. To start with it has an interface (IRequest) and the implementation (NameValueCollectionRequest). I assume this has been done for testing (see the ASP.Net MVC blogs and how they have gotten around this same issue), but it also allows them to do some custom processing.

As part of SetupObjectSourceForRequest() in CruiseObjectSourceInitializer, the initial ASP.Net Request is stored in the ObjectSource, plus an instance of NameValueCollectionRequest with the relevant data from the Request. This includes the QueryString/Form values (stored together), the headers, path and URL. Looking through the code of this method, I also saw it sets another ‘request’ instance – this time ICruiseRequest/RequestWrappingCruiseRequest.

While IRequest is a more general ‘request’ interface, ICruiseRequest is specifically for dealing with CruiseControl.Net requests. This is a wrapper around IRequest and it has three properties of interest: ServerName, ProjectName and BuildName. This is the class that is responsible for breaking apart a URL to return the server, project and build information.

At this point I should explain the format of the URLs that the dashboard generate. They use a form of REST for the URLs, including the server, project and build names (hence why they had to implement an HttpHandler). For example, to view all the projects on a server the following URL is used:

http://ccnetlive.thoughtworks.com/ccnet/server/local/ViewServerReport.aspx

To view the builds for a project:

http://ccnetlive.thoughtworks.com/ccnet/server/local/project/CCNet/ViewProjectReport.aspx

To view a project build:

http://ccnetlive.thoughtworks.com/ccnet/server/local/project/CCNet/build/log20080919051543Lbuild.1.4.0.3690.xml/ViewBuildReport.aspx

The first part (http://ccnetlive.thoughtworks.com/ccnet/) is the server and applicaton folder (yes, I’ve used CCNetLive ). Next (server/local) is the server that is being viewed, followed by the project (project/CCNet), the build (build/log20080919051543Lbuild.1.4.0.3690.xml) and finally the page name. The page name (minus the extension) is the name of the action to perform. All names (server, project and build) are optional, but it is up to the action to handle any missing values.

In the URLs the name of an item is prefixed with the type of the item. This opens up possibilities later on for adding additional values to the URL (e.g. a session token). RequestWrappingCruiseRequest is the class that knows about these item types and how to handle them.

Generating URLs is handled by ILinkFactory/DefaultLinkFactory and a series of type-specific classes that implement IAbsoluteLink. These classes are:

• BuildLink
• ProjectLink
• ServerLink
• FarmLink

Internally these classes rely on ICruiseUrlBuilder/DefaultCruiseUrlBuilder to generate the actual URLs (yes, layer upon layer!) This simply goes through and appends server/project/build. If any of the previous items are missing (e.g. server or build) then the subsequent items are skipped.

That hopefully covers everything on requests and URLs.

Actions

For this post I’m going to dissect three different actions: ViewFarmReport, ViewBuildLog and ForceBuild. These three actions use different ways of achieving their goals.

ForceBuild

I’m going to start with ForceBuild, since this is the simplest of the three. The class the handles this action is ForceBuildXmlAction - it is one of the default actions that is registered in CruiseObjectSourceInitializer.SetupObjectSourceForRequest().

In its Execute() method it calls the ForceBuild() method on an IFarmService instance. This instance is passed in via the constructor and holds a link (via .Net remoting) to the actual CruiseServer. Once this has been done it generates an XML fragment containing the results – which is always success (there is no checking of whether the request was actually sent or not).

ViewBuildLog

Next is ViewBuildLog, which is a little more complex. This action is handled by HtmlBuildLogAction. This is a configurable, plug-in action (i.e. not hard-coded in CruiseObjectSourceInitializer.SetupObjectSourceForRequest().) Adding this action to the ObjectStore is the job of BuildLogBuildPlugin. This is an Exortech.NetReflector decorated class that simply registers HtmlBuildLogAction.

The Execute() method generates HTML from an NVelocity template. The templates are in the templates folder, and the dashboard has a number of classes for working with them. But to keep things simple, basically all the valves for the template are stored in a Hashtable. This hashtable and the name of the template are passed into an IVelocityViewGenerator instance (again from the constructor). This instance is then responsible for converting the template into HTML.

The actual template is a simple fragment of HTML with some NVelocity bookmarks:

<p>Click <a href="$logUrl">here</a> to open log in its own page</p>
<pre class="log">$log</pre>

The items that start with a dollar sign ($) are the NVelocity bookmarks. These are the values that are set in HtmlBuildLogAction. The other point is this is only a fragment of the full document – this is the job of IVelocityViewGenerator and its supporting classes to turn it into a full HTML document (I’ll cover these some other time).

ViewFarmReport

Now I’ve left ViewFarmReport to last as it’s the most complex of the three actions I’m covering. The other two both performed their actions and directly returned responses. In contrast, this action uses IProjectGridAction to do its actual processing. But first, the class FarmReportFarmPlugin is both the ICruiseAction and also the plugin for this action, and it is loaded via the configuration file.

Currently the only implementation of IProjectGridAction is VelocityProjectGridAction. This generates a grid containing all the projects for either a server or the whole server farm. Internally it uses ProjectGrid and ProjectGridRow to achieve this. Again, it uses an NVelocity template to generate the actual HTML – although this time the template is more complex as it needs to iterate through an array. The values for the hashtable are set in VelocityProjectGridAction, including an array of ProjectGridRow entities.

Otherwise, this action is no more complex than the others

Until Next Time

Most actions are similar to these three, with varying levels of complexity.This covers (almost) the complete process from requesting an action to the final result. I now know how to get to an action, how it is loaded into the application and where NVelocity is used.

Now while understanding the basic workings of actions does tend to simplify things, I’ve left out how IVelocityViewGenerator works and how the other page content is generated. This includes the breadcrumbs across the top, the list of actions down the side and the builds list. But I’ve covered enough for now, I’ll leave these final details to my next post

A Challenging Bug – Revisited

Returning to the Bug

Last month I investigated a bug where the wrong server name was being displayed in CCTray (CCNET-1179 – read it here). Now I thought I’d found a nice solution to the problem, but it turns out that I had only covered part of the problem.

A quick recap – I had modified CCTray to store the project configuration so it could be used to retrieve the server name. While this fixed the problem for .Net Remoting connections, it didn’t properly fix it for HTTP connections

Now, this problem did correctly tell which server CCTray was connected to in order to retrieve the data. For .Net remoting it connects directly to the CruiseControl.Net process, so this information is correct. The dashboard however could add a layer of redirection – CCTray points to points to the dashboard, which could then be pointing to a different server! So, dashboard needs to be modified to return the remote server, so CCTray can then know which server it is really looking it!

Starting at the Root

Dahboard uses a class called CruiseXmlWriter to generate the actual data that is sent to the client. This class actually resides in the Core project, so it’s obviously a more generic class than just for the dashboard. This takes a CruiseServerSnapshot and writes the contents to an XML string. Digging through this class I found a method called WriteProjectStatus() which turns a ProjectStatus into XML – so it looks like the server name needs to be in this class.

Now this class is in the Remote project – and it’s a very core class! To keep things simple I added a new field called _serverName, which has a default property of Environment.MachineName. Since this is a field it will be serialised/deserialised over .Net remoting. I also added a property called ServerName to allow it to be retrieved and changed (just in case somebody wants to play with it later on). Since it has a default value of the machine name (which is the server name) I don’t need to do anything else in Core

Transportation and Display

.Net remoting automatically passes the server name, so this simplified part of the fix. The only transport I needed to add it to was the dashboard XML file. This was merely a matter of modifying CruiseXmlWriter to add it to the XML string.

At the other end, on CCTray a class called DashboardXmlParser converts the XML back into a ProjectStatus via the XML serialisation attributes. So I modified DashboardProject (the class that represents the XML) and DashboardXmlParser to first recognise the XML attribute and secondly populate the ProjectStatus. Again nice and simple.

The final step was to modify ProjectStatusListViewItemAdaptor to use the new property instead of the value from the configuration. Since ServerName is handled both via .Net Remoting and the dashboard there’s no need to do any fancy hacking! And it’s an even simpler solution than my original fix

What About the Original Fix?

Even though my final fix is a simpler solution, my original fix does still have value. If the remote dashboard hasn’t been updated, then the server will be blank. If this is the case I’m using the values from the configuration so people can at least see that the project has come from a dashboard, and thus the actual server is lost somewhere in translation!

But Wait! There’s More!

There’s also another issue in CCTray around the projects – if someone goes into settings and checks/un-checks a project that is on multiple servers, then all instances are affected.

This is a variation of the original problem – the code is only checking the project name instead of all the details. This fix was even simplier – the ProjectConfigurationListViewItemAdaptor for the project is stored in the Tag property of the item, and this contains a reference to the original project. So when the item changes all I needed to do is set the ShowProject property of the project using the values in the Tag property on the item.

And that hopefully fixes all the issues with duplicate project names on multiple servers!