The Idea
This is another idea that I am playing around with under the FastForward.NET banner – a cache service. Before I go into how I am implementing the cache, first lets look at how clients communicate with a CruiseControl.NET server.
Some Background
A CruiseControl.NET (CC.NET) server only natively supports one protocol – .NET Remoting. So, in order to connect to the server a client must use .NET Remoting.
But, you say, what about HTTP? Yes, it appears that CC.NET has an HTTP protocol, but this is actually done via the web dashboard. This is a separate application that runs in a web server, which connects to the actual CC.NET server via .NET Remoting.
Additionally, with CC.NET 1.4.4 or later, it is possible to add a custom extension that adds a new protocol – not that I am aware of anyone doing this yet.
So, the following options are available:
Often the web dashboard and the CC.NET server will be on the same machine, but there is no requirement for this. Additionally, a single dashboard can be the front-end for multiple CC.NET servers – providing more functionality and more complexity to the mix!
How, if all of these are on a single network, which is reasonably fast, this infrastructure is not a problem. But add a bit of network lag, or a large number of servers, and suddenly things will slow right down.
One Scenario
To show what I am proposing as a solution, I want to look at a reasonably simple scenario. In this scenario we have five CC.NET servers that need to be monitored. These are all monitored by a single dashboard, and the client then connects to the dashboard to see the project details:
(Yes, I did choose FastForward.NET Monitor as the client 
)
Whenever the client wants to get the status of the servers, it will send a request to the dashboard. The dashboard then sequentially goes through and sends a request to each of the servers, then finally send the response to the client. Now imagine there is a 0.25s lag in connecting to each server – it would take 3s just to get one response back to the client (2.5s for the dashboard to query all the servers, plus the lag between the client and the dashboard)!
Now, expand that out slightly – imagine three clients:
The dashboard does not cache any results – so every time a client wants to refresh the status it has to go through the process of querying each server. So, with the default polling interval for FastForward.NET (and CCTray as well), this means the dashboard is going to be asking at least one of the CC.NET servers for its status at any point in time!
Now a .25s lag might seam like a lot, but over an Internet connection this is not at all unreasonable (actually, with some of the dashboards I connect to, a 0.25s lag would be very quick!)
The problem in this scenario is the dashboard just acts as a pass-through layer – it doesn’t provide any value on its own. And that’s just for status upgrades – there are also all the other items that CC.NET provides (e.g. build logs, RSS feeds, etc.)
Instead, what I am thinking of is to replace the dashboard as the intermediate server. Instead there would be the FastForward.NET cache service:
This looks very similar to the previous picture, but with one major difference – the cache service would have a local database.
Now, if we open up the hood of the cache service, it will work something like this:
When a client wants to request some data, the request comes in over a client connection (1). The client connection receives the request and checks the local database to see if the data is there (2). If the data is there, then this data is used (3). Otherwise a request is sent to the server connections for the data (4). This request gets passed on the server (or servers) (5). When the data is received it is stored in the database (6) as well as being passed back to the client.
This would be the standard model for most requests – for status requests it would be possible to have an additional disconnect:
In this model, there is only the client connection loop (1-3) – if there is no data in the local database then no status information will be returned to the client. Instead, the server connections will be constantly polling the CC.NET servers (a) and then storing the status information in the local database (b). This means when the client needs the information, they just use the local data – saves having to query one or more servers for their status.
Limitations
Of course, this model is not without its limitations. First, this would only work for retrieving data – action requests (force/abort build, start/stop project, etc.) would pass through directly to the CC.NET servers.
Second, this does not take into account security – all secure requests would again pass right-through (especially if they are encrypted).
Third, it means the data the client is getting can potentially be very out-of-date. For some data this is not an issue – e.g. build logs are only written once and then never change. Other data could be potentially very wrong – e.g. the project statuses.
Any Comments
I have already started working on a service to do this – although at the moment it is very experimental. However I do think this could have a lot of value, especially for large installations and even more so for those with slow networks (or slow Internet connections).
Does anyone have any comments?
