I have an interesting problem with SvnBridge.

After around 5000 full revision request (a set of requests that can occur), the application get hung making a web service call to TFS. This comes after making quite a few calls to TFS, and is generally fairly easily reproducible. The actual call being made is not an expensive one (nor is it the same call). TFS is responsive during that time, so it is not its fault.

It looks very much like I am hitting the 2 concurrent HTTP requests, except that all requests are serialized, and there is no multi threaded work involved.

I have been unable to reproduce this under a profiler or debugger...

Thoughts?

Glenn posted about a test I wrote PrismShouldNotReferenceUnity, in which I codified an assumption that the team made.

This is something that I try to do whenever I decide that I will have some sort of convention. If at all possible, I will try to make sure that the compiler will break if you don't follow the convention, but that is often just not possible, therefor, tests and active enforcement of the system convention fill that place.

Wait a minute, I haven't even defined what a convention is! By convention, I means things like:

• All services should have a Dispose() method - add a Dispose abstract method to AbstractService
• All messages should be serializable - create a test that scan the message assemblies and check for non serializable messages.
• You may only call the database twice per web request - create an http module that will throw an exception if you call it more than that
• A request should take less than 100 milliseconds - add an interceptor that would fail if this is not the case.
• The interface assembly may not contain logic - add a test that would fail if it find a class with a method on the interface assembly.

All of those are ways to increase the feedback speed. This is especially true if there is some extra step that you need to perform, or you want to draw a line in the sand, from which you will not deviate.

Actively enforced conventions keep you honest.

I am using SVK as a way to learn Distributed SCM, and it has been fine so far, except for one big issue. I am trying to apply SVN patches against SVK repositories. Actually, I setup SVK so I work against a local mirror, so the workflow is exactly the same, but the revision numbers are different. Trying to apply a patch just doesn't work because of that.

You can go into the patch and edit the root revision, so it is not hard to make this work, except...

SVK doesn't give me any mapping between the local revision and the source revisions (maybe it does, and someone will tell me). This means that in order to find the right version I have to do some annoying digging.

Here is a small program to solve the problem. Usage: svkrev [repository path] [source revision to look for]

Example:

C:\Users\Administrator>svkrev //mirror/nhcontrib 195
exec: svk log --xml //mirror/nhcontrib
exec: svk propget svm:headrev //mirror/nhcontrib --revprop -r 10298
exec: svk propget svm:headrev //mirror/nhcontrib --revprop -r 10260
:=349
exec: svk propget svm:headrev //mirror/nhcontrib --revprop -r 10092
:=195
Found: 10092

By the way, can you find the huge perf boost here?

class Program
{
    static void Main(string[] args)
    {
        string path = args[0]; // "//mirror/nhcontrib"
        var revisionToSearchFor = int.Parse(args[1]); // 195
        string logAsXml = ExecuteProcessAndReturnOutput("log --xml "+ path);
        var doc = new XmlDocument();
        doc.LoadXml(logAsXml);
        var list = new ArrayList();
        foreach (var node in doc.SelectNodes("/log/logentry/@revision"))
        {
            list.Add(node);
        }
        for (int i = 0; i < list.Count;i++ )
        {
            XmlNode node = (XmlNode)list[i];
            int revision = int.Parse(node.Value);
            string output = ExecuteProcessAndReturnOutput("propget svm:headrev " + path + " --revprop -r " + revision);
            if (string.IsNullOrEmpty(output.Trim()) == false)
            {
                var currentRevision = int.Parse(output.Split(':')[1].Trim());
                Console.WriteLine(":=" + currentRevision);
                if (currentRevision == revisionToSearchFor)
                {
                    Console.WriteLine("Found: " + revision);
                    return;
                }
                i += (currentRevision - revisionToSearchFor -1);
            }
        }
    }

    public static string ExecuteProcessAndReturnOutput(string args)
    {
        var output = new StringBuilder();
        string err = null;
        var psi = new ProcessStartInfo
                      {
                          FileName = @"c:\Program Files\svk\svk.bat",
                          Arguments = args,
                          RedirectStandardOutput = true,
                          RedirectStandardError = true,
                          CreateNoWindow = true,
                          UseShellExecute = false
                      };
        Console.WriteLine("exec: svk "+ args);
        Process svk = Process.Start(psi);
        ThreadPool.QueueUserWorkItem(delegate
        {
            err = svk.StandardError.ReadToEnd();
        });
        ThreadPool.QueueUserWorkItem(delegate
        {
            string line;
            while ((line = svk.StandardOutput.ReadLine()) != null)
            {
                Debug.WriteLine(line);
                output.AppendLine(line);
            }
        });
        svk.WaitForExit();
        if (string.IsNullOrEmpty(err) == false)
            throw new InvalidOperationException(err);
        svk.WaitForExit();
        Thread.Sleep(500);
        return output.ToString();
    }
}

Here is something that most people just don't think about, but it hits some people at the high end. Rhino Mocks is extremely efficient mocking framework, for just about any scenario. However, when it comes the time to mock large amount of interfaces (where large is in the hundreds to thousands), or if you have large interfaces (with dozens or hundreds of methods), it is costly to create the mocks. Rhino Mocks does some caching internally, but it is noticeable issue to some people.

We are talking about an extra second or two for even most pathological cases here, so in general it wouldn't warrant anything special from the library, except that there is a twist to that. Under the debugger, the amount of time spent actually generating the mock can go up by several orders of magnitudes. To add insult to injury, Visual Studio 2005 has several bugs in the debugger that will cause it to crush & burn under several such scenarios.

Karl Lew has been kind enough to not only provide a patch for Rhino Mocks to help this problem, but also document it.

Thanks, and sorry for taking so long to get to it.

I was very impressed when I saw how Subversion handles the complexity of having data of hierarchical nature that needs to be serialized to XML. Check this out.

<?xml version="1.0" encoding="utf-8"?>
<S:editor-report xmlns:S="svn:">
  <S:target-revision rev="11"/>
  <S:open-root rev="-1"/>
  <S:open-directory name="tags" rev="-1"/>
  <S:add-directory name="tags/asd"/>
  <S:close-directory/>
  <S:close-directory/>
  <S:close-directory/>
</S:editor-report>

And here is another one:

<?xml version="1.0" encoding="utf-8"?>
<S:editor-report xmlns:S="svn:">
  <S:target-revision rev="15"/>
  <S:open-root rev="-1"/>
  <S:open-directory name="trunk" rev="-1"/>
  <S:open-file name="trunk/a.txt" rev="-1"/>
  <S:apply-textdelta checksum="eabc96676e7defda414a1eed33bdfb09">
    U1ZOAAAQEwETk2FzZDENCjINCjMNCjQNCjUNCjY=
  </S:apply-textdelta>
  <S:close-file checksum="c6301e5dad1330a7b9bd5491702c801b"/>
  <S:close-directory/>
  <S:close-directory/>
</S:editor-report>

I was, as they say incredibly happy with this Work Time Fun.

About a year and a half ago, I start an exciting new project (there is a demo of the actual project here). The actual application is fairly complex, and has some it gave me the chance to explore some very interesting ideas. Rhino Security is a generalization of the security scheme used in this project, and it is pretty much the driving force for Rhino Igloo. But that is not what I want to talk about.

What I do want to talk about is the infrastructure that we used for the project. We used IoC, OR/M, AoP, MVC and many other buzz worthy TLD. It was the first time that I had the chance in implementing real high end complexity reduction techniques. I left the team 10 months ago. In the meantime, the application was under active development, got a few new team members and had two major releases.

I am really proud of that system.

A few weeks ago I got a phone call from the current team lead, asking me about the windsor.boo file that is sitting there. The last time anyone touched it was shortly after I left, after which, it just... existed. I had the chance to do a code review on the new stuff that the team developed, about three months ago. I couldn't find any real difference between the code develop before and after I left.

Anyway, I had to spend 15 minutes on the phone, explaining the process that was going on there. Before I left (and during the time I was the team lead), I made sure that I passed on all the knowledge that I had about the system, the design decisions and the overall picture. However, there was a period of nearly three months in which I forgot that we even had this infrastructure, because we hadn't have to deal with it for that time period. After I left...

• 9 months.
• 2 major releases.
• Zero issues with the infrastructure.

I asked the team lead what she thinks about that. Since it is her project now, and if she thinks that it was the right decision to make. She love the infrastructure, and wouldn't hear about using a lower end solution. Most of what we did was actually going over the file and explaining historical decisions, for that matter.

As an additional data point, I was able to look at a piece of code I have last seen over a year ago and figure out not only what it does, but the how and why of it with no ramp up time.

I consider this a success.

I have an interesting problem. I am currently working on adding sync support to SvnBridge. This mostly involves tracking down what SVN does and duplicating it myself. There isn't that many new code (I had to add a class and a method). I have nothing that I can _unit_ test. Oh, I could probably craft something, but I am reluctant to mock the world when what I want is to test the actual integration between the SVN client and the TFS server.

The problem is that I have no real way of testing this. Why do I mean by that? syncing is an operation that works over the entire repository, from start to finish. I can't perform an integration test, because that would take too long in most scenarios. I would need to setup a whole new TFS server per test. Not really a good solution, no matter how you turn the dice. And the other problem is that I need a rich set of situations to actually test this.

Right now I am driving that by going against the production CodePlex servers, and trying to see if I can get all the information from there. This is incredibly valuable, because it gives me access to a lot of source control practices that are out there, and expose a lot of false assumptions. But I can't write tests for those, because they are too big a scenario.

I can probably set an integration test that would execute against the production servers (they are publicly exposed, after all, so no issue there), but I am... less than thrill about having a single test that can potentially run for hours or days.

Right now I think that I am leaning toward partial fake of both the client and the server. That is, create a set of input values which, while not being the real world values, would still test that logic.

Considering that the actual interaction we are talking about take place over many requests, this is tricky, but possible.

Suggestions?

I was asked this a few days ago, should we spend time creating proper unit tests for our code?

The team in question already have a set of integration tests, but very few tests that qualify as unit tests.

My response was rambling and long, but it can be put down to the following statement: TDD is a design technique, not a testing technique. TDD, and especially test first, have the nice side affect of leaving tests as part of the process, which can be incredibly helpful when you are working with the code. But retrofitting tests? That tends to be a waste of time.

Writing a unit test before touching the code is absolutely the way to go, but going and adding unit tests, as a task of its own? I don't see the value in that.

If you have integration tests there, that tends to be good enough, and you will write unit tests when you change the code, so eventually you'll have enough unit tests ( eventually you will have enough unit tests on the hot spots ).

For some reason, it looks like it is a good day to be impressed by numbers, here it the most interesting one...

And here is the reason you don't want to use the file system as your database:

A while ago I posted how to handle dynamic mapping with Active Record, it was incredibly easy to do, because Active Record has a lot of smarts internally, and output the XML, on top of which NHibernate adds quite a bit of convention over configuration as well. Doing the same using NHibernate directly is possible, but a bit long winded. Here is the sample code, which link all the Employee properties to the correct entity:

Configuration cfg = new Configuration()
    .AddAssembly(typeof (Employee).Assembly)
    .AddAssembly(typeof(ScheduledTask).Assembly);
Mappings mappings = cfg.CreateMappings();
foreach (PersistentClass persistentClass in mappings.Classes)
{
    if (persistentClass.MappedClass.GetProperty("Employee") == null)
        continue;
    Property prop = new Property();
    PersistentClass employeeClass = cfg.GetClassMapping(typeof (Employee));
    Table table = employeeClass.Table;
    ManyToOne value = new ManyToOne(table);
    value.ReferencedEntityName = typeof (Employee).FullName;
    Column column = new Column("Employee");
    value.AddColumn(column);
    prop.Value = value;
    prop.Name = "Employee";
    prop.PersistentClass = employeeClass;
    persistentClass.AddProperty(prop);
    persistentClass.Table.AddColumn(column);
    persistentClass.Table.CreateForeignKey("FK_EmployeeTo" + persistentClass.MappedClass.Name,
                                           new Column[] {column,}, typeof (Employee).FullName);
}
cfg.BuildSessionFactory();
new SchemaExport(cfg).Execute(true, true, false, true);

As you can see, there is a lot that needs to be done, we have to tell NHibernate a lot of things it would generally be able to figure out on its own. We can shove this to an extension method and get really nice syntax:

public static void MapManyToOne<TEntityInterface, TEntity>(this Configuration cfg)
{
    Mappings mappings = cfg.CreateMappings();
    foreach (PersistentClass persistentClass in mappings.Classes)
    {
        var propertyNames = new List<string>();
        foreach (PropertyInfo property in persistentClass.MappedClass.GetProperties())
        {
            if (property.PropertyType == typeof (TEntityInterface))
            {
                propertyNames.Add(property.Name);
            }
        }
        if (propertyNames.Count == 0)
            continue; 

        var prop = new Property();
        PersistentClass targetClass = cfg.GetClassMapping(typeof (TEntity)); 

        foreach (string propertyName in propertyNames)
        {
            Table table = targetClass.Table;
            var value = new ManyToOne(table);
            value.ReferencedEntityName = typeof (TEntity).FullName;
            var column = new Column(propertyName);
            value.AddColumn(column);
            prop.Value = value;
            prop.Name = propertyName;
            prop.PersistentClass = targetClass;
            persistentClass.AddProperty(prop);
            persistentClass.Table.AddColumn(column);
            string fkName = string.Format("FK_{0}To{1}", propertyName, persistentClass.MappedClass.Name);
            persistentClass.Table.CreateForeignKey(fkName,
                                                   new[] {column,}, typeof (TEntity).FullName);
        }
    }
}

Now we can use this with the following syntax:

cfg.MapManyToOne<IEmployee, Employee>();

Which is much nicer.