I wrote a test for this piece code:

public virtual void View([ARFetch("id")]Webcast webcast)
{
    PropertyBag["webcast"] = webcast;
}

Now, this looks like utter lunacy, isn't it? Especially when the test looks like:

[Test]
public void When_view_called_will_send_webcast_to_view()
{
	var webcast = new Webcast();
	controller.View(webcast);

	controller.PropertyBag["webcast"].ShouldBeTheSameAs(webcast);
}

Except that now that I have covered the very basic item, I now have another few tests:

[Test]
public void When_view_called_with_null_webcast_will_render_missing_webcast_view()
{
	controller.View(null);

	controller.SelectedViewName.ShouldEqual(@"Webcast\Missing");
}

[Test]
public void When_view_called_with_unpublished_webcast_will_render_unpublished_webcast_view()
{
	controller.View(new Webcast());

	controller.SelectedViewName.ShouldEqual(@"Webcast\Unpublished");
}

And the simplest thing that can make those test pass is:

public virtual void View([ARFetch("id")]Webcast webcast)
{
	if(webcast==null)
	{
		RenderView("Missing");
		return;
	}
	if(webcast.PublishDate==null)
	{
		RenderView("Unpublished");
		return;
	}
        PropertyBag["webcast"] = webcast;
}

By having tests from the beginning, even for trivial piece of code, I ensured both that the trivial case will work and that I am already starting with the right tone.

I am trying to get BDD. As such, I started with building a list of specifications. Here is what I have so far. Now you get the chance to tell me why I am wrong...

As a user
I want to be able to see the latest webcast summary on the homepage
So that I have great visibility on what is new
-------------------
Acceptance criteria:
- Can see the name of the latest webcast
- Can see the date it was published
- Can read the short description
===============================================================
As a user
I want to be able to see the details of a webcast
So that I can see more details about the webcast
-------------------
Acceptance criteria:
- Can see the name of the latest webcast
- Can read the short description
- Can view number of downloads
- Can view relevant files
===============================================================
As a user
I want to be able to browse the archives of the site
So I can see old shows
-------------------
Acceptance criteria:
- Can see the names of all published webcasts
- Can click to the details of each webcast
- Can view number of downloads
- Can sort the list
===============================================================
As a user
I want to be able to subscribe to an RSS feed of the webcasts
So that I wouldn't have to check the site all the time
-------------------
Acceptance criteria:
- Each page on the site should have RSS link for news
- Publishing a new webcast would appear on the RSS
===============================================================
As an administor
I want to be able to add a new webcast
So that I can publish new webcasts
-------------------
Acceptance criteria:
- Can get to admin part of the site
- Can add a new webcast
- Will go through validation
- Can use rich text editor for the description
- Can set publish date
===============================================================
As an administrator
I want to be able to edit the details of an existing webcast
So that I can fix typos and change the data if I need to
-------------------
Acceptance criteria:
- Requires login
- Goes through validation
- Will update the RSS feed
===============================================================
As an administrator
I want to be able to login to the site
So that I can perform administrative functions
-------------------
Acceptance criteria:
- Requires login
- Unauthenticated users cannot login
- Non administrators cannot login
- Passwords are not kept in the clear
===============================================================

Take a look at this test:

[Test]
public void When_asking_for_latest_webcast_will_not_consider_webcasts_published_in_the_future()
{
	var webcast = new Webcast { Name = "test", PublishDate = DateTime.Now.AddDays(-2) };
	With.Transaction(() => webcastRepository.Save(webcast));
	var webcast2 = new Webcast { Name = "test", PublishDate = DateTime.Now.AddDays(2) };
	With.Transaction(() => webcastRepository.Save(webcast2));
	Assert.AreEqual(webcast.Id, webcastRepository.GetLatest().Id);
}

It looks like a valid test, doesn't it? It has a huge problem. It is brittle.

I just added a new non null property, and all the tests broke. I started to add the new property value to the test, before I realize what I was doing. I run into a friction point, and I was trying to cover it with code. Next time I would add such a property, I would run into the same problem.

This is unacceptable.

The standard solution for this is to create a factory for this, or use an Object Mother. This was never something that I was fond of. I always need more flexibility than I can usually get from it, and I hate building builders, that is so boring.

Turn out, I can eat the cake and keep it.

I created the following test class:

[ActiveRecord("Webcasts")]
public class TestableWebcast : Webcast
{
	public TestabbleWebcast()
	{
		Name = "Test name";
		Description = "Test description";
	}
}

And now the test change to:

[Test]
public void When_asking_for_latest_webcast_will_not_consider_webcasts_published_in_the_future()
{
	var webcast = new TestableWebcast { PublishDate = DateTime.Now.AddDays(-2) };
	With.Transaction(() => webcastRepository.Save(webcast));
	var webcast2 = new TestableWebcast { PublishDate = DateTime.Now.AddDays(2) };
	With.Transaction(() => webcastRepository.Save(webcast2));
	Assert.AreEqual(webcast.Id, webcastRepository.GetLatest().Id);
}

I get to keep the nice object initializer syntax, and I even get more clarity, since I can now specify only the properties that I am actually interested in.

The only annoying thing is that I have to define the TestableWebcast as an entity as well, but I can live with it.

I have been waiting on the release for Rhino Mocks 3.5, seeing what kind of feedback I can get from users. I also had another reason, I hadn't had the chance to really give it a good testing, in the only manner that matter, using it to develop production ready software.

Here is my latest test, which I am fairly pleased with.

[TestFixture]
public class WebcastControllerTest
{ 
	private WebcastController controller;
	private IRepository<Webcast> repositoryStub;
	private StubEngineContext engineContext;
	private IUnitOfWork unitOfWorkStub;
	private IDisposable disposeGlobalUnitOfWorkRegistration;

	[SetUp]
	public void Setup()
	{
		repositoryStub = MockRepository.GenerateStub<IRepository<Webcast>>();
		unitOfWorkStub = MockRepository.GenerateStub<IUnitOfWork>();
		controller = new WebcastController(repositoryStub)
		             	{
		             		ControllerContext = new ControllerContext{Name = "webcast"},
		             	};
		engineContext = new StubEngineContext();
		controller.SetEngineContext(engineContext);
		disposeGlobalUnitOfWorkRegistration = UnitOfWork.RegisterGlobalUnitOfWork(unitOfWorkStub);
	}

	[TearDown]
	public void TearDown()
	{
		disposeGlobalUnitOfWorkRegistration.Dispose();
	}

	[Test]
	public void When_index_called_will_send_existing_webcasts_to_view()
	{
		var webcasts = new List<Webcast>();
		repositoryStub.Stub(x => x.FindAll()).Return(webcasts);

		controller.Index();

		Assert.AreSame(webcasts, controller.PropertyBag["webcasts"]);
	}

	[Test]
	public void When_display_called_will_send_webcast_to_view()
	{
		var webcast = new Webcast();
		controller.Display(webcast);

		Assert.AreSame(webcast, controller.PropertyBag["webcast"]);
	}

	[Test]
	public void When_edit_called_with_webcast_will_send_webcast_to_view()
	{
		var webcast = new Webcast();
		controller.Edit(webcast);

		Assert.AreSame(webcast, controller.PropertyBag["webcast"]);
	}

	[Test]
	public void When_saving_invalid_webcast_will_redirect_to_edit()
	{
		var webcast = new Webcast();

		controller.Validator.IsValid(webcast);
		controller.PopulateValidatorErrorSummary(webcast, controller.Validator.GetErrorSummary(webcast));

		controller.Save(webcast);

		Assert.Contains(engineContext.MockResponse.RedirectedTo, "edit");
	}

	[Test]
	public void When_saving_invalid_webcast_will_flash_error_summary_and_webcast_in_flash()
	{
		var webcast = new Webcast();

		controller.Validator.IsValid(webcast);
		controller.PopulateValidatorErrorSummary(webcast, controller.Validator.GetErrorSummary(webcast));

		controller.Save(webcast);

		Assert.AreSame(webcast, controller.Flash["webcast"]);
		Assert.AreSame(controller.Validator.GetErrorSummary(webcast), controller.Flash["errorSummary"]);
	}

	[Test]
	public void When_saving_valid_webcast_will_redirect_to_display()
	{
		var webcast = new Webcast();

		// explicitly not calling validation, the controller will assume 
		// that this is a valid object

		controller.Save(webcast);

		Assert.Contains(engineContext.MockResponse.RedirectedTo, "display");
	}

	[Test]
	public void When_saving_valid_webcast_will_save_and_flash_unit_of_work()
	{
		var webcast = new Webcast();

		// explicitly not calling validation, the controller will assume 
		// that this is a valid object

		controller.Save(webcast);

		repositoryStub.AssertWasCalled(x=>x.Save(webcast));
		unitOfWorkStub.AssertWasCalled(x=>x.TransactionalFlush());
	}
}

Two days ago I asked how many tests this method need:

///<summary> 
///Get the latest published webcast 
///</summary>
public Webcast GetLatest();

Here is what I came up with:

[TestFixture]
public class WebcastRepositoryTest : DatabaseTestFixtureBase
{
	private IWebcastRepository webcastRepository;

	[TestFixtureSetUp]
	public void TestFixtureSetup()
	{
		IntializeNHibernateAndIoC(PersistenceFramework.ActiveRecord, 
			"windsor.boo", MappingInfo.FromAssemblyContaining<Webcast>());
	}

	[SetUp]
	public void Setup()
	{
		CurrentContext.CreateUnitOfWork();
		webcastRepository = IoC.Resolve<IWebcastRepository>();
	}

	[TearDown]
	public void Teardown()
	{
		CurrentContext.DisposeUnitOfWork();
	}

	[Test]
	public void Can_save_webcast()
	{
		var webcast = new Webcast { Name = "test", PublishDate = null };
		With.Transaction(() => webcastRepository.Save(webcast));
		Assert.AreNotEqual(0, webcast.Id);
	}

	[Test]
	public void Can_load_webcast()
	{
		var webcast = new Webcast { Name = "test", PublishDate = null };
		With.Transaction(() => webcastRepository.Save(webcast));
		UnitOfWork.CurrentSession.Evict(webcast);

		var webcast2 = webcastRepository.Get(webcast.Id);
		Assert.AreEqual(webcast.Id, webcast2.Id);
		Assert.AreEqual("test", webcast2.Name);
		Assert.IsNull(webcast2.PublishDate);
	}

	[Test]
	public void When_asking_for_latest_webcast_will_not_consider_any_that_is_not_published()
	{
		var webcast = new Webcast { Name = "test", PublishDate = null };
		With.Transaction(() => webcastRepository.Save(webcast));

		Assert.IsNull(webcastRepository.GetLatest());
	}

	[Test]
	public void When_asking_for_latest_webcast_will_get_published_webcast()
	{
		var webcast = new Webcast { Name = "test", PublishDate = null };
		With.Transaction(() => webcastRepository.Save(webcast));
		var webcast2 = new Webcast { Name = "test", PublishDate = DateTime.Now.AddDays(-1) };
		With.Transaction(() => webcastRepository.Save(webcast2));

		Assert.AreEqual(webcast2.Id, webcastRepository.GetLatest().Id);
	}

	[Test]
	public void When_asking_for_latest_webcast_will_get_the_latest_webcast()
	{
		var webcast = new Webcast { Name = "test", PublishDate = DateTime.Now.AddDays(-2) };
		With.Transaction(() => webcastRepository.Save(webcast));
		var webcast2 = new Webcast { Name = "test", PublishDate = DateTime.Now.AddDays(-1) };
		With.Transaction(() => webcastRepository.Save(webcast2));

		Assert.AreEqual(webcast2.Id, webcastRepository.GetLatest().Id);
	}

	[Test]
	public void When_asking_for_latest_webcast_will_not_consider_webcasts_published_in_the_future()
	{
		var webcast = new Webcast { Name = "test", PublishDate = DateTime.Now.AddDays(-2) };
		With.Transaction(() => webcastRepository.Save(webcast));
		var webcast2 = new Webcast { Name = "test", PublishDate = DateTime.Now.AddDays(2) };
		With.Transaction(() => webcastRepository.Save(webcast2));
		Assert.AreEqual(webcast.Id, webcastRepository.GetLatest().Id);
	}
}

And the implementation:

public class WebcastRepository : RepositoryDecorator<Webcast>, IWebcastRepository
{
	public WebcastRepository(IRepository<Webcast> repository)
	{
		Inner = repository;
	}

	public Webcast GetLatest()
	{
		var publishedWebcastsByDateDesc =
			from webcast in Webcasts
			where webcast.PublishDate != null && webcast.PublishDate < SystemTime.Now()
			orderby webcast.PublishDate descending 
			select webcast;

		return publishedWebcastsByDateDesc.FirstOrDefault();
	}

	private static IOrderedQueryable<Webcast> Webcasts
	{
		get { return UnitOfWork.CurrentSession.Linq<Webcast>(); }
	}
}

I think it is pretty sweet.

Here is an interesting question, how many tests do you need for the following method?

///<summary> 
///Get the latest published webcast 
///</summary>
public Webcast GetLatest();

Here is how the Webcast entity looks like:

Any thoughts?

Just got a rather pointed reminder why you should try to get as many tests as possible. The code is:

public override void Execute()
{
	Version version = typeof (VersionCommand).Assembly.GetName().Version;
	Writer.WriteLine("VERSION " + version);
}

And the test is:

[Test]
public void Will_return_assembly_version_as_memcached_version()
{
	var stream = new MemoryStream();
	var cmd = new VersionCommand(stream);
	cmd.Execute();
	Assert.AreEqual("VERSION " + typeof(VersionCommand).Assembly.GetName().Version 
		+ "\r\n", ReadAll(stream));
}

Now, tell me why this is not a stupid test & useless test.

I am considering having a language that mandates tests. If you don't have a matching test for the code in question, it will refuse to run. If the tests fail, it will refuse to run. If the tests takes too long, they are considered failed and the code will refuse to run.

This certainly ensure that there would be test. It wouldn't ensure that they would be meaningful, however. That is fine by me. I am not interested in policy through enforcement, just gentle encouragement in the right direction.

The technical challenges of implementing such a system are nil. The implications on the workflow and ease of use for such a system are unknown. On the surface, checked exceptions are great. In practice, they are very cumbersome. This is why I am warning that I have only toyed with the idea, not implemented it.

Thoughts?

Roughly speaking, a DSL is composed of the following parts:

It should come as no surprise that when we test it, we test each of those components individually. When the time comes to test a DSL, I have the following tests:

• CanCompile - This is the most trivial test, it assert that I can take a known script and compile it.
• Syntax tests - Didn’t we just test that when we wrote the CanCompile() test? When I am talking about testing the syntax I am not talking about just verifying that it can compile successfully. I am talking about whatever the syntax that we have created has been compiled into the correct output. The CanCompile() test is only the first step in that direction. Here is an example of such a test.
• DSL API tests - What exactly is the DSL API? In general, I think about the DSL API as any API that is directly exposed to the DSL. The methods and properties of the anonymous base class is an obvious candidate, of course. Anything else that was purposefully built to be used by the DSL also fall into this category. Those I test using standard unit tests, without involving the DSL at all. Testing in isolation again.
• Engine tests - A DSL engine is the responsible for managing the interactions between the application and the DSL scripts. It is the gateway to the DSL in our application, allowing us to shell out policy decisions and oft-changed rules to an external entity. Since the engine is usually just a consumer of the DSL instances, we have several choices when the time comes to create test cases for the engine. We can perform a cross cutting test, which would involve the actual DSL, or test just the interaction of the engine with the provided instances. Since we generally want to test the engine behavior in invalid scenarios (a DSL script which cannot be compiled, for example), I tend to choose the first approach.

Testing the scripts

We have talked about how we can create tests for our DSL implementation, but we still haven’t talked about how we can actually test the DSL scripts themselves. Considering the typical scenarios for using a DSL (providing a policy, defining rules, making decisions, driving the application, etc), I don’t think anyone can argue against the need to have tests in place to verify that we actually do what we think we do.

In fact, because we usually use DSL as a way to define high level application behavior, there is an absolute need to be aware of what it is doing, and protect ourselves from accidental changes.

One of the more important things to remember when dealing with Boo based DSL is that the output of those DSL is just IL. This means that this output is subject to all the standard advantages and disadvantages of all other IL based languages.In this specific case, it means that we can just reference the resulting assembly and perform something write a test case directly against it.

In most cases, however, we can safely utilize the anonymous base class as a way to test the behavior of the scripts that we build. This allows us to have a nearly no-cost approach to building our tests. Let us see how we can test this piece of code:

specification @vacations:
	requires @scheduling_work
	requires @external_connections

specification @scheduling_work:
	return # doesn't require anything

And we can test this with this code:

[Test]
public void WhenUsingVacations_SchedulingWork_And_ExternalConnections_AreRequired()
{
	QuoteGeneratorRule rule = dslFactory.Create<QuoteGeneratorRule>(
		@"Quotes/simple.boo",
		new RequirementsInformation(200, "vacations"));
	rule.Evaluate();

	SystemModule module = rule.Modules[0];
	Assert.AreEqual("vacations", module.Name);
	Assert.AreEqual(2, module.Requirements.Count);
	Assert.AreEqual("scheduling_work", module.Requirements[0]);
	Assert.AreEqual("external_connections", module.Requirements[1]);
}

Or we can utilize a test DSL to do the same:

script "quotes/simple.boo"

with @vacations:
	should_require @scheduling_work
	should_require @external_connections	

with @scheduling_work:
	should_have_no_requirements

Note that creating a test DSL is only worth it if you expect to have a large number of DSL scripts of the tested language that you want to test.

How do I test the syntax in this DSL? HandleWith should translate to a method call with typeof(RoutingTestHandler) and a delegate.

import BDSLiB.Tests

HandleWith RoutingTestHandler:
	lines = []
	return NewOrderMessage( 15,  "NewOrder", lines.ToArray(OrderLine) ) 

Well, I use interaction based testing, obviously. I find this test utterly fascinating, because it is fairly advance, in a roundabout sort of way, and yet it is so simple.

[Test]
public void WillCallHandlesWithWithRouteTestHanlderWhenRouteCalled()
{
	const IQuackFu msg = null;

	var mocks = new MockRepository();

	var routing = dslFactory.Create<RoutingBase>(@"Routing\simple.boo");

	var mockedRouting = (RoutingBase)mocks.PartialMock(routing.GetType());
	Expect.Call(() => mockedRouting.HandleWith(null, null))
		.Constraints(Is.Equal(typeof(RoutingTestHandler)), Is.Anything());

	mocks.ReplayAll();

	mockedRouting.Initialize(msg);

	mockedRouting.Route();

	mocks.VerifyAll();
}