<< Rhino Mocks 3.5 Design Decisions: To be strict or not? | Home | What hid under the bed... >>

Rhino Mocks 3.5 Design Decisions: The role of Stub vs. Mock

Let us say that we have the following code that we want to test:

public void ForgotMyPassword(string username)
{
   var user = users.GetUserByName(username);
   user.HashedPassword = "new pass";
   users.Save(user);
}

One way to write the test is:

public void When_user_forgot_password_should_save_user()
{
    var mockUserRepository = MockRepository.GenerateMock<IUserRepository>();
    var stubbedSmsSender = MockRepository.GenerateStub<ISmsSender>();

    var theUser = new User{HashedPassword = "this is not hashed password"};    

    mockUserRepository.Stub(x => x.GetUserByName("ayende")).Return(theUser);

    mockUserRepository.Expect( x => x.Save(theUser) );

    var controllerUnderTest = new LoginController(mockUserRepository, stubbedSmsSender);

    controllerUnderTest.ForgotMyPassword("ayende");

    mockUserRepository.VerifyAllExpectations();
}

Another would be:

public void When_user_forgot_password_should_reset_password()
{
    var stubUserRepository = MockRepository.GenerateStub<IUserRepository>();
    var stubbedSmsSender = MockRepository.GenerateStub<ISmsSender>();

    var theUser = new User{HashedPassword = "this is not hashed password"};    

    stubUserRepository.Stub(x => x.GetUserByName("ayende")).Return(theUser);

    var controllerUnderTest = new LoginController(mockUserRepository, stubbedSmsSender);

    controllerUnderTest.ForgotMyPassword("ayende");

    stubUserRepository.AssertWasCalled( x => x.Save(user));
}

Again, we have the question of should we even allow asserts on stubs. But beyond that, consider this two tests, both are functionally the same. Thoughts?

Print | posted on Sunday, June 29, 2008 4:55 AM

Feedback

Gravatar

# re: Rhino Mocks 3.5 Design Decisions: The role of Stub vs. Mock 6/29/2008 6:22 AM Jeremy Gray

I probably would have answered this differently before the 3.5 beta came out and I started using it regularly, but the more I've used the AAA syntax the more I've been tempted by the idea of allowing expectations to be set on both stubs and mocks, complete with ordered/unordered control, Repeat, etc. but with the primary difference being:

- the stubs would attempt to provide a reasonable response to all calls, throwing only when a call specifically violates one of their expectations. Unordered unless otherwise stated, with multiple calls matching a given expectation allowed unless specifically constrained via Repeat.

- the mocks would respond only as per their specific expectations, throwing on anything else. Ordered unless otherwise stated, with multiple calls matching a given expectation only allowed where specified via Repeat.

As much as this potential mental model has resonated better with me and my team than any other I've cooked up, I'm still quite curious how much it might match with that of other Rhino.Mocks users. (No use living in a vacuum, after all ;)

Gravatar

# re: Rhino Mocks 3.5 Design Decisions: The role of Stub vs. Mock 6/29/2008 6:37 AM Chad Myers

According to Fowler's "Marks aren't stubs" article, Mocks are expectations, Stubs are just containers for returning dummy data and receiving data from the test.

I would stay that Stubs should not allow for expectations at all. In fact, if possible, you might even remove the various expectation-style syntax from the fluent interface for the .Stub() extension method.

For example, this should not be valid:

authSvc.Stub(x=>x.SignOut()).Repeat.Once().

Stubs should always be repeat: 0 or infinity

Gravatar

# re: Rhino Mocks 3.5 Design Decisions: The role of Stub vs. Mock 6/29/2008 7:46 AM Jeremy Gray

I'm certainly not offended by that interpretation, and in fact still hold it when looking at things in the strictest sense, as opposed to looking at it in light of current Rhino.Mocks AAA behaviour.

If stubs were to be restricted in that way, however, I would like to make sure that mocks would still have both Stub and Expect (/AssertWasCalled/VerifyAllExpectations) available on them, and that "PropertyBehaviour"-style functionality could be applied to them when necessary, along with an equivalent for events. That and/or some kind of "Dynamic"-style variation which supported both Stub and Expect, so that I don't lose one of the nicest things I've found with the beta 3.5 AAA behaviour: the ability to set up common objects, more stub-like in their behaviour, and then selectively add expectations to them and verify those expectations in specific tests.

Gravatar

# re: Rhino Mocks 3.5 Design Decisions: The role of Stub vs. Mock 6/29/2008 7:48 AM Jeremy Gray

PS - with those common stub-like objects of course having common stubbed behaviour (which is then extended with either additional stubbed behaviour that is specific to a given test and/or expectations specific to a given test.)

Gravatar

# re: Rhino Mocks 3.5 Design Decisions: The role of Stub vs. Mock 6/29/2008 9:14 AM Miika Makinen

To me allowing asserts on stubs makes a whole lot more understandable code. This way you can clearly do

// Given (or Arrange)
...
// When (or Act)
..
// Then (or Assert)
...

I was always a bit confused with the mock expectations kinda belonging to two stages. To me asserting that "something has expected value" is similar activity to asserting "something has been called the expected way".

Gravatar

# re: Rhino Mocks 3.5 Design Decisions: The role of Stub vs. Mock 6/29/2008 12:43 PM Ferry

I consider the distinct functionality between mocks and stubs very clear. The only point I see is adding assertions to stubs is for convenience. Because after all you can do exactly the same with Mocks. That is somewhat more verbose tough, but it makes the intentions obvious.

Also I wonder whether it will be more confusing for beginners. Implementing this more or less intersecting functional boundaries will only add to the learning curve.

Gravatar

# re: Rhino Mocks 3.5 Design Decisions: The role of Stub vs. Mock 6/29/2008 4:50 PM Jeremy Gray

Note that the "somewhat more verbose" for "you can do exactly the same with Mocks" translates to some combination of putting PropertyBehaviour on them, hooking up their events, putting Repeat.Any() on all of the mocks' Stub() calls, etc. It gets rather ugly rather quickly.

I find myself torn, however. I too would like to have stubs have their nice, clear, commonly-understood definition. To do that, though, we could really do with three types, as eluded to in my earlier comment:

1. The stubs, attempting to respond reasonably to every call, with the ability to stub behaviour when called with specific inputs, always unordered, and with no support for expectations.

2. The mocks, only responding to specifically-configured calls (whether mocks should even allow .Stub(...) to be called on them becomes another question at this point), perhaps ordered by default, and definitely supporting expectations.

3. A hybrid, attempting to respond reasonably to every call on members without stubbed behaviour or expectations defined, only responding to calls on members with stubbed behaviour or expectations based on the the definitions of said stubbed behaviour and expectations, automatic property behaviour, automatic support event (un)subscription, with the ability to stub behaviour when called with specific inputs, perhaps unordered by default for automatic behaviour while ordered for expectations, with support for expectations.

This third one admittedly gets rather hard to describe. As I said before, I'm torn on this issue, as while I would like to keep the stub and mock definitions and behaviour clean and simple I find myself needing (or at least wanting) the hybrid more and more often, as it really helps share common stubbed behaviour across tests and helps avoid over-specified tests.

What a toughie!

Gravatar

# re: Rhino Mocks 3.5 Design Decisions: The role of Stub vs. Mock 6/29/2008 5:33 PM Nathan

I would like it if a stub corresponded to the behavior of a "loose" mock in Moq.

That seems to be what the example in this post is going towards.

Gravatar

# re: Rhino Mocks 3.5 Design Decisions: The role of Stub vs. Mock 6/29/2008 5:48 PM Jonathan Moffatt

I'm with Jeremy and Miika on this one.

To me

stubUserRepository.AssertWasCalled( x => x.Save(user));

reads beautifully. The alternative

mockUserRepository.VerifyAllExpectations();

is not nearly as nice as it requires looking through the setup portion of the test to determine what the expectations are.

I don't think it's a big deal combining mocks and stubs. RhinoMocks has had the DynamicMock for a while after all. The new 3.5 syntax is effectively giving us the equivalent by saying "start with a stub, and if there are parts of the stubbed object that you want to treat as mocks go ahead".

I like it.

Gravatar

# re: Rhino Mocks 3.5 Design Decisions: The role of Stub vs. Mock 6/29/2008 9:41 PM sam

This is off topic... but this method is a perfect example for a quesiton I've struggled with for a long time about unit test design.

Good unit test design is to have one Assert per method. But if I wanted to add an assert to test that the password actually changed... I'd have to duplicate a lot of code. Sometimes I extract to method, but then juggling all these methods in a class seems painfull.

What's the good solution to this? Would it be okay to have a class just called when_changinging_password? Just violate DRY?

Gravatar

# re: Rhino Mocks 3.5 Design Decisions: The role of Stub vs. Mock 6/29/2008 9:49 PM Ayende Rahien

sam,
The approach you suggest is called context in BDD.
It is considered a best practice , having a single class for each scenario

Gravatar

# re: Rhino Mocks 3.5 Design Decisions: The role of Stub vs. Mock 6/29/2008 11:29 PM Aaron Jensen

The objects themselves should not be classified as mocks or stubs. The methods on those objects are either stubbed or mocked (read: expecting to be called). It is perfectly reasonable for a single object to have both stubbed methods and a method that you expect to be called.

Creating the differentiation at the object level only confuses the issue. If I want the equivalent of a strict mock, I would probably want to test that nothing but stubbed methods were called (mock.AssertNothingButStubsCalled()) or something like that.

That said, I find very little use for strict mocks at all, they seem to unnecessarily couple the test to the implementation.

Gravatar

# re: Rhino Mocks 3.5 Design Decisions: The role of Stub vs. Mock 6/30/2008 1:56 AM Jeremy Gray

@Jonathan - I suspect that whatever the outcome, mocks will still be left with both the AssertWasCalled and VerifyAllExpectations available, with stubs having whatever Oren ends up deciding about stubs. ;)

@sam - I have historically been in the habit of extracting common test code, and still impress upon my team members to do it when appropriate, but a degree of caution is needed. Sometimes, over-extraction of common test code can make the test harder to quickly grok and somewhat more brittle. I tell the guys to extract the really ugly, really common bits, but to not get too abstract about the whole thing. That said, it is a balancing act that requires constant adjustment, that's for sure.

@Aaron - I think I may be starting to lean in your direction. The whole mock, stub, dynamic mock, partial mock arrangement was (and is) really hard for my current crew to get used to, and the same was true for the team from my previous engagement.

What might be nicest of all would be if there was one and only one call, perhaps something ala (Generate(Test))Double<T>(). This would produce a test double that starts off using whatever semantic (in terms of stubbed behaviour by default versus strict-mock-style throwing by default) Oren prefers the most, and could be switched into the other mode at any time before calls start being made against the object.

That way, the four types of doubles go away, replaced with _how_ you set the object up (in terms of Stub calls, Expect calls, and whether or not you later call AssertWasCalled or VerifyAllExpectations), and the automatic stubbing of behaviour would be controlled solely by making a call pushing the object in one direction or the other (eg. double.StubUnexpectedInteractions(), double.ThrowUnexpectedInteractions() or what have you, restricted to whichever of the two that would push the object away from the constructed default behaviour of the test double.)

As for which unexpected interaction behaviour to start with upon construction, who knows, but I'd be tempted to start it off with unexpected interactions stubbed and then push towards strict mock behaviour via something like a double.ThrowUnexpectedInteractions() when necessary.

Thoughts?

Gravatar

# re: Rhino Mocks 3.5 Design Decisions: The role of Stub vs. Mock 6/30/2008 2:09 AM Ayende Rahien

Jeremy,
I like this idea very much.
Any chance for a patch?

I still think that the distinction between a mock and a stub is important, and deserve its API, but the ability to immediately move between the different states would make a lot of complexity much easier

Gravatar

# re: Rhino Mocks 3.5 Design Decisions: The role of Stub vs. Mock 6/30/2008 4:30 AM Miika Makinen

Duck typing is nice.

Gravatar

# re: Rhino Mocks 3.5 Design Decisions: The role of Stub vs. Mock 6/30/2008 7:16 AM Steven Harman

These days I'm finding myself more and more in agreement with Aaron's thinking - don't define the object, but only the methods on that object... that is, the only important/interesting things we care about are the individual object-interactions the Test Double take part in.

@Sam, remember that in TDD/BDD, keeping things DRY is 2nd chair to solubility. So first make sure your tests are easy to read and understand, and then think about *possibly* refactoring some of the common context of each test into a higher-level, shared context.

Gravatar

# re: Rhino Mocks 3.5 Design Decisions: The role of Stub vs. Mock 6/30/2008 5:53 PM Steven Harman

I just realized that in your examples you are making use of the static MockRepository.GenerateXXX methods to build your Stub/Mock objects, and then arranging them. After that you go straight to Act and then Assert. This looks good!

However, it seems that when using an explicit MockRepository I have to Arrange, Move-to-Replay via mocks.Replay(myTestDouble), Act, Assert.

Is this by design? Is the move-to-replay step really required? If so, why?

I'm just wondering how the various ways of creating a test double are intended to be used.

Gravatar

# re: Rhino Mocks 3.5 Design Decisions: The role of Stub vs. Mock 6/30/2008 6:09 PM Jeremy Gray

(The comment system is erroring when I try to post all of this, so I'm going to break it up. This is part 1 of 3.)

@Oren - "Any chance for a patch" ahh, the classic test of true contribution. ;) I would quite honestly love to, but given my project load at the moment I highly doubt I'd be able to make any progress for quite longer than would be relevant, let alone desirable.

As for "the distinction between a mock and a stub", I too believe it is important, but I have also found that across the three teams I've used Rhino.Mocks with, no one has really _understood_that difference anyway. I think that those of us familiar with mocks, those of us who have read articles and positions on the different types, etc. might need to remind ourselves that we are in the vast minority. The only thing my current team has really grasped without significant effort and confusion is the difference between calling .Stub or .Expect on an instance. For this kind of audience, having GenerateMock and GenerateStub doesn't seem to add clarity, in fact it seems to reduce it, as strange as that may seem to us more experienced mockers.

Gravatar

# re: Rhino Mocks 3.5 Design Decisions: The role of Stub vs. Mock 6/30/2008 6:09 PM Jeremy Gray

(Part 2 of 3)
Here's what we are asking that audience to understand now:
1. First, you need to decide whether to call GenerateStub or GenerateMock (or perhaps a third call on that results in something more like a dynamic mock.)
2. Then, you need to decide whether to call .Stub or .Expect. If stubs are changed to not support .Expect, you'll need to first remember which Generate* method was called.
3. Next you need to decide whether or not to make any .AssertWasCalled calls. Again, this may be dependent on which Generate* method was called. Additionally, you need to understand the relationship between this AssertWasCalled call and any .Stub or .Expect calls against the same member, with the same arguments (and/or constraints), etc. Do you use only AssertWasCalled for those calls that didn't need return values, and use .Expect for the rest? Or do you use .Stub with a Return, and then also use AssertWasCalled?
4. Finally, you need to decide whether or not to call VerifyAllExpectations, and this is again dependent on all sorts of things. Was it a GenerateStub call, and do stubs support expectations? Did you set any expectations? Have you already called AssertWasCalled? If so, should you call VerifyAllExpectations or not?

As I hope is made clear by the above, we're asking people to remember an awful lot of things just to use a mock. While I grok this, and while basically everyone reading this groks this, I have found that even the most intelligent, diligent developers are having difficulty with the above unless they are following mocks closely, reading this blog regularly, etc. Do we want them to have to follow the mocking framework community so closely? Do we want them to have to remember and execute so many steps as described above?

Gravatar

# re: Rhino Mocks 3.5 Design Decisions: The role of Stub vs. Mock 6/30/2008 6:10 PM Jeremy Gray

(Part 3 of 3)
Let me tell you the subset of steps I have been able to get people to understand:

1. Double(). (note no distinction between GenerateStub, GenerateMock, something for dynamic, etc.)
2. .Stub and/or .Expect
3. .VerifyAllExpectations

(I personally use .AssertWasCalled at times for expectations without return values, but can't do so in team projects as it isn't in the three steps above. They start asking how it is different from .Expect and deciding which to use only slows them down.)

Obviously, we would still need some way to stub all members on request. Some would argue that GenerateStub versus GenerateMock would cover that, but only if GenerateStub still allows expectations to be set, at which point the name "Stub" at the end of that method call confuses people and we'd still need to add the third Generate* call for a mock with stubbed unexpected interactions. This is also made more complicated by the fact that in the 3.5 beta (I haven't had a chance to update to the RC to check this yet, but will this week) GenerateMock produces loose mocks. Alternative to GenerateMock versus GenerateStub versus Generate[INSERT_HYBRID_NAME_HERE] would be the kind of behaviour I suggested earlier and Oren suggested I provide a patch for ;), something ala .StubUnexpectedInteractions().

If we could get it down to what would then be a list of four steps, which is about as simple as I can envision it at the moment, I think we might find that a lot more people can productively make use of this kind of technology.

1. Double(). (note no distinction between GenerateStub, GenerateMock, something for dynamic, etc.)
2. .Stub and/or .Expect
3. Optionally .StubUnexpectedInteractions
4. .VerifyAllExpectations

Can anyone get the list even shorter? ;)

Gravatar

# re: Rhino Mocks 3.5 Design Decisions: The role of Stub vs. Mock 6/30/2008 8:15 PM Ayende Rahien

Steven,
The explicit move to replay is required.
This is because when you get an item from the static method calls, it is in replay mode.
If you want to do things like orderring calls, you have to use the instance methods, move to replay mode, and move on

Gravatar

# re: Rhino Mocks 3.5 Design Decisions: The role of Stub vs. Mock 6/30/2008 8:37 PM Steven Harman

@Ayende,
Was this changed between the 3.5 beta and RC?

I ask because I upgraded my project from the beta to the RC and suddenly several hundred tests started breaking. I am thinking they are breaking because the mocks created by the explicit MockRepository are still in record mode, but in the beta they too were already in replay mode, yes?

We use a test super-class [http://code.google.com/p/codeincubator/source/browse/Tools/trunk/CodeInc.Tools/CodeInc.Commons/Testing/Specification.cs] to provide some sugar for cleaning up line-noise in our tests, including Mock<T>() and Stub<T>() methods that wrap the explicit mocks.CreateMock<T>() and mocks.Stub<T>() calls. Those methods really help clean up the code when using them within the Record/Playback syntax... and were were also using them to generate our test doubles for the AAA syntax.

I'd like to continue using them, but its causing some friction b/c now we need to explicitly move to replay mode anytime we have a test double created by an explicit repository.

With that in mind, I think Jeremy raises some good points in that its hard to get developers to remember all of the details and nuances around the various types of test doubles created, how to set results and/or expectations, and managing their state (record vs. replay modes). I'm not sure I have solution in mind... just saying I think he's got me thinking... :)

Gravatar

# re: Rhino Mocks 3.5 Design Decisions: The role of Stub vs. Mock 6/30/2008 8:51 PM Jeremy Gray

@Steven re: "I'm not sure I have solution in mind... just saying I think he's got me thinking... :) " I still have _me_ thinking too. ;) There are so many nuances in play here that it is easy to chase an option too early and end up down a blind alley. Heck, I've already gone back and forth a bit in the last day or so in this set of comments alone. Progress does seem to be happening (see Oren's latest post at http://www.ayende.com/Blog/archive/2008/06/30/Rhino-Mocks-3.5-Design-Decisions-Getting-closer-to-conclusion.aspx) but it would be good if:

a) a bunch more people were chiming in a bit more, lest we end up down a blind alley on account of my ramblings. ;)
b) things get narrowed down over on Oren's latest post and then all parties stick it in the backs of their heads for a while to gel before anything gets firmly set in stone.

Gravatar

# re: Rhino Mocks 3.5 Design Decisions: The role of Stub vs. Mock 7/1/2008 3:10 AM Ayende Rahien

Steven,
No, this is something that I am thinking for RC2.
Can you create a test case for this?

Calling the static methods from MockRepository will return mocks in replay mode.
Calling the instance methods will return mocks in record mode.

Gravatar

# re: Rhino Mocks 3.5 Design Decisions: The role of Stub vs. Mock 7/5/2008 9:21 PM Daniel Cazzulino

This again one of those cases where trying to stick to rigid theoretical definitions (i.e. Fowler's) hinders usability and forces newbies to understand more than they need to get their work done.

That's why Moq does away with the distinction, and you can express the test with AAA syntax as:

public void When_user_forgot_password_should_save_user()
{
var userRepository = new Mock<IUserRepository>();
var smsSender = new Mock<ISmsSender>();

var theUser = new User{HashedPassword = "this is not hashed password"};

userRepository.Expect(x => x.GetUserByName("ayende")).Returns(theUser);

var controllerUnderTest = new LoginController(userRepository.Object, smsSender.Object);

controllerUnderTest.ForgotMyPassword("ayende");

userRepository.Verify(x => x.Save(theUser));
}

Note how you use Expect...Returns to stub return values, but just by appending .Verifiable() at the end of it, you can make it behave like a "true" mock which responds to .Verify() (with no expression, just verifies whatever you marked as verifiable).

Alternatively, you could call .VerifyAll() on the mock and it would verify all expectations regardless of them being marked "Verifiable" or not.

I believe this is simpler and more intuitive for newcomers. If you want a full proper "strict" mock you can just say it via the ctor:

var strict = new Mock<ISmsSender>(MockBehavior.Strict);


As you see, the user never gets exposed to the concepts (and differences) of Stub vs Mock

Gravatar

# re: Rhino Mocks 3.5 Design Decisions: The role of Stub vs. Mock 7/5/2008 9:55 PM Jeremy Gray

@Kzu - While I'm not yet sold on using .Expect for everything and adding .Verifiable() (as the .Stub/.Expect pair was just about the only thing everyone I've introduced this technology to has understood essentially immediately and I suspect that clarity may be reduced by asking people to always call .Expect, for things that aren't actually expected but instead are just supported by the mock object) I'm in total agreement with you on the issue with sticking to the rigid theoretical definitions.

It is quite surprising, come to think of it, that with all of the increased attention these days on things like dynamic typing (especially with obvious converts like Bellware hanging around) and with people focusing so much on behaviour meaning more than (or at least as much as) type, that many of those very same people get all in a twist when we try to apply those same concepts to mocks/stubs/doubles. They want looser typing, they want ducks, they want to focus on behaviour instead of type, but don't dare apply any of that to their precious mocks! ;)

Comments have been closed on this topic.