Unit-Testing Tutorial 03: Dummies and Mocks

I. Introduction  —  What’s a dummy?  What’s a Mock?

When we write a unit-test for a method, we’re primarily interested in that method.  Ideally, we’re not testing other objects or methods used by that method.

(Apply a liberal helping of common sense, however.  If method X uses methods Y and Z in the same class, we might find it easier to test all 3 together.  Or testing X may inescapably involve partial testing of Y, even if we already have separate tests for Y.)

This opens a can of worms, however, if class A’s method M uses some other (complicated) objects B and C.  The outcome of the test may well depend on the behavior of (and all the work involved in initializing the state of) B and C.  This is especially true if B or C talk to some external service of data source — a complete no-no for unit-tests.  So what’s a developer to do?

The answer is, we can use “pretend” versions of B and C.  We pre-program these pretend objects to just do the things we need in order to test M.  These pretend versions are typically called “dummies”, “mocks”, or “stubs”.  To implement them, we can:

1. Extend and override the needed classes.  E.g. class DummyB extends B, and @Override’s the needed methods.  Then we pass in or inject a DummyB into A for M to use.
2. Build from an interface.  It’s even better if B is really an interface, and then we can simply pass in or inject our own implementation of B.
3. Use a third-party mocking framework.  Typically these use all sorts of interesting java reflection, at run-time, to imitate the needed objects.  EasyMock is the probably the best-known framework, but there are several others (notably mockito and jmock).

Naming of Names
I recommend a simple naming convention to help distinguish real objects from these pretenders:

• A “dummy” is either #1 or #2 above.  I use “dummy” as a prefix, e.g. for a DummyLogger class, or a dummyResultsItem variable.
• A “mock” is a pretend object built from #3, a mocking framework.  So I’ll have variables like mockLogger or mockDatabaseListManager.

The value of this convention will become apparent when we cover setting expectations for mocks, a little later on.

Which to use?
Which should we use, dummies or mocks?  The answer, of course, is “it depends”.

• If it’s easy to build a dummy, I’ll use a dummy.
• If I need the object to remember how I used it, so I can interrogate it later in my assert’s, I’ll usually build a dummy. (I call this an “instrumented test-dummy”, using the metaphor of the dummies used in testing car crashes.)
• If the object is really complicated, and I only need a few pieces of it, and the results it returns are pretty arbitrary, I’ll use a mock.  Or worse, if I simply don’t understand the object at all, but I know its output, I’ll use a mock.
• If initializing the (real) object is really complicated, I’ll probably use a mock.
• If the object is really just a bean… then I won’t use either!  I’ll use the real bean.  (I’ve seen entirely too much code that mocks beans for no good reason.  It’s usually just as easy to call the setters on a bean as it is to pre-program a dummy or a mock for that bean.)

Examples

1. In my production code, I have a class DummyLogger which implements the org.slf4j.Logger interface.  Lots of production code calls a logger to log stuff (typically going to the Tomcat log files) for various reasons – and the calls, and the reasons for them, are frequently subject to change.  In a unit-test, there is no “real” logger (no Tomcat underneath!), so we have to use something else for the logger dependency in the classes being tested.So we have a dummy that basically does nothing when it gets called, and returns false for all the various “isEnabled?” calls (e.g. isDebugEnabled()).  We pass or inject the dummy into the class being tested, and the test runs just fine.In most cases, we don’t really care if the logger gets called – the purpose of the unit-tests is to test the functionality of some methods, i.e. what they actually do or return – not whether they called a logger or not.  In the (rare) cases where we do care, DummyLogger can be lightly “instrumented” – meaning, you can ask it at the conclusion of the unit-test, how many calls were made to which methods….
2. (insert more)

II. EasyMock

EasyMock is a Java library that helps us write unit-tests.  It provides the ability to easily make mocks (hence the name) of other objects that are used as dependencies in our tests.  We can quickly and inexpensively instruct these mock objects on how they’re supposed to behave when they’re used in a test.

For example, suppose I am writing a test for MyObject, which in turn uses (has a dependency on) ComplicatedObject.  ComplicatedObject may be very, well, complicated to instantiate, may have dependencies of its own, might depend on an external service… a dozen different things that would get in the way of writing a unit-test on MyObject.  With EasyMock, I can say “create me something that looks like ComplicatedObject, but doesn’t have any of its dependencies, and will do exactly what I tell it to do”.

EasyMock comes with some costs and caveats, however:

• You have to be careful about setting up the mocks, their “expectations” (instructions), and verifying that they were really used the way you expected.
• EasyMock won’t solve every problem, in fact it has some very specific limitations (more on that later).
• It’s easy to overuse EasyMock: I’ve seen (and fixed!) code that used EasyMock to mock a simple List of objects.  That’s just silly.  Only mock what you need to mock.

There are other good mocking libraries out there, notably Mockito, JMockit, and PowerMock.  Currently I mostly use EasyMock, although once you’ve learned one of these packages, the concepts are very similar across them all.

Most other object-oriented languages have something similar, e.g. Python has the embedded unittest.mock, “Dingus”, and others.  Perl has Test::Mock, Magpie, etc.

III. A Metaphor: the Secret Agent

My favorite metaphor for EasyMock is the “secret agent” of the Cold War spy thrillers.  A mock object created by EasyMock (henceforth just “mock”) is like a secret agent who “crosses behind the lines” and pretends to be someone else — say an official in the spied-upon government.  The agent goes through 6 phases:

1. Recruitment.  EasyMock.createMock() creates a new mock.
2. Training.  EasyMock.expect() sets up the “if this, do that” conditions.  E.g. “if you’re asked for the secret nuclear formula, show them this paper.”.  “If you’re asked for your mother-in-law’s pet’s name, it’s Fluffy.”.  Etc.
3. Finish training, and begin acting in character.  EasyMock.replay() says “OK, you’re done training, you’re on your own now.”
4. Assignment.  The mock is handed to, or injected into, the object to be tested.
5. Active duty.  The mock is used by the object under test.
6. Recall and debriefing.  Once the test proper is over, we call EasyMock.verify() to ensure that the operation was a success.  In particular, verify() checks that every expectation happened.  In the secret-agent scenario, if the agent was trained with the nuclear formula and the mother-in-law pet, and s/he were not asked questions about them… then something’s wrong.  The other side knows you’re a planted agent.

IV. A First (Contrived) Example

Here’s a simple (and highly contrived) example of using EasyMock in a unit-test.  In this scenario, I’m writing a WeatherWizard class that tells me things about the weather at my location.  When I construct a WeatherWizard, I give it two objects (dependencies):

• IpToLatLongConverter: takes an IP address, and converts it to a latitude and longitude
• TemperatureReporter: given a latitude and longitude, reports back the temperature at that location, in that location’s normal measurement system.

Both of these objects could be from third-party libraries, or even external web services, that I would normally have no control over.  So I’m writing tests for the method WeatherReporter.getTemperatureAt (String ipAddress).  Here’s how the first test might look:

 @Test
 public void shouldGetTemperatureForAnAmericanIpAddress() {
    // Make a mock "controller": rather like James Bond's "M" who oversees several agents.
    IMocksControl ctrl = EasyMock.createControl();

    // Make the mocks for the dependencies, overseen by the controller.
    IPtoLatLongConverter mockConverter = ctrl.createMock(IPtoLatLongConverter.class);
    TemperatureReporter mockReporter = ctrl.createMock(TemperatureReporter.class);

    // Train the converter to map my IP address into my lat/long.
    expect(mockConverter.convert("165.215.94.6")).andReturn("42.244849,-83.735787");

    // Train the reporter to map my lat/long into a specific temperature.
    expect(mockReporter.getTemperature("42.244849,-83.735787")).andReturn("35F");

    ctrl.replay(); // Done training both mocks.
    // Create a WeatherWizard in the usual way, but pass in the mock dependencies
    // instead of real ones.
    WeatherWizard wizard = new WeatherWizard (mockConverter, mockReporter);
    assertEquals ("35F", wizard.getFahrenheitTemperatureAt("165.215.94.6");
    ctrl.verify(); // Debrief the mocks: were they used?
 }

Notes about this example:

1. Notice the method name: it’s (almost) a sentence about what action (and what outcome) the method is testing for.  (I hate method names like “testGetTemperature”.  Tell me something I didn’t know!  Ideally a test method name tells you most of what you’d want to write in a comment, if you wrote a comment.)  Also, this method name is a hint about the next test that we’re going to write.
2. Notice the naming pattern for the mocks.  I like to clearly distinguish between the “real” objects and the mock objects.  This is not required, but I find it helps a lot when the tests get longer.  I’ve fixed many tests that messed up because the developer confused a real object for a mock object, or vice-versa.
3. The first expect() call is just a way of saying “I expect that there will be a call to mockConverter.convert, with that specific IP address as an argument, that normally returns a String.  In this case it should return the specific lat/long address that I gave in ‘andReturn()’.”
4. The real “action” in the test takes place between the replay() and the verify().  In this case, it’s really just in the assertEquals() call.  That assert looks pretty wimpy, but it will get more interesting in the next test.
5. The ctrl.verify() call checks to make sure that each of the expectations (from the expect() calls) really happened.  If the conditions of the expect’s (e.g. the arguments to convert() or getTemperature()) were not what was expected, EasyMock will complain and fail the test.  If, say, convert() got called with the right argument, but was called twice, EasyMock will fail the test.  If the calls didn’t happen at all, EasyMock will fail the test.This is important.  There’s a tendency for people to leave off the verify() calls.  DON’T.  The temptation is to think “well, the assert — the important part — passed, why do I really care if all of the expectations happened?  What harm is done if some of them didn’t happen?“.The answer is two-fold:
   1. Using verify() helps keep your tests clean.  Expect’s that are never used are useless code.  Useless code makes tests longer, harder to understand, and harder to change.
   2. Much more importantly, the verify() call helps test that you didn’t introduce any bugs in the test itself.  For example, I have fixed tests that (somehow) managed to call expect() on a real object… and still “pass”.  The verify() prevents that.

V. Multiple (contrived) Tests

Now let’s expand the example to handle the case I’ve been hinting at, where the local temperature is measured in Celsius.  I’m going to rearrange some of the previous test method, and combine it with a new one.

 

Notes about the combined example:

1. DRY.  (“Don’t Repeat Yourself”.)  Combining as much of the initialization as possible in the @Before method makes each of the tests simpler and more straight-forward.  Remember that for each @Test method, Junit constructs a new, separate, instance of the test class, runs the @Before method, and then runs the individual @Test method.  So there’s a new ctrl, mockConverter, etc. etc. for each @Test run.
2. MnM.  (“Make no Mistakes”).  IP addresses and latitude/longitude numbers are easy to goof up, so I moved them to static constants.  Nothing special about EasyMock here, just good clean coding.
3. I cheated a little in the @Before — I assumed that the constructor for WeatherWizard didn’t actually use mockConverter or mockReporter, just “squirreled them away” for use, later on, in the getFahrenheitTemperature() call.  If the constructor did use them (did make calls on their methods), then my tests would fail, and EasyMock would complain about a “missing behavior definition”.  To fix that, I’d have to rearrange the code, probably by moving the WeatherWizard constructor call to right after the ctrl.replay() in both tests.
4. Keep the action short and clear.  I really like to keep the “action” between replay() and verify() as small as possible.  It really helps focus attention on the “meat” of the test.

 

VI. Great(er) Expectations

1. Extending the expectation syntax.  The EasyMock “expect” syntax has a lot of powerful options, that provide for much more flexible (and complicated) expectations.  The full documentation can be found at easymock.org/user-guide.html, but the examples below should be fairly self-explanatory:

   expect(mockConverter.convert(PARIS_IP)).andReturn(PARIS_LATLONG);       // expect this... expect(mockReporter.getTemperature(PARIS_LATLONG)).andReturn("1.66C");  // ...and this... but in ANY order. expect(mockReporter.getTemperature(PARIS_LATLONG)).andReturn("1.66C");  // expect one of these... expect(mockReporter.getTemperature(PARIS_LATLONG)).andReturn("2.01C");  // ...FOLLOWED by one of these, in this order. // expect 2 calls expect(mockReporter.getTemperature(PARIS_LATLONG)).andReturn("1.66C").times(2); // expect any # of calls >= 1 expect(mockReporter.getTemperature(PARIS_LATLONG)).andReturn("1.66C").atLeastOnce(); // expect any # of calls >= 0 (try to avoid these!) expect(mockReporter.getTemperature(PARIS_LATLONG)).andReturn("1.66C").anyTimes(); // instead of returning normally.. expect(mockReporter.getTemperature(PARIS_LATLONG)).andThrow(new BadWeatherException());

2. Handling void.  If you want to define an expectation for a mock that returns void, you can’t wrap the “expect()” call around the mock call.  You just write the mock call by itself — which can appear confusing.  For example:

   @Test public void shouldConfirmLoggerRecordsError() {    Logger mockLogger = ctrl.createMock(Logger.class);    mockLogger.error("Logging an error");   // implicit expect!    ... }

   The apparent call to mockLogger.error() really is defining the expectation!  This can cause “WTF?” moments for people first reading the code.  It’s also another reason why it’s really, really, helpful to name mocks in some clear way — so that it’s obvious this really is an expectation on a mock, and not a call on a method of a real object.

   If you need to extend this expectation, EasyMock provides the “magic” expectLastCall() syntax, as in:

   mockLogger.error("Logging an error"); expectLastCall().times(2);                // Expect the mockLogger.error() call twice (total).     mockLogger.error("Logging an error"); expectLastCall().andThrow(new SomeExceptionOrOther());  // throw an exception on the call.

   …and basically all of the forms shown for mockReporter, above.

3. Mock argument matching.  The examples so far require that each expect() on a mock will include the precise arguments to the mocked method.  In many cases, we may not really care what the arguments were — we just know the behavior (the return value) that we want to mock.  EasyMock provides a set of “matchers” to match method arguments.  For example:

   mockLogger.error(EasyMock.isA(String.class));  // match *any* String expect(mockConverter.convert(EasyMock.startsWith("165.215"))    .andReturn(PARIS_LATLONG);         // match only specific strings

   The one surprising thing about these matchers is this: if any argument in a single expect() call uses a matcher, then all of them must use a matcher.  EasyMock provides eq() and aryEq() matchers for the arguments that you do have known, specific values for.  For example:

   //  Expect myUserid with *any* password. expect(mockUseridPasswordAuthenticator.isValid(EasyMock.eq("myUserid"),    EasyMock.isA(String.class)).andReturn(true); String [] countryCodes = new String [] {"us", "gb", "jp"}; expect(mockUnitedNations.isSecurityCouncil(EasyMock.aryEq(countryCodes)).andReturn(true);

   The last example is important because eq() does strict object equality matching — whereas aryEq() matches the values of contents of an array — not the actual array object itself.

   There are many, many more matchers described in documentation.  This includes matchers for Java primitives like ‘int’.  Generally, when I use a matcher, I usually do a static import of it, so that I can leave off the “EasyMock.” part.  DRY!

 

VII. Partial Mocks

Sometimes we want to mock only part of an object.  I.e., I might want to test method A in a class, that in turn uses a (complicated, slow, or otherwise hard to work with) method B.  I can mock B but still test the “real” A.  For example, I can do:

This creates a “mostly real” DatabaseListManager, but with a mocked version of findDatabaseById(), on which we can now set expectations.  (Notice, by the way, that the mocked method returns a result which is also a mock.  This kind of mock-within-a-mock can be very handy.)

If you have multiple methods called findDatabaseById(), EasyMock will complain that the addMockedMethod() call is ambiguous.  You can fix this by adding the types of the arguments to the desired method, as arguments to the addMockedMethod call.  E.g. addMockedMethod(“findDatabaseById”, String.class).  The list of arguments is a ‘varargs’ of type Class.  If you need to specify the one no-argument version of the method, hand it a zero-long array of Class objects, e.g. addMockedMethod(“findDatabaseById”, new Class[0]).  (Weird, huh?)

If you need to supply arguments to the constructor (of the object you are partially-mocking), use this syntax:

If you have a more complicated constructor (or multiple constructors), and you need to pass in nulls for some of the arguments, you may need to specify both the class and value(s) of the arguments (EasyMock isn’t always smart enough to figure it out by itself).   In that case, use something like this:

 

VIII. More on Controllers

1. Flavors.  There are actually three different flavors of mock controllers, and also the option to have stand-alone or “uncontrolled” mocks (think of a rogue 007 operating on his own w/o a spymaster!):
   • Regular mocks.  What we’ve been doing so far.
   • Strict mocks. The order of calls on mocks must exactly match the order of expect() calls that you wrote.  Use EasyMock.createStrictControl() to make a strict controller.
   • Nice mocks.  If you call a “nice” mocked method that doesn’t have an expect(), it returns null, 0, or false (as appropriate to the return signature) and keeps on going, instead of failing like a regular mock would.  Use EasyMock.createNiceControl() to make nice mocks.  Basically, this saves you from writing a lot of pointless expect()’s when you really don’t care about the results, you just need those mocks for your code to work.
   • Stand-alone mocks.  You can create a mock directly, e.g. “MyClass myMock = EasyMock.createMock(MyClass.class);“.  But then you must specifically call EasyMock.replay(…) and EasyMock.verify(…) on each and every such individually-created mock.  This is the way EasyMock originally worked, and the controller concept was added later.  Usually stand-alone mocks are a bad idea, since it’s far easy to lose track of them.  Don’t do this unless there’s a very good reason.
2. Multiple controllers.  In theory, you can create and use multiple controllers in the same test.  Usually this is a bad idea, or is the result of trying to test some very tangled code.  (And if so, then write the tests as needed, then use those as a safety net to untangle the code, and then rewrite the tests to match and also be simpler!  But I digress…)There are two cases where this can be useful (but still beware the slippery slope!):
   • Some of the mocks should be regular mocks, but others can be “nice” mocks
   • Some mocks need to be replay()’d before other mocks are even defined(!).  This most often occurs when the constructor for the object under test makes active use of a mock, long before the rest of the mocks can even be defined.  This may well be a code “smell” that the constructors are being too aggressive.
3. Resetting a controller.  It is possible to reset (really, “wipe”) a controller, and (to use the secret agent metaphor) start a whole new set of missions with a new set of agents (mocks).  Most of the time, this is pretty pointless — it’s easier and cleaner to just write a new test, which will get its own (new) controller.  Occasionally, if there’s a lot of non-mock-related setup at the start of the test, it might make sense to create a controller, make the expect()’s, replay, test, verify… and then call ctrl.reset(), make new expects(), replay, test, and verify.  Or perhaps the expect/replay/test/verify phases occur many times inside a “for” loop, and resetting the controller is cheaper than completely recreating it.

 

IX. Patterns and Best Practices

Most of the “best practices” for EasyMock are just ordinary Java best practices.  Write clean code, document the test with good method and variable names, break longer methods into shorter methods that operate at the same level of abstraction.  Don’t Repeat Yourself.  Etc.

Nonetheless, there are some specific patterns that are worth noting:

1. Use one controller.  Create it in your @Before method.  If you’re tempted to use more than one controller, stop and think about why.  Occasionally they are necessary.  So are blue moons.
2. DRY your mocks.  If you’ve got a bunch of mocks in common, create them in your @Before method.  Even if some of them go unused in some test methods — who cares?
3. Inject consistently.  Similar to #2, often you’ll find that the same mocks are being injected.  If possible, do them all in @Before.
4. Define, train, and inject your mocks together.  I’ve seen entirely too much code that is a wilderness of mock creation, injection, and expect() calls, scattered willy-nilly across a method.  Do everything possible to create a mock, inject the mock, and define its expectations in one place.  Normally it doesn’t matter if the mock is used in a bunch of different places throughout the object under test.  Put the full behavior definition for the mock all together.  The next person to read your code doesn’t care about the sequence of operations in the object under test.  S/he cares about understanding what your mock is supposed to do
5. Test the results of the code, not the execution.  With EasyMock and an object that has a lot of dependencies, it is far to easy to write code that tests “…that the code that was written is the code that was written”.  E.g. suppose I’m testing a method in object A, which calls a mock B, then a mock on C, and returns something.  I’ve seen entirely too many tests that just verify that mocks B and C get called with the right arguments.  Now, sometimes it’s hard to get around that: if most of the work in A is done by B and C, you may be stuck.  But always try to find some way of asserting that some result was returned (or at least happened!).  If the “result” is the change of state of something private in A, you could even use reflection to “cheat” and examine the private state of A. (But, yuck!) That’s far better than just relying on the EasyMock replay() and verify() calls.
6. Watch out for shadowed members.  If class A has a member X, and class B extends A and also has a member X… injecting a mock X can fail in fascinating and aggravating ways.  This is a noxious code smell.  Remove B’s X member, and use A’s getter and setter instead.  If A doesn’t have a getter or setter, create them as needed.  This is not an EasyMock problem per se, but it seems to crop up most frequently in EasyMock tests.
7. Don’t mock something just because you can.  In particular, don’t mock beans!  For example, in the Morningstar code, there are plenty of mocks of the User object.  This is almost always silly.  It’s almost always easier to use a real User than it is to mock one.  Plus, if the definition of User changes, your test gets the change “for free” — you don’t have to update the mock.Along the same lines, often it’s easier to subclass and override a method, than it is to write a partial mock with EasyMock.  (In fact, you could call this a “partial dummy”, parallel to the concept of a “partial mock”.)   EasyMock is a great hammer, but that doesn’t make everything a nail.

X. Limitations

EasyMock has some significant limitations that are not immediately obvious.

1. You can’t mock a private method.  Period.  Frankly, if that stops me from writing an important test, my standard practice is to “promote” the method to “package protected” (aka default protection), and mark it with the comment “/*TestScope*/” –  even though there’s no “production” reason to do this.  I strongly believe tested code trumps obsessively-over-controlled permission schemes.  (See, for example, Uncle Bob’s article on The Little Singleton.)
2. You can’t mock “final” methods.
3. You can’t mock a constructor call.
4. You can’t mock static methods.  (OTOH, I think most static methods are evil.  Most of the time, there’s no functional reason why a method should be static.  Quite the reverse: static methods force tight coupling.  The code that uses them is forced to use one and only one implementation. YMMV.)

The good news (sort of), is that there is an add-on called PowerMock that extends EasyMock so that you can write mocks for these cases.  The bad news is that PowerMock comes with its own baggage, e.g. it doesn’t play well with EclEmma (code coverage analyzer that “colorizes” your tested and untested code), and may have difficulties with Java 1.7 or higher.  Avoid PowerMock unless absolutely, absolutely, necessary.  (For example, I’ve used PowerMock to get good test coverage for a class, so that I could then refactor it so that I no longer needed PowerMock!)

XI. Advanced Features

1. EasyMock provides a “capture” feature that lets you capture the value of an argument to a mock, and then do something with it.  But that’s getting awfully close to just writing real Java code (e.g. an “instrumented test dummy”).   Read the EasyMock documentation if you really want to know about this feature.  I’ve never really found a good use for it.

XII. Other Resources

1. ibm.com/developerworks/java/library/j-easymock/index.html is a good article on EasyMock with (more interesting) examples… although it is somewhat dated, and based on earlier versions of EasyMock and Junit.