Net Objectives

Net Objectives
If you are interested in coaching or training in ATDD or TDD please click here.

Friday, September 25, 2015

TDD: Specifying the Negative

One of the issues that frequently comes up is "how do I write a test about a behavior that the system is specified not to have?"  It's an interesting question given the nature of unit tests.  Let's examine it.

The things that you cannot do

Sometimes part of the specification of a system is to disallow certain actions to be taken on it.

The simplest example of this is an immutable object.  Let's say there exists in our system a USDollar class that represents an amount of money in US dollars.  Such a class might exist in order to restrict, validate, or perfect the data it holds.

But it's believable that we'd want an instance of USDollar, once created, to be immutable... that is, the value it was created to represent cannot be changed afterwards.  We might want this for thread-safety, or security, or any number of reason.  Many developers strongly believe in immutable objects as a general principle of good design.  Whether you agree or not, it would bring up the question "how do I specify in a test that you cannot change the value?"

public class USDollar
    private readonly double myAmount;
    public USDollar(double anAmount)
        myAmount = anAmount;

    public double GetValue()
        return myAmount;

How can we test-drive such an entity when part of what we wish to specify is that the amount, once established in an instance of this class, cannot be changed from the outside?  The most common way this question is phrased is "how can I show in a test that there is no SetAmount() method?"  Any test that references this method simply will not compile because the method does not exist.

Developers will typically suggest two different ideas:
  1. Add the SetValue() method, but make it throw an exception if anyone ever calls it.  Write a test that calls this method and fails if the exception is not thrown.[1]  Sometimes other "actions" are suggested if the method gets called, but an exception is quite common.
  2. Use reflection in the test to examine the object and, if SetValue() is found, fail the test.
The problem with option #1 is that this is not what the specification is supposed to say, it is not what was wanted.  The specification should be "you cannot change the value" not "if you change the value, thing x will happen."  So here, the developer is creating his own specification and ignoring the actual requirements.

The problem with option#2 is twofold:  First, reflection is typically a very sluggish thing, and in TDD we want our tests to be extremely fast so that we can run them frequently without this slowing down our process.  But even if we overcame that somehow, what would we have the test look for?  SetValue()ChangeValue()PutValue()AlterValue()?  The possibilities are infinite.

The key to solving this is in reminding ourselves, once again, that TDD is not about testing, but creating a specification.  Professional developers have always worked from some form of specification, it's just that the form was usually a document or something.

So think about the traditional specification, the one you're likely more familiar with.  Ask yourself this: Does a specification indicate everything the system does not do?  Obviously not, for this would create a document of infinite length.  Every system does a finite set of things, and then there is an infinite set of things it does not do.

So here's the test:

public class USDollarTest
    public void TestUSDollarPersistence()
        const double anyAmount = 10.50d;
        USDollar testDollar = new USDollar(anyAmount);

        double retrievedValue = testDollar.GetValue();

        Assert.AreEqual(retrievedValue, anyAmount);

"Huh?" you're probably saying.  "How does this specify that you cannot change the value?"

Examine the test, then the code, and ask yourself the following question:  If we are doing TDD to create this USDollar object, and if the object had a method allowing the value to be changed (SetValue() or whatever), how would it have gotten there?  Where is the test that drove that mechanism into existence?  It's not there.

In TDD we never add code to the system without have a failing test first, and we only add the code that is needed to make the test pass, and nothing more.  It must be this way, or the process does not work.

Put yet another way, if a developer on our team added a method that allowed such a change, and did not have a failing test written first, then she would not be following the process.  TDD does not work if you don't do it.  I don't know of any process that does.

In fact, if someone did add the method without a failing test, we would say that person was maliciously attacking the code.  If you have someone doing that, there is nothing that will save you.  If I wanted to attack you, and I can change your code, I'll write something that checks to see if the code is currently running in the test environment and, if so, will behave correctly. Otherwise I'll launch the nuclear arsenal. How would a test ever detect that?  Impossible.

And if we think back to the concept of a specification there is an implicit rule here, which basically has two parts.
  1. Everything the system does, every behavior must be specified.
  2. Given this, anything that is not specified is by default specified as not a behavior of the system.
In TDD anything the system does must have a test, which is ensured if we are disciplined about following the process.  Thus, rule two is also ensured.

But this may not be quite enough, it may not be clear enough or reliable enough.  We never want to lose knowledge.  So if we think of this requirement in terms of acceptance testing [3] we could express it using the Given/When/Then nomenclature:

Given: A USDollar class exists in the system
When: An attempt is made to change the value it holds after it is created
Then: The system will fail to compile

This, of course, implies a strongly-typed, compiled language with access-control idioms (like making things "private" and so forth).  What if your technology does not support this view?  What if it is an interpreted language, or one with no enforcement mechanism to prevent access to internal variables?

The answer is: You have to ask the customer.  You have to tell them that you cannot do precisely what they are asking for, and consider other alternatives in that investigation.

The things that must not occur

This is subtly different.  Let's add a requirement to our USDollar class.  If the context of this object was, say, an online book store, the customer might have a maximum amount of money that he allows to be entered into a transaction.

We used a double-precision number to hold the value in USDollar. A double can hold an incredibly large value.  In .net, for example, it can hold a value as high as 1.7976931348623157E+308[2].  It does not seem credible that any purchase made at our customer's site could total up to something like that!  So the requirement is: Any USDollar object that is instantiated with a value greater than the customer's maximum credible value should raise a visible alarm, because this probably means the system is being hacked or has a very serious calculation bug.

As developers, we know a good way to raise an alarm is to thrown an exception.  We can do that, but we also capture the customer's view of what the maximum credible value is, so we specify it.  Let's say he says "nothing over $1,000.00 makes any sense".  But... how much "over"?  A dollar?  A cent?  We have to ask, of course.  Let's say the customer says "one cent"."

In TDD everything must be specified, all customer rules, behaviors, values, everything.  So we start with this:

public void SpecifyMaximumDollarValue()
    Assert.AreEqual(1000d, USDollar.MAXIMUM);

...which won't compile, of course, so we add the constant to the USDollar but give it a 0 amount.  So we run the test, and watch it fail (which proves the test is valid).  Then we set the amount to the correct one, 1,000, and watch it pass.

Now we can write this test, which will also fail initially of course:

public void TestUSDollarThowsUSDollarValueTooLargeException()
        new USDollar(USDollar.MAXIMUM + .01);
        Assert.Fail("USDollar created with amount over the maximum"+

                    " should have thrown an exception");
    catch (USDollarValueTooLargeException)

But now the question is, what code do we write to make this test pass?  The temptation would be to add something like this to the constructor of USDollar:

if(anAmount > MAXIMUM) throw new USDollarValueTooLargeException();

But this is a bit of a mistake.  Remember, it's not just "add no code without a failing test", it is "add only the needed code to make the failing test pass."

Your spec is your pal.  He's there at your elbow saying "don't worry.  I won't let you make a mistake.  I won't let you write the wrong code, I promise."  He's not just your pal, he's your best pal.  Here, the spec/test is just a mediocre friend because he will let you write the wrong code and say nothing about it.  He'll let you do this, and still pass:

throw new USDollarValueTooLargeException();

No conditional.  Just throw the exception all the time.  That's wrong, obviously. This behavior has a boundary (as we discussed in our blog about test categories) and every boundary has two sides.  We need a little more in the test.  We need this:

    new USDollar(USDollar.MAXIMUM);
catch (USDollarValueTooLargeException)
    Assert.Fail("USDollar created with amount at the maximum"+

                " should not have thrown an exception");

Now the "if(anAmount > MAXIMUM)" part must be added to the production code or your best buddy will let you know you're off track.

[TODO: The spec is the guys who takes your keys away]


[1] There are a variety of ways to do this, but most frameworks have built-in assertions that cause a failure if an expected exception is not thrown.
[2] For those who dislike exponential notation, this is:
...and no cents. :)
[3] [TODO] Link to ATDD blog

Wednesday, September 23, 2015

TDD and Its (at least) 5 Benefits

Many developers have concerns about adopting test-driven development, specifically regarding:
  • It's more work.  I'm already over-burdened and now you're giving me a new job to do.
  • I'm not a tester.  We have testers for testing, and they have more expertise than I do.  It will take me a long time to learn how to write tests as well as they do.
  • If I write the code, and then test it, the test-pass will only tell me what I already know: the code works.
  • If I write the test before the code the failing of the test will only tell me what I already know: I have not written the code yet.
Here we are going to deal with primarily the first one:  It's going to add work.

This is an understandable concern, at least at initially, and it is not only the developers that express it.  Project managers will fear that the team's productivity will decrease, which they are accountable for.  Project sponsors fear that the cost of the project will go up if the developers end up spending a fair amount of their time writing tests.  The primary cost of creating software is developer time.

The fact is, TDD is not about adding new burdens to the developers, but rather it is just the opposite: TDD is about gaining multiple benefits from a single activity.

In the test-first activity developers are not really writing tests.  They look like tests, but they are not (yet).  They are an executable specification (this is a critical part of our redefinition of TDD entry).  As such, they do what specifications do: they guide the creation of the code.  Traditional specifications, however, are usually expressed in some colloquial form, perhaps a document and/or some diagrams.  Communication in this form can be very lossy and easy to misinterpret.  Missing information can go unnoticed.

For example, one team decided to create a poker game as part of their training on TDD.  Often an enjoyable project is good when learning as we tend to retain information better when we're having a good time.  Also, these developers happened to live and work in Los Vegas. :) Anyway, it was a contrived project and so the team came up with the requirements themselves; basically the rules of poker and the mechanics of the game.  One requirement they came up with was "the system should be able to shuffle the deck of cards into a reordered state."  That seemed like a reasonable thing to require until they tried to write a test for it.  How does one define "reordered?"  One developer said "oh, let's say at least 90% of the cards need to be in a new position after the shuffle completes."  Another developer smiled and said "OK, just take the top card and put in on the bottom.  100% will be in a new position.  Will that be acceptable?"  They all agreed it would not.  This seemingly simple issue ended up being more complicated than anyone had anticipated.

In TDD we express the specification in actual test code, which is very unforgiving.  One of the early examples of this for us was the creation of a Fahrenheit-to-Celsius temperature conversion routine.  The idea seemed simple: take a measurement in Fahrenheit (say 212 degrees, the boiling point of water at sea level), and convert it to Celsius (100 degrees).  That statement seems very clear until you attempt to write a unit test for it, and realize you do not know how accurate the measurements should be.  Do we include fractional degrees?  To how many decimal places?  And of course the real question is what is this thing going to be used for?  This form of specification will not let you get away with not knowing because code is exacting like this.

Put another way, a test would ask "how accurate is this conversion routine?"  A specification asks "how accurate does this conversion routine need to be" which is of course a good question to ask before you attempt to create it.

The first benefit of TDD is just this: it provides a very detailed, reliable form of something we need to create anyway, a functional specification.

Once the code-writing beings, this test-as-specification serves another purpose.  Once we know what needs to be written, we can begin to write it with a clear indication of when we will have gotten it done.  The test stands as a rubric against which we measure our work.  Once it passes, the behavior is correct.  Developers quickly develop a strong sense of confidence in their work once they experience this phenomenon, and of course confidence reduces hesitancy and tends to speed us up.

The second benefit of TDD is that it provides clear, rapid feedback to the developers as they are creating the product code.

At some point, we finish our work.  Once this happens the suite of tests that we say are not really tests (but specifications) essentially "graduate" into their new life: as tests, in the traditional sense.  This happens with no additional effort from the developers.  Tests in the traditional sense are very good to have around and provide three more benefits in this new mode...

First, they guard against code regression when refactoring.  Sometimes code needs to be cleaned up either because it has quality issues (what we call "olfactoring"[1]), or because we are preparing for a new addition to the system and we want to re-structure the existing code to allow for a smooth introduction of the enhancement.  In either case, if we have a set of tests we can run repeatedly during the refactoring process, then we can be assured that we have not accidentally introduced a defect.  Here again, the confidence this yields will tend to increase productivity.

The third benefit is being able to refactor existing code in a confident and reassured fashion.

But also, they provide this same confirmation when we actually start writing new features to add to an existing system.  We return to test-as-specification when writing the new features, with the benefits we've already discussed, but also the older tests (as they continue to pass) tell us that the new work we are doing is not disturbing the existing system. Here again, allows us to be more aggressive in how we integrate the newly-wanted behavior.

The fourth benefit is being able to add new behavior in this same way.

But wait, there's more!  Another critical issue facing a development team is preventing the loss of knowledge.  Legacy code often has this problem:  the people who designed and wrote the systems are long gone, and nobody really understands the code very well.  A test suite, if written with this intention in mind, can capture knowledge because we can consider it any time to be "the spec" and read it as such. 

There are actually three kinds of knowledge we need to retain.
  1. What is the valuable business behavior that is implemented by the system?
  2. What is the design of the system?  Where are things implemented?
  3. How is the system to be used?  What examples can we look at? 
All of this knowledge is captured by the test suite, or perhaps more accurately, the specification suite.  It has the advantage over traditional documentation of being able to be run against the system to ensure it is still correct.

So the fifth benefit is being able to retain knowledge in a trustworthy form.

Up do this point we've connected TDD to several critical aspects of software development:
  1. Knowing what to build (test-first, with the test failing)
  2. Knowing that we built it (turning the test green)
  3. Knowing that we did not break it when refactoring it (keeping the test green)
  4. Knowing that we did not break it when enhancing/tuning/extending/scaling it (keeping the test green)
  5. Knowing, even much later, what we built (reading the tests after the fact)

All of this comes from one effort, one action.

And here's a final, sort of fun one:  Have you ever been reviewing code that was unfamiliar to you... perhaps written by someone else or even by you a long time ago, and you come across a line of code that you cannot figure out.  "Why is this here?   What is it for?  What does it do?  Is it needed?"  One can spend hours poring over the system, or trying to hunt down the original author who may herself not remember.  It can be very annoying and time-consuming.

If the system was created using TDD, this problem is instantly solved.  Don't know what a line of code does?  Break it, and run your tests.  A test should fail.  Go read that test.  Now you know.

Just don't forget to Crtl-Z. :)

But what if no test fails?  Or more than one test fails?  Well, that's why you're reading this blog.  For TDD to provide all these benefits, you need to do it properly...

[1] We'll add a link here when we've written this one