Test-Driven Development in the Larger Context: Pt 1. Audience

One difference between ATDD and UTDD becomes clear when you examine who is involved in the process.  We will call this the “Audience” for the process in this blog.

Audience

 

ATDD

Acceptance tests should be created by a gathering of representatives from every aspect of the organization: business analysts, product owners, project managers, legal experts, developers, marketing people, testers, end-users (or their representatives), etc.  

ATDD is a framework for collaboration that ensures complete alignment of the development effort that is to come with the business values that should drive it.  These business values are complex and manifold, and so a wide range of viewpoints must be included.  Acceptance tests should be expressed in a way that can be written, read, understood, and updated by anyone in the organization.  They should require no technical knowledge, and only minimal training.

The specific expression of an acceptance test should be selected based on the clarity of that form given on the nature of the organization and its work.  Many people find the “Given, When, Then” textual form (often referred to as “Gherkin”) to be the easiest to create and understand.  Others prefer tables, or images, or other domain-specific artifacts.

We once worked with an organization that did chemical processing.  We noted that in all their conversations and meetings, they used images like this in their PowerPoint slides and on the whiteboards, etc.:

To most people (including myself) these would not be easy to understand, but for this organization it was obvious and clear.  For them, expressing their acceptance tests this way lowered the bar of comprehension.  Why make them convert this into some textual or other representation?  Use their language, always.

Typical business analysts spend much of their time looking at spreadsheets, or Gantt charts, or Candlestick charts, etc…  The point is, once the stakeholders of a given piece of software are identified, then the form of the expression chosen should be whatever is clearest for them.  They should be able to write the tests, read the tests, and update the tests as needed, without any understanding of computer code (unless the only stakeholders are literally other developers).

The notion of automating these tests should never drive their form.  Any representation of acceptance can be make executable given the right tools, even if those tools must be created by the organization.  Choosing, say, Robot or Fit or Specflow to automate your acceptance tests before you identify your stakeholders is putting the cart before the horse.

 

UTDD

Unit tests should be written by technical people: developers and sometimes testers as well.  They are typically written in the computer language that will be used to develop the actual system, though there are exceptions to this.  But in any case, only people with deep technical knowledge can write them, read them, and update them as requirements change. 

To ensure the suite of tests itself does not become a maintenance burden as it grows, developers must be trained in the techniques that make the UTTD effort sustainable over the long haul.  This includes test structure, test suite architecture, what tests to write and how to write them, tests that should be avoided, etc.   Training of the development team must include these critical concepts, or the process rapidly becomes too expensive to maintain.

Intro
Part 1
Part 2
Part 3
Part 4

Test-Driven Development in the Larger Context: TDD & ATDD (Introduction)

Introduction

A question that often arises in our consulting and training practices concerns the relationship between Test-Driven Development (TDD) and Acceptance-Test-Driven Development (ATDD).  Which is “really” TDD, and which should an organization focus on in terms of which comes first, and which should achieve the most attention and resources?

It’s not uncommon for someone to refer to TDD as “developers writing unit tests before the production code”.  That is one form of TDD, certainly, but is a subset of a larger set of ideas and processes.

Similarly, it’s not uncommon for someone to refer to ATDD as “acceptance tests being written before the development cycle begins”.  Again, this is not incorrect, but it is usually seen as a separate thing from what the developers do which is typically thought of as “TDD”.

The truth is they are both TDD.  They are not conducted in the same way nor by the same people (entirely), and the value they provide is different (but compatible) … but at the end of the day TDD is the umbrella that they both fall under.  TDD is a software development paradigm.

This blog series will begin by investigating the differences between these two forms of the overall “Test-Driven” process, which we will call (for clarity) ATDD and UTDD (or “Unit-Test-Driven Development”, which most people think of as TDD).  We will see how they differ in terms of:

1. Who is involved?  Who creates these “tests”, updates them, and can ultimately read and understand them in the future?  We will use the term “Audience” to describe the different groups that conduct ATDD and UTDD.
2. When they are done, and the granularity of their activities.  This “Cadence”, as we shall see, is quite different and for good reason.
3. Finally, we will examine the difference in the value and the necessity of the effort to automate these two different processes.  Both certainly can be automated, but how critical is such automation, what value is derived with and without it, and where should the effort to automate fall in the adoption process?  We will call this section “Automation”.

After having examined the differences between ATDD and UTDD, we will then address how they are the same, why they both fit under the overall rubric “TDD” and how, ultimately, they synergize into a very powerful way to create alignment around business value in software development.  These similarities include:

1. Shared understanding of the valuable behaviors that make up a requirement, and the way those behaviors satisfy the expectations of the stakeholders.
2. The preservation of high-worth enterprise knowledge in a form that can retain its value over time.
3. The creation of a high-quality architecture and design, and all the value that this brings in terms of the ability of the organization to respond to market challenges, opportunities, and changes.

Regardless of your process, every activity conducted by every individual in a software development organization should be ultimately traceable back to business value: every bit of code that’s written, every test that is defined and executed, every form that’s filled out, every meeting that’s called, everything.  We want to show that TDD is in complete alignment with this concept, and, when properly conducted, increases the value of every investment of time and effort, and its ultimate return to the bottom line.

Links to the parts of the blog:

Intro
Part 1
Part 2
Part 3
Part 4

TDD: Testing Behavior in Abstract Classes

Interfaces vs. Abstract Classes

In languages like Java and C#, developers can use either an interface or an abstract class to create object polymorphism.  It’s a common question in technical training: “It is best to use an interface, or an abstract class?”

Furthermore, many teams adopt the “I” naming convention for interfaces; namely that an interface’s name should start with a capital I, whereas other classes (including abstract classes) should not.  The problem with this convention is that it creates design coupling.  Client objects that contain references to service objects must be changed when a simple, concrete class must be changed to become an abstraction, if an interface is to be used to model it.  Should client objects care whether a service is a concrete class, abstract class, or interface? No.  This would seem to argue against this naming convention in the first place.

But the real problem stems from the fact that the “interface” type is commonly used for two very different purposes: to create polymorphism and to mark a class as a valid participant in a framework process.  For example, a class can implement “ISerializible”, not for casting purposes per se, but so it can be serialized by .Net or a similar framework.  This may be a tangential issue to the class’ core responsibility.  On the other hand, 10 different versions of a tax calculation algorithm implemented by 10 different tax calculation classes can all implement “ITaxCalc” so that they can be cast up and dealt with in the same way by various client classes.  This would create polymorphism around the central responsibility of all the classes involved: calculating taxes.  If we had started with a single algorithm, a concrete class called TaxCalc, and this was referred to across the system by that name, then when we evolve the system to support different algorithms and thus the class becomes an interface, then the type name would change (if the “I” convention is used) and all client code will have to be maintained.

Different Purposes, Different Approaches

It seems like a bad idea to use one idiom for two unrelated purposes.
Personally, I prefer to create polymorphism using abstract classes, and to mark a class for participation in a framework process using interfaces.

Part of my argument is this:  when many different classes have a conceptual relationship, such as the tax calculators mentioned above, then it is likely they will also contain some code in their implementation that is the same.  This yields redundancy that creates maintenance problems when requirements due to tax laws and regulations change (for example).  An abstract class can implement common functionality, whereas interfaces cannot.  Even if a set of related classes contains no redundant implementation today, redundancies can emerge over time.  Abstract classes make this problem easy to solve whenever it arises.

Also, if I limit the use of interfaces to process flags, then the “I” convention is less of an issue.  I do not create design coupling within my system if I use it, because, for example, “IComparable” is not my interface, it allows a collection of classes to be sorted by a framework.  It belongs to that framework and is highly unlikely to be changed, due to the chaos this would create in everyone’s code if it were to be.  In any case, I don’t control its name.

TDD and Common Behaviors

If an abstract class is used to create polymorphism, and if there is indeed some common functionality in the base class, then the question arises: how do I test that behavior?  One cannot instantiate an abstract class, and thus its behavior cannot be triggered by a test unless that behavior is in a static method.  Static methods are disfavored for a number of reasons (I’ll deal with those in another blog), and I certainly would not make the behavior static just for testing purposes.  So what should a TDD practitioner, or a traditional tester, do about testing instance behavior that is implemented in an abstract class?

Here is a completely generic example:

Each “ConcreteService” version would have its own test, for each implementation version of the “VaryingFunction()” method.  But how would one write a test for the “CommonFunction()” method if it were an instance method, and one cannot create a instance?

Initially you might say “well, just pick any of the subclasses, create an instance of it and test the common function there, as they all have access to it.”  The problem is that this creates coupling in the test to the concrete service class that you arbitrarily chose.  If you happened to pick “ConcreteService1”, for instance, and later that class were to be retired/eliminated due to changing requirements, then the test of the common function would break even though that function is working fine.  Similarly, if "ConcreteService1" at some point in the future were to be changed to override the "CommonFunction()" method, this will also break the test.  We want tests that fail only for the reason we wrote them to.

Another Use for a Mock Object

Mock objects[1] are used to break and control dependencies in testing.  Here we can use a mock to eliminate coupling from the test of the common function to any of the concrete production classes.

This mock, like any subclass, has access through inheritance to the common function, but unlike other subclasses the mock:

1. Is not part of the production code, but actually part of the test namespace/package/etc…
2. Is never eliminated due to a changing requirement.  It is really part of the test.
3. Is not a public class.  It is only visible to the tests.

Another advantage of this approach is that it makes it easier to test base-class behavior that is not exposed to the system in general (not public).

This is a pattern, a “Testing Class Adapter”[2].  It works because the test will hold the “Mock Service” by its concrete type, not in an upcast, and thus this new accessor method, which is public, can be called to access the protected method in the base class.  Again, this mock is not part of production, and thus does not break encapsulation in general, only for testing.

Summary

I prefer to use abstract classes to create polymorphism, and use interfaces to flag classes as participants in framework services.  When you do this, you can easily eliminate functional redundancies in derived classes by pushing them up into the base class.  To test this otherwise-redundant functionality use a mock object/testing class adapter to access it.

---

[1] For more on Mock Objects see:

http://www.netobjectives.com/PatternRepository/index.php?title=TheMockObjectPattern

[2] For more on the Adapter Pattern see:

http://www.netobjectives.com/PatternRepository/index.php?title=TheAdapterPattern

Magic Buttons and Code Coverage

This will be a quickie.  But sometimes good things come in small packages.

This idea came to us from Amir's good friend Eran Pe'er, when he was visiting Net Objectives from his home in Israel.

I'd like you to imagine something, then I'm going to ask you a question.  Once I ask the question you'll see a horizontal line of dashes.  Stop reading at that point and really try to answer the question.  Once you have your answer, see if you can work out why you answered the way you did.  Consider putting a comment below with your thoughts.

Here is the thing to imagine:

In front of you are two magic buttons, A and B.  You can only push one of them.

• If you push A, you will instantly achieve 80 percent code coverage, only 20 percent of your code will be left without test coverage
• If you push B, you will only get 70 percent code coverage, leaving 30 percent of your code uncovered

But there is one more difference.  If you push A you will have no idea where in your code the lack of coverage is, you just know that some 20% of your code is not covered by tests. If you push B, you will know precisely where your covered and uncovered code is, though of course there will be more uncovered code.

Which would you press, and why?

--------------------------------------Stop and Answer------------------------------------

For me, the answer is unequivocally B.  A big part of the value of TDD is reducing uncertainty and communicating with precision.  Coverage is all about adding a degree of safety when making changes whether they be via refactoring or when making enhancements to existing systems.  If I know where the uncovered code is, I know where I have to be more careful, and conversely if I know where the covered code is I know where I can be aggressive.

Also, if I push A, then I know that for every line of code there is a 1-in-5 chance that it is uncovered.

But I think this is only part of the answer.  We'd like to hear from you...

TDD and Design: Frameworks

Increasingly, in modern software development, we create software using components that are provided as part of a language, framework, or other element of an overall development ecosystem.  In test-driven development this can potentially cause difficulties because our code becomes dependent on components that we did not create, that may not be amenable to our testing approaches, and that the test cannot inherently control.

That last part is particularly critical.  In TDD, we want to create unique, isolated tests that fail for one specific reason and thus specify one narrowly-defined behavior of the system.  Tests that involve multiple behaviors are hard to read (as a specification) and will have multiple reasons to fail.  When a test fails, we want to know unequivocally why it failed so that we can efficiently address the issue.

We write tests in TDD to specify the proper behavior of the system, allowing us to confidently create the right things.  But they also, later, serve as tests to ensure that defects have not been introduced.  In this second mode, what does the test actually test?

A test will always test everything that is in scope which it does not control.  If a test is to be narrowly focused on just one thing, then everything else that is in scope must be brought under the control of the test, otherwise it is testing those things as well.

Let's roll the dice and look at an example:

Most people are familiar with the game Yahtzee.  Briefly, you roll five dice and try to make the best pattern you can.  Examples are three of a kind, or numbers in a sequence (a "straight") and so forth, all the way up to a "yahzee" which means all five dice are the same.  "Chance" means you have no pattern at all, just five unrelated numbers.

public enum Result { CHANCE, ACES, TWOS, THREES, FOURS, PAIR,
                     THREEOFAKIND, FOUROFAKIND, FULLHOUSE,
                     SMALLSTRAIGHT, LARGESTRAIGHT, YAHTZEE }

public class Yahtzee
{
    public Result RollDice() {
        char[] dice = new char[5];
        Result myResult = Result.CHANCE;
        Random rand = new Random();

        dice[1] = (char)rand.Next(6);
        dice[2] = (char)rand.Next(6);
        dice[3] = (char)rand.Next(6);
        dice[4] = (char)rand.Next(6);
        dice[5] = (char)rand.Next(6);

        // Logic to determine the best result and set myResult

        return myResult;
    }
}

The idea here is to roll five dice and have the game tell you what the best pattern is that you can make from the five random results that you got.  The default is "Chance" unless something better can be made from the die rolls you got.

What we would want to specify here is that the logic (which is commented out for brevity) would correctly identify various patterns of die rolls.  If we rolled 4 5's, for example, it would identify it as Result.FOUROFAKIND even though it is also true that we have three of a kind.

The problem is that Random is in scope... we are using it.  But unless we bring it under the control of the test we are also testing Random, which is not what we want.  Also, we cannot predict what Random will do.  We could seed the Random class with a known value, but even so we are testing more than we wish to.  How can we truly bring random under control?

This same issue would exist in code that is dependent upon any component: a GPS module, the system clock, a network socket, etc...

A Design Principle

In seeking to ensure high-quality designs, we need standards or rubrics to apply to any proposed design.  We want to check out thinking, to make sure we're not fooling ourselves or missing anything.  One such rubric is this:

When examining an entity (class, method, whatever) in our system we ask: is this entity aware of the framework, or aware of the application logic?  If the answer is "both", then we seek some way to separate the two aspects of the entity from each other.

If you examine the code above you'll see that this game entity is aware of the framework (how you create a use the Random class) and also the application logic (the rules of this particular game).  This is a clue that we should reconsider the structure of our code.  Note that this concern also impacts the testability of the code because, as we've already noted, the test does not want to be vulnerable to the Random component.  We want the test to be solely concerned with the game logic.

A Testing Adapter

One way to bring the framework component under the control of the test is to wrap it in an adapter, and then mock the wrapper:

class DieRoller
{
    Random rand;
    public DieRoller() {
        rand = new Random();
    }

    public virtual char RollDie() {
        return (char)rand.Next(6);
    }
}

..and then change the product code to use this class instead of using the framework element directly.  For the test, we could mock [1] this class and inject the mock instead of the adapter, bringing the die rolled in each case under the control of the test.

One example:

class MockDieRoller : DieRoller {
    private char roll;

    public void setRoll(char aRoll) {
        roll = aRoll;
    }

    public override char RollDie() {
        return roll;
    }
}

An Endo Test

Creating an adapter class is a viable option when it comes to framework components, but it may seem like overkill in some cases.  When the issue in question is very simple, as in our example, you could also control the dependency through a simple technique called "endo-testing."

public class Yahtzee
{
    public Result RollDice()
    {
        char[] dice = new char[5];
        Result myResult = Result.CHANCE;

        dice[1] = rollDie();
        dice[2] = rollDie();
        dice[3] = rollDie();
        dice[4] = rollDie();
        dice[5] = rollDie();

        // Logic to determine the best result and set myResult

        return myResult;
    }

    protected virtual char rollDie() {
        return (char)(new Random().Next(6));
    }
}

All we have done is extracted the use of the Random class into a local, protected virtual method.  This is a very simple and quick refactor; virtually any decent IDE will do this for you.  The test will look like this:

[TestClass]
public class YahtzeeTest
{
    [TestMethod]
    public void TestGameResults(){
           
        TestableYahtzee myGame = new TestableYahtzee();
        // conduct the test against controllable results
    }

    private class TestableYahtzee : Yahtzee {
        private char roll;
        public void setRoll(char aRoll) {
            roll = aRoll;
        }

        protected override char RollDie() {
            return roll;
        }
    }
}

Now the test can control what dice are rolled and conduct all the various scenarios to ensure that the rules of the game are adhered to.  Also, we've satisfied our design principles by separating game logic and framework knowledge into two methods, rather than two classes.




[1] See our blogs and podcast on mocking for more details:
http://www.sustainabletdd.com/2012/05/mock-objects-part-1.html