Net Objectives

Net Objectives
If you are interested in coaching or training in ATDD or TDD please click here.

Friday, March 2, 2012

ATDD and TDD

Download the Podcast Part 1
Download the Podcast Part 2



A question that we are often asked is: “What is the difference between Acceptance Test Driven Development (ATDD) and Test Driven Development (TDD)?” These two activities are related by name but otherwise seem to have little to do with each other. 

ATDD is a whole-team practice where the team members discuss a requirement and come to an agreement about the acceptance criteria for that requirement. Through the process of accurately specifying the acceptance criteria -- the acceptance test -- the team fleshes out the requirement, discovering and corroborating the various assumptions made by the team members and identifying and answering the various questions that, unanswered, would prevent the team from implementing or testing the system correctly.

The word acceptance is used in a wide sense here:

  • The customer agrees that if the system, which the team is about to implement, fulfills the acceptance criteria then the work was done properly
  • The developers accept the responsibility for implementing the system
  • The testers accept the responsibility for testing the system

This is a human-oriented interaction that focuses on the customer, identifying their needs. These needs are specified using the external, public interfaces of the system. 

TDD, on the other hand is a developer-oriented activity designed to assist the developers in writing the code by strict analysis of the requirements and the establishment of functional  boundaries, work-flows, significant values, and initial states. TDD tests are written in the developer’s language and are not designed to be read by the customers. These tests can use the public interfaces of the system, but are also used to test internal design elements. 

We often see the developers take the tests written through the ATDD process and implement them with a unit testing framework.

Requirements from the customer

Before we continue, we need to ask ourselves -- what is a requirement? It is something that the customer needs the system to do. But who is the customer? 

In truth, every system has more than one customer... dozens at times:

  • Stakeholders
  • End users, of different types
  • Operators
  • Administrators (DB, network, user, storage)
  • Support (field, customer, technical)
  • Sales, marketing, legal, training
  • QA and developers (e.g., traces and logs, simulators for QA)
  • etc...
All requirements coming from all of these different customers must be addressed, identified and expressed through the ATDD process. For example:

  • The legal department needs an End User Legal Agreement (EULA) to be displayed when the software is first run, and for the end user to check off the agreement before the system can be used.  This is of no interest to the end users (who we sometimes think of as ‘the customers’), in fact might be an annoyance to them, but is required for the system to be acceptable to the lawyers.
  • The production support team needs all error messages in the system to be accompanied by error codes that can be reported along with the condition that caused the error.  Here again, end users are not interested in these codes, but they can be crucial for the system to be acceptably supported.

And let us not forget the the developers are customers too, who else do we build tracers and loggers for? This is an obvious, publicly visible facet of the developer’s work. But when do we need these facilities? When we try to fix bugs. When we want to understand how the system works. When we work on the system for any reason.  

In other words, when we do maintenance to the system.

Maintainability is a requirement

We need our maintenance to be as easy as possible. No car owner would like to disassemble the car’s engine just to change a windshield wiper; nor would they want to worry that by changing a tire they have damaged the car’s entertainment system. 

Maintainability is a crucial requirement for any software system. Software system maintenance should be fast, safe and predictable. You should be able to make a change fast, without breaking anything, and you need to be able to tell me reliably how long it will take. We expect this of our car mechanic as well as our software developer. So although maintainability is primarily the concern of the developer it definitely affects the non-technical customers. 

The way maintainability manifests itself in software is through design. Design principles are to developers as mathematics is to physicists. It’s the basis of everything that we do. If we do not pay attention to the system’s design as it is developed,it will quickly become unmanageable. 

How often, however, have you seen “maintainability” as a requirement? We’ve never seen it. We call it the “hidden requirement.” It’s always there but no one talks about it. And because we don't talk about it, we forget it about it; we focus on fulfilling the written requirements thinking that we will be done when we complete them. And very quickly, the system turns very hard and unsafe to change.  We are accumulating technical debt, which we could just call “the silent killer.” 

If maintainability is such a crucial requirement, where is the acceptance criteria for it? Who is the customer for this requirement? The development team.  

We need to prove to the customer that the design as was perceived was implemented, and that this design is in fact maintainable, that the correct abstractions exist, that object factories do what they are supposed to, that the functional units operate the way they should, etc... 

Indeed there is something that we can do, in the developers’ own language -- code -- that does precisely these things. It’s TDD. 

One key purpose of TDD is to prove and document the design of the system, hence proving and documenting its maintainability.

TDD is developer-facing ATDD


ATTD is about the acceptability of the system to its various customers.  When the specific customer is the development team then the tests are about the acceptability of the system’s design and resulting maintainability.  Our focus in this work is acceptability in this sense: is the design acceptable?  Is our domain understanding sufficient and correct?  Have we asked enough questions, and were they the right ones?  A system that fails to meet these acceptance criteria will quickly become too expensive to maintain and thus will fail to meet the needs of those who use it. 

Software that fails to meet a need is worthless.  It dies.  So, here again, failing to pass the “maintainability” acceptance criteria is the silent killer.  TDD is the answer to this ailment. 

Note to readers: This was a philosophical treatise. Specific, practical examples abound and will constitute much of our work here, so, read on.


16 comments:

  1. Okay. I definitely agree with the fundamental tenet that unit test driven development is a way for developers to enforce their standards and ensure a product stays maintainable going forward. I'm going to challenge the "requirement" thing just because... well... because that's what I do.

    I think that, given current trends, putting maintainability up on the requirement pedestal actually may cut away from your goal rather than toward it. The way most people see requirements right now is as things that are weighed against other requirements. You can choose to have this requirement now or that one, then you can have the other one usually after you finish the one on which you choose do to first.

    That's not how maintainability works. Getting it back once you choose not to have it in the first place comes at great expense so we don't ever want to give someone the chance to choose against it.

    Acceptance criteria for a "real" requirement, however work exactly as you describe. The product people can choose whether or not to create a piece of value for the end users of a product but they cannot have it unless the implementation satisfies the legal people, the user experience people, the operations people, and the people who are going to have to maintain the code.

    Note that this is really very consistent with what you said in this post with the only difference being the label that gets applied to maintainability.

    ReplyDelete
  2. I will let Scott chime in as well, but I feel that the sorry state of software development nowadays stems exactly from the attitude that maintainability is not a requirement.

    All requirements are statements of change -- "I want to change X in my system so that I will benefit thusly..."
    When it comes to maintainability the statement is "I want to change the design of the system after I have made all functional changes in it so that I will be able to make the next changes just as easily and safely."

    The developer's work is not complete unless he address this "silent requirement." It is not optional. It is REQUIRED.

    ReplyDelete
  3. Looking forward to read the specific part, especially your take on sustainability of TDD vs sustainability of ATDD (if you plan to cover it). From my amateur point of view, this is a little different, because in case of TDD, I want to attain sustainability by proper encapsulation and introducing "seams" to let me break dependencies. On the other hand, in ATDD (especially in tools using cucumber or other such language) the focus is on step reuse and encapsulating each step as reusable part of code (because I will end up performing some steps in almost every acceptance test), whereas in developer TDD, I want to limit it as much as possible (i.e. when I think about creating new helper method or using Setups, the first question I ask myself is "am I not trying to encapsulate in testing code what I failed to encapsulate properly in production code?" and only if the answer is yes, I would go and create this helper).

    ReplyDelete
  4. I think Max's point was that *other* things we call "requirements" can be de-prioritized and may even be eliminated if the customer decides they are not worth doing, but that maintainability must never be left out. Therefore, the term "requirement" may not be *strong* enough for maintainability. In other words, I think he is saying what you (Amir) are saying, that maintainability is truly required whereas other things we call requirements may come and go. Is that what you meant, Max?

    ReplyDelete
  5. By not treating it as a requirement it moves into the world of the "stuff that the developers do that we don't quite understand but they probably do because they like to play with the system rather than do real work."

    On the contrary - it is a non-negotiable requirement. A *REAL* requirement, as in the adjective - REQUIRED.

    ReplyDelete
  6. My point was that I think Max and you are like minded in saying maintainability is not optional.

    ReplyDelete
  7. Thanks for the post, and I am excited about your podcast versions -- but the iTunes link isnt working for me (says it's not available in the US), and it'd be ideal to just have an RSS/ATOM feed just for the podcasts. Would this be possible? Thanks!

    ReplyDelete
  8. You can download the podcasts directly from us, by clicking on the "podcasts" tab above, then right click/save-as any of the 'casts you want...

    ReplyDelete
  9. Except (Oops) we don't have this podcast up yet. Jim Trott is still producing it. Stay tuned!

    ReplyDelete
  10. Question I have....
    If maintainability is a requirement, how do you demonstrate you product covers and fulfill this requirement?
    Above all when you have a 1st initial release/version.

    I mean, you can know if the product you developed can cover/or not maintainability aspects after several releases/versions but I would like to know how do you demonstrate it contains that property (maintainability) in the very first initial release?

    On the other hand, a product can be very maintainable in some modules/use cases/scenarios while in other scenarios can be very difficult, or perhaps the effort to completely cover the acceptance of new capabilities/functions by the customer can be very large even if yo feel or perceive the code you wrote is well designed, so in that case how can you decide or judge if that module (or product) covers well the maintainability requirement? by just looking at the effort required to change a piece of code?

    ReplyDelete
  11. Maintainability is all about dealing with change. If we encapsulate the change, then the rest of our system will not be affected by it. We can achieve maintainability by anticipating all possible changes to our code and created the ultimate encapsulation. This, however is an unlikely endeavor as there is ALWAYS something new to change.
    So the maintainability of code depends on two things:
    1. The current design
    2. Our skill in modifying that design to accommodate new change.

    The ease with which we modify the extant design to accommodate new change (aka Emergent Design) relates to the clarity and appropriateness of the existing design.

    So, given a design, how do we know it is implemented correctly - that is, that every class and method is doing what it was supposed to do? And after writing the code, how can we understand what the classes are doing, so that we can change where change is called for?

    The answer - TDD. When you are writing the 'tests' first, you are in fact documenting what each entity in your design is supposed to do, which is what we are expanding on in this blog.

    ReplyDelete
  12. When you are adding the podcasts into the blog posts RSS feed, could you please add the podcasts as an "enclosure", instead of just an html link?

    This will allow Android podcast players such as BeyondPod to recognize and download the podcasts for offline listening.

    See this forum post for a brief explanation:

    http://www.beyondpod.com/forum/showthread.php?107-Podcast-RSS-feed-download-not-recognized&p=304&viewfull=1#post304

    For a good example see:

    feed://feeds.feedburner.com/NetObjectivesPodcast_LAST

    For a bad example see:

    feed://feeds.feedburner.com/NetObjectivesPodcast_LASTgoofb

    View the XML source to see the difference.

    (if Jim and Alan can add "enclosure" podcasts back into their new feed too, that would also be nice).

    Thanks!

    ReplyDelete
    Replies
    1. We'll look into it, Chris. Thanks for the suggestion.

      Delete
    2. Chris:

      From what I can tell, Blogger does generate the sort of feed you're looking for. Have you tried pointing BeyondPod to:

      http://www.sustainabletdd.com/feeds/posts/default
      -or-
      http://www.sustainabletdd.com/feeds/posts/default?alt=rss

      The first is an Atom feed (RSS 1.0) whereas the second is RSS 2.0.

      Let me know if this helps, or if I am still missing the point. :)

      Delete
    3. Scott,

      Nope, the current Blogger atom and rss feeds don't appear to work.

      There used to be a setting in Blogger to turn on link fields and then to include an enclosure link, but that may have been removed as part of recent redesign "improvements"
      http://blogging.nitecruzr.net/2011/12/new-gui-does-not-support-podcasting.html

      Could you try to set up a feedburner feed with smartcast and see if it will wrap the appropriate link with an enclosure tag so that other non-iTunes podcatchers will work?

      You may also need to create a separate blog entry if you have multiple parts to the podcast, as most podcatchers tend to only grab the first one from a feed entry.

      Thanks!

      Delete
    4. We also have a feedburner feed:

      http://feeds.feedburner.com/SustainableTest-drivenDevelopment

      Perhaps this has the proper enclosures? It is how we end up on iTunes.

      Delete