In the first article in this series, Test-Driven Development with Python, Part 1, I started to build an event tracking application using the principle of test-driven development. Before writing any new code or making changes to any existing code, I wrote a failing test first. When the new test started passing, I stopped and moved on. It's a simple technique, but requires discipline in order to apply it successfully.
If you haven't read part one yet, I suggest you do so before continuing on in this article. That will familiarize you with the problem I'm trying to solve and bring you up to speed on my current predicament.
As indicated in the previous article, I'm starting to think that the single
DatePattern class I have so far is responsible for too much. It's
only 30 lines long at this point, but I don't mean its length when I say "too
much." What I mean is, conceptually, it does four different things; I even
started adding a fifth before I stopped myself.
What are its four different responsibilities? It has to determine if the year matches. That's one. The month is another, as are the day and the weekday, too. Can I break those responsibilities into four different classes? I'm sure I can, but should I?
Not only that, but for each component of the pattern, I have to check to see if it's a wild card and act appropriately. Maybe I need yet another class to encapsulate that logic.
I can't answer these questions by just thinking about them, so I will explore by writing a few tests:
class NewTests(unittest.TestCase):
def testYearMatches(self):
yp = YearPattern(2004)
d = datetime.date(2004, 9, 29)
self.failUnless(yp.matches(d))
Note that I've put this new test case in a new fixture. If this turns out to
be a dead end, I can easily delete this fixture and continue onward with the
old. If this ends up being a good idea, I can just as easily delete the old
fixture (and the code it exercised, the DatePattern class).
This new test case, of course, fails. Here's the code required to make it pass:
class YearPattern:
def __init__(self, year):
pass
def matches(self, date):
return True
Boring. How about another test?
def testYearDoesNotMatch(self):
yp = YearPattern(2003)
d = datetime.date(2004, 9, 29)
self.failIf(yp.matches(d))
Now to make the new test pass without breaking the old one:
class YearPattern:
def __init__(self, year):
self.year = year
def matches(self, date):
return self.year == date.year
Perfect.
What's the point? Before I can show you that, I need to write a few more tests:
def testMonthMatches(self):
mp = MonthPattern(9)
d = datetime.date(2004, 9, 29)
self.failUnless(mp.matches(d))
def testMonthDoesNotMatch(self):
mp = MonthPattern(8)
d = datetime.date(2004, 9, 29)
self.failIf(mp.matches(d))
def testDayMatches(self):
dp = DayPattern(29)
d = datetime.date(2004, 9, 29)
self.failUnless(dp.matches(d))
def testDayDoesNotMatch(self):
dp = DayPattern(28)
d = datetime.date(2004, 9, 29)
self.failIf(dp.matches(d))
def testWeekdayMatches(self):
wp = WeekdayPattern(2) # Wednesday
d = datetime.date(2004, 9, 29)
self.failUnless(wp.matches(d))
def testWeekdayDoesNotMatch(self):
wp = WeekdayPattern(1) # Tuesday
d = datetime.date(2004, 9, 29)
self.failIf(wp.matches(d))
Here's the code to make these tests pass:
class MonthPattern:
def __init__(self, month):
self.month = month
def matches(self, date):
return self.month == date.month
class DayPattern:
def __init__(self, day):
self.day = day
def matches(self, date):
return self.day == date.day
class WeekdayPattern:
def __init__(self, weekday):
self.weekday = weekday
def matches(self, date):
return self.weekday == date.weekday()
Is it obvious where I'm going with this yet? If not, this test should make it clear:
def testCompositeMatches(self):
cp = CompositePattern()
cp.add(YearPattern(2004))
cp.add(MonthPattern(9))
cp.add(DayPattern(29))
d = datetime.date(2004, 9, 29)
self.failUnless(cp.matches(d))
What I've stumbled across here is an instance of the Composite Pattern. (Interestingly, this is the same name as my class--that wasn't intentional, I promise.)
|
A composite is basically an object that contains other objects, where both the composite object and its contained objects all implement the same interface. Using the interface on the composite should invoke the same methods on all of the contained objects without forcing the external client to do so explicitly. Whew, that was a mouthful.
Here, that interface is the matches method, which accepts a
date instance and returns a bool. Python is a
dynamically typed language, so I don't need to define this interface formally
(which is fine by me).
How do I implement the composite? Like this:
class CompositePattern:
def __init__(self):
self.patterns = []
def add(self, pattern):
self.patterns.append(pattern)
def matches(self, date):
for pattern in self.patterns:
if not pattern.matches(date):
return False
return True
The composite pattern asks each of its contained patterns if it matches the specified date. If any fail to match, the whole composite pattern fails.
I have to confess that I cheated here. I wrote more code than I needed to pass the test! Sometimes I get ahead of myself. Sorry. It turned out OK this time because all of the tests are passing, but I need to create a test that should not match, just to be sure I have everything working correctly:
def testCompositeDoesNotMatch(self):
cp = CompositePattern()
cp.add(YearPattern(2004))
cp.add(MonthPattern(9))
cp.add(DayPattern(28))
d = datetime.date(2004, 9, 29)
self.failIf(cp.matches(d))
Cool. It passes.
It might be a little difficult to see this, but the composite contains a
DayPattern that matches the 28th and I'm matching it against the
29th, which is why I expect the matches method to return
False.
So I can match dates again. Big deal--I was already doing that. What about wild cards?
I'll write a test to match my anniversary with the new classes:
def testCompositeWithoutYearMatches(self):
cp = CompositePattern()
cp.add(MonthPattern(4))
cp.add(DayPattern(10))
d = datetime.date(2005, 4, 10)
self.failUnless(cp.matches(d))
It just works. Why?
There's no YearPattern in the composite requiring the passed-in
date to match any specific year. Wild cards now work by not specifying
any pattern for a given component. Remember when I thought I might need a
class to do the wild card matching? I was wrong!
At this point, I feel really good about the new approach and will just delete the old tests and code.
I'll also refactor the tests a bit. Did you notice that every one of the new tests contained a duplicate line? I did. It started to bother me, but that's what test fixtures are for:
class PatternTests(unittest.TestCase):
def setUp(self):
self.d = datetime.date(2004, 9, 29)
def testYearMatches(self):
yp = YearPattern(2004)
self.failUnless(yp.matches(self.d))
def testYearDoesNotMatch(self):
yp = YearPattern(2003)
self.failIf(yp.matches(self.d))
I've only shown the first two test cases (in the fixture previously known as
NewTests) but now all of the test cases refer to the date as
self.d instead of constructing a local date instance.
It's not a huge refactoring, but it makes me feel better. You do want me to
feel the best I can about my code, don't you? Of course you do.
I did have to change testCompositeWithoutYearMatches to use
this date instead of my anniversary. As cute as it was to throw that date in
there, I decided I'd rather have clean code without duplication than
cuteness.
I also took this opportunity to add some named constants for weekdays:
MONDAY, TUESDAY, WEDNESDAY, THURSDAY, FRIDAY, SATURDAY, SUNDAY = range(0, 7)Now I can use these instead of the hard-coded constants I had in the weekday tests, and also delete the comments I had explaining what the constants represented. Intention-revealing code beats comments any day of the week.
Where am I now? Before switching gears, I planned to write a test for a pattern that matched the last Thursday of every month. It's time to do that now:
def testLastThursdayMatches(self):
cp = CompositePattern()
cp.add(LastWeekdayPattern(THURSDAY))
self.failUnless(cp.matches(self.d))
Cool. A new class to implement!
The implementation for this class is slightly more complicated than the others:
class LastWeekdayPattern:
def __init__(self, weekday):
self.weekday = weekday
def matches(self, date):
nextWeek = date + datetime.timedelta(7)
return self.weekday == date.weekday() and nextWeek.month != date.month
Oops. It doesn't pass. Why?
The date I'm trying to match in the test is a Wednesday, not a Thursday! I
need to fix the test (not forgetting to rename it) to add a test where I expect
the match to fail (which I should have done before implementing
matches):
def testLastWednesdayMatches(self):
cp = CompositePattern()
cp.add(LastWeekdayPattern(WEDNESDAY))
self.failUnless(cp.matches(self.d))
def testLastWednesdayDoesNotMatch(self):
cp = CompositePattern()
cp.add(LastWeekdayPattern(WEDNESDAY))
self.failIf(cp.matches(self.d))
Rats. The first test passes but the second one fails. The date created in
setUp is the same for every test case so it will always be a
Wednesday, but I need a date that's not on a Wednesday to make this test pass.
Rather than creating a new date in this test case (and ignoring the one created
in setUp), I'll move both of these tests into a new fixture--one
specific for testing the LastWeekdayPattern class:
class LastWeekdayPatternTests(unittest.TestCase):
def setUp(self):
self.pattern = LastWeekdayPattern(WEDNESDAY)
def testLastWednesdayMatches(self):
lastWedOfSep2004 = datetime.date(2004, 9, 29)
self.failUnless(self.pattern.matches(lastWedOfSep2004))
def testLastWednesdayDoesNotMatch(self):
firstWedOfSep2004 = datetime.date(2004, 9, 1)
self.failIf(self.pattern.matches(firstWedOfSep2004))
The Open-Closed PrincipleIsn't it nice being able to add new functionality without changing existing
classes? This is part of what Bertrand Meyer called the Open-Closed Principle:
"Software entities (classes, modules, functions, etc.) should be open for
extension, but closed for modification." With the new approach of using the
Composite pattern, I can extend the behavior of the system by writing a new
class with a |
Now they both pass and the tests use everything they create. Nice.
While moving these tests into a new fixture, I also noticed that I created a
CompositePattern that contained only one pattern. That's kind of
pointless, so I stopped doing it.
Should I move the test cases that exercise the various pattern classes into their own fixtures? That's a tendency that many, including me, often have. It's sometimes useful to resist that urge, though. As Dave Astels, author of Test-Driven Development: A Practical Guide, puts it: a fixture is "a way to group tests that need to be set up in exactly the same way." In other words, a fixture is not a container for all of the tests for a single class, or at least, it doesn't have to be.
Having said that, I prefer it when all of the test cases in a fixture exercise the same class. In harmony with Dave's definition of fixtures, I just don't require that all of the test cases that exercise the same class be in the same fixture. Make sense?
Suppose that I have four test cases for the class Foo but half
require different setUp code than the other half. I'd split those
cases up into two fixtures, even though they both exercise the same class. If I
then started writing tests for the class Bar and discovered that some
of its tests could use the same setUp code as one of the class
Foo fixtures, I would not just shove those tests into one
of the existing fixtures.
Wouldn't that mean I've duplicated the setUp code for the two
fixtures? Duplication is evil! If I thought the duplication was enough to be a
problem, I would Extract
Superclass the duplicated code out of the two fixtures. Yes, you can--and
should--refactor your tests, too.
When starting on a new project, I create one fixture with no
setUp method and add all of my test cases to that one fixture.
Eventually, I reach the point where I need to refactor the fixture, and I do it.
Remember: do the simplest thing that could possibly work first. Then refactor
if necessary.
What, though, is the benefit of ensuring that all of the test cases in a fixture only exercise one class? Well, besides making it more cohesive (and obeying the Single Responsibility Principle (PDF)), think about what might happen when you decide a class is no longer necessary. You'll need to delete the tests for that class, too. It's a lot easier to delete a whole test fixture than to look at each test case in a fixture to see if it exercises the class you just deleted.
Think you won't delete classes? Think again. You saw me delete the
DatePattern class and all of its tests earlier, didn't you? It
wasn't hard. I felt good about it, too.
|
Because it's so easy and fun, I want to add another pattern:
class NthWeekdayPatternTests(unittest.TestCase):
def setUp(self):
self.pattern = NthWeekdayPattern(1, WEDNESDAY)
def testMatches(self):
firstWedOfSep2004 = datetime.date(2004, 9, 1)
self.failUnless(self.pattern.matches(firstWedOfSep2004))
def testNotMatches(self):
secondWedOfSep2004 = datetime.date(2004, 9, 8)
self.failIf(self.pattern.matches(secondWedOfSep2004))
I don't have an example of this in my use cases as listed at the beginning of this article, but it's a feature that both calendar and pal support, so I expected to add it at some point.
Making these tests pass shouldn't be too hard:
class NthWeekdayPattern:
def __init__(self, n, weekday):
self.n = n
self.weekday = weekday
def matches(self, date):
if self.weekday != date.weekday():
return False
n = 1
while True:
previousDate = date - datetime.timedelta(7 * n)
if previousDate.month == date.month:
n += 1
else:
break
return self.n == n
OK, it was harder than I thought. I'm not a huge fan of the way that algorithm looks, but it's OK for now. I really should at least extract it into its own method so I can give it an intention-revealing name:
def matches(self, date):
if self.weekday != date.weekday():
return False
return self.n == self.getWeekdayNumber(date)
def getWeekdayNumber(self, date):
n = 1
while True:
previousDate = date - datetime.timedelta(7 * n)
if previousDate.month == date.month:
n += 1
else:
break
return n
Better.
The last example I do have at the beginning of this article is the "last day
of the month" case. That should match days "in reverse." I could modify the
existing DayPattern class, but instead I want to add a new pattern
class:
class LastDayInMonthPatternTests(unittest.TestCase):
def testMatches(self):
lastDayInSep2004 = datetime.date(2004, 9, 30)
pattern = LastDayInMonthPattern()
self.failUnless(pattern.matches(lastDayInSep2004))
While typing in that test, I decided that I couldn't think of a reason to
support the second-to-last day in a month, or the third-to-last day, and so on.
Can you? I made it easy on myself and decided to implement a class called
LastDayInMonthPattern. People usually argue that writing tests up
front takes too much work, but writing a test first this time actually saved me
from writing code I would never use!
The implementation of this new pattern is:
class LastDayInMonthPattern:
def matches(self, date):
tomorrow = date + datetime.timedelta(1)
return tomorrow.month != date.month
Nice.
Although I just realized I'm cheating again. Here's how the fixture should
have looked (after extracting out the setUp method) before fully
implementing the matches method:
class LastDayInMonthPatternTests(unittest.TestCase):
def setUp(self):
self.pattern = LastDayInMonthPattern()
def testMatches(self):
lastDayInSep2004 = datetime.date(2004, 9, 30)
self.failUnless(self.pattern.matches(lastDayInSep2004))
def testNotMatches(self):
secondToLastDayInSep2004 = datetime.date(2004, 9, 29)
self.failIf(self.pattern.matches(secondToLastDayInSep2004))
I feel pretty good about myself right now. That usually means it's time to refactor. Now I really want to do some renaming.
For the last pattern I implemented, I was very explicit about what it did:
LastDayInMonthPattern only matches the last day in a month and
there's no further clarification needed. What about
NthWeekdayPattern and LastWeekdayPattern? I really
want to add InMonth to the end of both of those class names. Yes,
I'm that picky.
|
Related Reading 
|
I also took this time to rename a few of the test cases and reorder some of the class definitions. I won't bore you with the details, but you can see the final results for yourself if you download the code at the end of the article.
This type of tidying up may seem trivial but it's extremely important. If your code doesn't look clean, you (and others who find themselves working on your code) won't have any incentive to keep it clean. The Pragmatic Programmers call this the Broken Window Theory. If you live with broken windows, don't be surprised when your neighbors start using your lawn as a junk yard.
I have about 60 non-blank lines of code so far, spread across eight classes. That's not too much, but the code is very simple and yet highly flexible. I seriously doubt I would have been able to conceive of this design without writing my tests first.
What's even cooler is that I have about 90 non-blank lines of test code. Yes, I have more test code than I have "real" code. Is that wrong? Absolutely not. That's wonderful! I feel extremely confident about the quality of the code that I have so far. Is it perfect? I doubt it. When I discover a bug, though, I can add a new test to demonstrate it and fix it so that it never happens again. If I need to perform an optimization, I'll have a suite of tests I can use to verify that I didn't screw anything up while applying the optimization.
What's also interesting to note is that the design that emerged from this work. I spent zero time in front of a modeling tool trying to create a design that would both meet my needs today and still be elegant enough to (hopefully) meet all of tomorrow's needs, as well. I didn't intend for this to happen. It just magically happened that way. This isn't rare--this almost always happens when I do test-driven development.
How is this design more flexible than I originally intended? Suppose that I want to create a pattern that matches every Friday the 13th. That wasn't one of my original use cases, and I gave no thought to it while writing the tests. The classes I came up have no trouble representing that pattern:
>>> import DatePatterns
>>> fri13 = DatePatterns.CompositePattern()
>>> fri13.add(DatePatterns.WeekdayPattern(DatePatterns.FRIDAY))
>>> fri13.add(DatePatterns.DayPattern(13))
>>> import datetime
>>> aug13 = datetime.date(2004, 8, 13)
>>> aug13.strftime('%A')
'Friday'
>>> fri13.matches(aug13)
True
>>> sep13 = datetime.date(2004, 9, 13)
>>> sep13.strftime('%A')
'Monday'
>>> fri13.matches(sep13)
FalseWhile I'm not done with the application yet, I do have a solid foundation to build on. Next, I need to add some parsing code so that I can read a file containing events in order to construct and use the patterns I implemented above. I'll visit that task next time.
The code and tests are available for download and inspection.
Jason Diamond is a consultant specializing in C++, C#, and XML, and is located in sunny Southern California.
Return to the Python DevCenter.
Copyright © 2009 O'Reilly Media, Inc.