Home > Software & IT > Bugs? Features? Defects? Who Cares? I Do.

Bugs? Features? Defects? Who Cares? I Do.

2008.12.07

There was a bit of debate on the issue of bugs versus features I raised in Coding Horror-ibly.  I tried my best to keep it concise, but apparently, didn’t explain myself well.  It seems worthwhile to clarify, Mythbusters-style.

Myth 1: All defects are bugs

This Myth is FALSE.  However, the converse, all bugs are defects, is TRUE.

Defects can fall into a variety of categories, including:

  1. Defective product management
  2. Defective requirements / specification
  3. Defective coding
  4. Defective implementation

Bugs are synonymous with the third category – defective coding – either through mistranslation of requirements or syntactical or logical errors of the code itself.

As a result, there exists a semantic gap between the way users use this word (typically, synonymous with defect) and how programmers use this word (to mean a coding failure).  From this misunderstanding grave clashes often arise with devastating results.

Myth 2: Any time the software doesn’t do what the customer expects, it’s a defect.

This Myth is TRUE.

However, not everyone who purchases the software is a customer.

WTF you say?

Joe the Village Idiot wants a piece of software that will help him perform some useful business mathematics for his junk-removal business.  He wants to analyze some financial numbers arranged in columns, to understand their relative proportion to some other numbers arranged in a nearby column.  Hopefully the software will help him compute totals, averages, and ratios.  He’d also like it if the software would produce a presentation-quality pie chart.

After hasty and slipshod research, Joe purchased the software he thought would do the job best: Sybase e-Biz Impact.

Shortly after installing Sybase e-Biz Impact on his laptop, Joe the Village Idiot found himself unable to enter the numbers he collected into columns, so he called for assistance.  Following is an excerpt from the communication he had with the Sybase helpdesk:

Helpdesk:  Thank you for calling Sybase.  My name is Chandra.  May I please have your Sybase customer number or qualifying registered product serial number?

Joe TVI: (mumbling) The serial number is 443-A943F-1192.

Helpdesk:  Thank you, one moment please.  Am I speaking with Joe?

Joe TVI: Yep, I’m Joe.

Helpdesk: Thank you, Joe.  How may I be of assistance today.

Joe TVI: OK, I installed e-Biz Impact on my laptop today.  I’m reading the manual and looking through the online help, and I can’t find where to enter my sales figures.

Helpdesk: (without missing a beat) e-Biz Impact will run on a laptop, but it is a server product.  Do you have a server you can install e-Biz Impact on?

Joe TVI: No…

Helpdesk: No problem for now, but I’ll make a note in your customer file to have one of our configuration specialists give you a call.  Let’s continue.  You say you can’t find where to enter your sales figures?

Joe TVI: No…

Helpdesk: E-Biz Impact is a systems integration product for hospitals that supports the HL7 patient information integration standard.  It isn’t concerned with sales figures.  Are you trying to integrate a patient information system?

Joe TVI: I haul off junk from people’s front yards.  I don’t even have medical insurance.

Helpdesk: Hmmm…

At this point, Chandra the Sybase Helpdesk Guy is faced with a dilemma.  Does he

  1. Collect information about what Joe was trying to do with the product at the time of failure and submit an issue report to the software development team, or
  2. Politely tell Joe that perhaps e-Biz Impact isn’t the best product for his needs, and send him out to pick up a copy of Microsoft Excel?

Now, clearly, this is a very silly example.  But it is illustrative of the issue I’m trying to raise.  Not everyone who purchases your software is your customer.

The term “marketing” is used almost universally to refer to the targeting of advertising at potential customers to achieve greater sales.  However, the chief problem facing people who truly understand the true art of marketing is not how to sell more product to those people one has identified as potential customers.  No, the chief problem is how to clearly identify those people who are not potential customers.

Obviously, Joe is not a potential customer for Sybase e-Biz Impact.  But what about these potential customers:

  1. the user who would like to use Gmail as an RSS aggregator
  2. the user who would like to use Cakewalk Sonar as a MIDI lighting controller
  3. the user who would like to use Excel as a database middleware tool
  4. the user who would like to use Microsoft Word to create and edit web pages

Examples 1-3 are great examples of things that people might want to do with certain tools because they seem like other tools that do similar jobs.  Lots of email clients serve double-duty as RSS aggregators.  Why not Gmail?  Sonar is an excellent MIDI recorder, why not use it to run MIDI-controlled lighting systems?  Excel can connect to databases and has a programming language, so why not turn it into a piece of middleware?

The answer to these questions has to do with business and product management strategy, the underlying technical competencies of the tools, the current customer base of the product, and other factors.  The resulting answer is that these potential uses of the product are considered undesirable by the company – in other words, Gmail isn’t in the RSS aggregation business, Sonar isn’t in the lighting control business, and Excel isn’t in the integration business.  When users who are trying to use these products for these purposes discover flaws related to their application of the tool, they aren’t finding bugs.  They’re generating noise and should probably be given their money back and steered towards a different product.

Example 4 is a great example of what happens when a company fails to say “Microsoft Word is not a web page editor”.  Microsoft at some point decided to allow users to publish their Word documents as HTML, and hilarity and torture promptly ensued.  I hope you’ve never tried this.  It’s terrifying.  And the worst part is this: all the failures of Word to properly produce good looking and well-formed HTML are now bugs, because Microsoft created the expectation that Word should be used as an HTML authoring tool.

This is all just another way of saying just because you can doesn’t mean you should.

Clearly, if we can identify someone who is our archetypical user, we can produce a piece of software that conforms to their every need.  The problem isn’t finding this person.  They abound.  The problem is knowing who isn’t this person, and deciding not to meet their needs.

This leads us to our next myth:

Myth 3: Any time the software doesn’t do what the customer wants, it’s a defect.

This Myth is FALSE.

But how does this differ from Myth 2, which was TRUE?

It’s in the words “want” versus “expect”.

If you’re truly a customer, and you have expectations about the product, then chances are good that every failure to meet expectations is some kind of defect, although maybe not a coding defect (a.k.a. bug).  Something happened that either (1) incorrectly set expectations, a.k.a. bad customer communication, (2) incorrectly identified how to meet expectations, a.k.a. bad requirements / spec definition, or (3) attempted to meet expectations a certain way and failed, a.k.a. bug.

In fact, the exact definition of the difference between a “feature request” and a “defect” is the exact difference between what a customer wants and what a customer expects.  If you want it, but don’t expect it, it’s a feature.  If you expect it, but it isn’t there, it’s a defect.  If you expect it, and it was supposed to be there, but isn’t or doesn’t work, it’s a bug.

Myth 4: Customers don’t care about the distinction between bugs, defects, and feature requests

This Myth is FALSE.

To be sure, some customers don’t.  Some customers don’t care if their children get enough to eat.  But many customers do care about the difference.  The more vested the customer is in the quality of the product, the more the customer will care about the distinction.

Who, after all, are public beta testers, if not customers who care about the difference between bugs, defects, and feature requests?

Who, after all, are open source developers, if not customers who care about the difference between bugs, defects, and feature requests?

One thing I’ve learned is that there are always a set of leading edge customers – customers that get it.  These are the most important people in the world to listen to when it comes to designing a better product.  And let me tell you – these customers really, really care about the difference between bugs, defects, and feature requests.

Again – there are plenty of customers who don’t care about these distinctions.  But many do care.  And we should care.  And we should damn sure care when the customer cares.

Conclusion

If you read my blog, something should be apparent about my style of software project / product management: I’m terribly customer-focused.  And I’m terribly interested in identifying defects and understanding root causes.

As a result, I’m very likely to identify lots of things as defects that most software managers would never call defective.  Most software management is middle management, and does not want to be held responsible for anything if they can avoid it.  As Jeff pointed out in his misguided article, categorizing problems as feature requests is a way many managers and developers avoid responsibility.

But I’m not like that.  I say, call ‘em all defects unless they can be proven otherwise.

Of course I think some of this comes back to the fuzzy term “bug”.  When a developer says “bug” she usually means “broken code.”  I too subscribe to this definition – to me, a “bug” is a coding failure which includes broken code as well as bad requirements translation / implementation.  But when a customer uses the word “bug” she often means “something that doesn’t do what I expect” which isn’t necessarily a bug.

Perhaps a better solution to just dropping the whole distinction between “bugs” and “feature requests” would be to drop the term “bugs” in favor of the term “defects”.  Because while we can argue if a particular defect is a bug or not, we can’t argue that it’s a defect.

Not if we’re honest.

Advertisements
Categories: Software & IT Tags:
%d bloggers like this: