(this post originally written and published on September 17th, 2007)
I enjoyed Sam’s article and agree with most of it, but I’d like to take issue with two points. The first is the statement
Card sorting certainly can provide input into an organization system—what content goes together—and a labeling system—what to call things—but it’s got very little to do with a navigation system or a search system.
Taken at face value, the statement implies that a coherent organization of a body of information has little or no connection to navigating through the information. I respectfully disagree with that conclusion. Admittedly I’m not aware of any published studies specifically directed at the relationship between organizing and finding. However, my experience leads me to believe that organizing and categorizing a body of information according to the way people think about it will have a positive effect on their being able to navigate through it.
My second disagreement is with the statement
I’m no statistician, and neither are most usability professionals. So, frankly, I avoid statistical analysis methods that I cannot explain to others.
This is representative of a number of similar statements I’ve seen recently, directed against the use of Cluster Analysis and their attendant dendrograms (tree diagrams).
A dendrogram can simply be characterized as a kind of “average” of the way a group of participants have organized a set of content items in a card sorting task. Most people don’t have to be “statisticians” to understand the concept of an “average”. A dendrogram has all the advantages (e.g., one number or set of numbers used to represent something about a number of individuals) as well as all the disadvantages (e.g., no one individual’s behavior may match the average, exactly) of any average. For those reasons, we should never resort to an average as the sole source of information used to make decisions. However, that’s not a solid argument for avoiding quantitative methods altogether.
In addition, simply because a person doesn’t understand all of the calculations that go into the creation of a dendrogram, doesn’t mean they can’t find it a very useful tool in understanding the results. Perhaps a relevant analogy would be the pervasive use of weather maps.
In spite of the fact that dendrograms can look overwhelming they are very useful tools for making sense of a card sort. We will be writing more here soon about how to make the best use of your card sorting data.
