On interpreting data

Yesterday's New York Times carried a story reminding me again that when working with numbers, you can't look at them just once. The story, titled, "On Economy, Raw Data Gets a Grain of Salt," by Binyamin Appelbaum, explains how the federal government's Bureau of Economic Analysis changed its estimate of the growth rate of the economy for the first quarter of 2011. Instead of growing at a rate of 1.8%, the government now reports that the economy grew only the at the much smaller rate of 0.4% during that quarter.

The reason? The Bureau of Economic Analysis changed the value of vehicles awaiting sales at dealerships. In the initial reports, these numbers had been projections, not actual counts; those come later, and are starting to be available now. The change in that one number thus had a profound impact on our understanding of what happened earlier this year.

A couple of lessons:

   * In this kind of analysis, you're using certain numbers to signal or represent what's happening in the larger world (in this case, the economy). If you do use an index number like this, you need to revisit it periodically, to be sure it's representing the information you think it should.

   * It's important to understand when a number is solid (or as solid as possible) and when it is an estimate. As Appelbaum says in the story, "[P]oliticians and investors are placing a great deal of weight on a crude and rough estimate that has never been particularly reliable." Back in January, I wrote a blog post about a very useful book called Proofiness: The Dark Arts of Mathematical Deception, about the habit of succumbing to the authority of numbers. You can read the post here, but better yet, read the book!

Economic numbers may be particularly opaque to interpret, but the story has some relevance to anyone using numbers to manage and understand their business, whether it's for-profit or not-for-profit. Using numbers is an iterative process; you need to examine your numbers and think about what you're doing over and over again.

I'm back briefly between trips, and wanted to get this post up. More to come next week.