Turning Data Into Knowledge
A glance at my investment portfolio will quickly reveal that I have no real skill in picking stocks, but a recent Wall Street Journal article by Jason Zweig on data mining and stock picking provides an ideal opportunity to highlight our work here at TargetPoint Consulting.
As you likely already know, TargetPoint was an innovator of data mining voter files to predict voter behaviors and attitudes. Initially, our techniques were met with skepticism by some in the political marketplace. After all, how could we rely on computers to mine millions or billions of relationships and discover meaningful patterns? TargetPoint was fortunate to come along at a time where:
- There was a new richness and growing quantity of data available,
- We were just reaching the technical ability to store massive amounts of data and retrieve it quickly,
- The hardware and software required to mine data became reasonably inexpensive.
But as important as any of these factors was what TargetPoint added to the equation – knowledge.
It is knowledge that tells one not to rely on “butter production in Bangladesh” to pick stocks (a data-mining fallacy highlighted in the aforementioned WSJ article).
In fact, our internal discussions surrounding the recent redesign of the TPC website highlighted for us the importance of knowledge to our product, and ultimately led to our new tagline: “Data >> Knowledge >> Solutions.” (Of course, it also led to me writing blogs). While data alone might tell stock buyers to watch butter production in Bangladesh, I don’t think any knowledge would support that.
We have purposefully assembled a team of people with a range of experiences and different knowledge bases in order to fully leverage diverse, collective knowledge. And while we use the best data mining software available, it is the addition of our team’s knowledge that sets our product apart.
In his article, Zweig writes:
“That points to the first rule for keeping yourself from falling into a data mine: The results have to make sense. Correlation isn't causation, so there needs to be a logical reason why a particular factor should predict market returns. No matter how appealing the numbers may look, if the cause isn't plausible, the returns probably won't last.”
That “first rule” is something we live by. The computer cannot tell the difference in value between a 2004 Bush hard ID from a flag in the database that says a person owns a cat – but we can. And it is years and years of experience and innumerable data sequences analyzed and reviewed within these files that allow us to keep building knowledge and improving what we do.
Finally, it’s important to highlight that the book reviewed by Zweig, "Nerds on Wall Street" by David Leinweber, and it’s criticism of financial “data mining” is not meant to be a condemnation of all statistical data modeling; Leinweber has in fact had a successful career in quantitative investing. Rather, he appears to be providing a cautionary tale about how poor data-mining can go terribly bad. On the whole it appears to be an interesting book, and I look forward to reading to it.
- Brent Seaborn
Comments on this entry