Torturing The Data Til It Screams


We all have at least one in our office.  The guy who is quick to let you know that selling that call against Whole Foods Markets (WFMI) may be a mistake because supermarket stocks go higher on Thursdays after it rains in Akron, Ohio 77% of the time.  The Data Miners.

I’m cool with statistics junkies.  I love to hear stats of all kinds and certainly, the market produces so much data, there is no end to the amount of regressions and backtests that can be run.  On a purely intellectual level, these stats can be enjoyable and intriguing to hear, no matter how far out their connective conclusions may be.

That said, it is a major mistake to be confusing causation with correlation in many of these cases, which some quantitative analysts tend to do.  In fact, some quants have built whole funds or careers off of doing exactly this, finding relationships between data sets that just don’t really mean anything.

Jason Zweig, the Personal Finance columnist for the Wall Street Journal sat down with the author of a new book that, among other topics, touches on the subject of “stupid” data mining.  One example given is the belief that the Bangladesh butter industry’s output may have some sort of predictive powers over the S&P 500.  Yeah, I told you this stuff could get ridiculous.

Meanwhile, dozens — probably hundreds — of Web sites hawk “proprietary trading tools” and analytical “models” based on factors with cryptic names like McMillan oscillators or floors and ceilings.

There is no end to such rules. But there isn’t much sense to most of them either. An entertaining new book, “Nerds on Wall Street,” by the veteran quantitative money manager David Leinweber, dissects the shoddy thinking that underlies most of these techniques.

I’ll be interested to read this book, but regardless, I will still be a sucker for interesting correlatives…but that doesn’t mean I’ll feel compelled to invest based on them!


Nerds on Wall Street Author (WSJ)

Tags: , ,

This content, which contains security-related opinions and/or information, is provided for informational purposes only and should not be relied upon in any manner as professional advice, or an endorsement of any practices, products or services. There can be no guarantees or assurances that the views expressed here will be applicable for any particular facts or circumstances, and should not be relied upon in any manner. You should consult your own advisers as to legal, business, tax, and other related matters concerning any investment.

The commentary in this “post” (including any related blog, podcasts, videos, and social media) reflects the personal opinions, viewpoints, and analyses of the Ritholtz Wealth Management employees providing such comments, and should not be regarded the views of Ritholtz Wealth Management LLC. or its respective affiliates or as a description of advisory services provided by Ritholtz Wealth Management or performance returns of any Ritholtz Wealth Management Investments client.

References to any securities or digital assets, or performance data, are for illustrative purposes only and do not constitute an investment recommendation or offer to provide investment advisory services. Charts and graphs provided within are for informational purposes solely and should not be relied upon when making any investment decision. Past performance is not indicative of future results. The content speaks only as of the date indicated. Any projections, estimates, forecasts, targets, prospects, and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others.

Wealthcast Media, an affiliate of Ritholtz Wealth Management, receives payment from various entities for advertisements in affiliated podcasts, blogs and emails. Inclusion of such advertisements does not constitute or imply endorsement, sponsorship or recommendation thereof, or any affiliation therewith, by the Content Creator or by Ritholtz Wealth Management or any of its employees. Investments in securities involve the risk of loss. For additional advertisement disclaimers see here:

Please see disclosures here.

What's been said:

Discussions found on the web
  1. CI/CD commented on Feb 07

    … [Trackback]

    […] There you can find 86022 more Info to that Topic: […]