When a researcher conducts multiple analyses, and from them he reports only the best outcome, that finding is more likely to be false than if a single analysis would have been conducted. In the statistics literature, this problem is known as “selection bias under multiple testing” (SBuMT). The key to addressing SBuMT is to disclose the intermediate results that the researcher has discarded. With that information, it is possible to evaluate the probability that the best outcome is actually false, as a result of multiple testing. In this paper, we present a real example of how multiple testing information can be reported. We use that information to estimate the Deflated Sharpe Ratio of an investment strategy.
The information contained herein is only as current as of the date indicated, and may be superseded by subsequent market events or for other reasons. The views and opinions expressed herein are those of the author and do not necessarily reflect the views of AQR Capital Management, LLC, its affiliates or its employees. This information is not intended to, and does not relate specifically to any investment strategy or product that AQR offers. It is being provided merely to provide a framework to assist in the implementation of an investor’s own analysis and an investor’s own view on the topic discussed herein. Past performance is not a guarantee of future results.