Machine Learning

A Data Science Solution to the Multiple-Testing Crisis in Financial Research

Topics - Machine Learning Behavioral Finance

${ numberSection } ${ text }
A Data Science Solution to the Multiple-Testing Crisis in Financial Research

When a researcher conducts multiple analyses, and from them he reports only the best outcome, that finding is more likely to be false than if a single analysis would have been conducted. In the statistics literature, this problem is known as “selection bias under multiple testing” (SBuMT). The key to addressing SBuMT is to disclose the intermediate results that the researcher has discarded. With that information, it is possible to evaluate the probability that the best outcome is actually false, as a result of multiple testing. In this paper, we present a real example of how multiple testing information can be reported. We use that information to estimate the Deflated Sharpe Ratio of an investment strategy.

Published in

The Journal of Financial Data Science

AQR Capital Management, LLC, (“AQR”) provide links to third-party websites only as a convenience, and the inclusion of such links does not imply any endorsement, approval, investigation, verification or monitoring by us of any content or information contained within or accessible from the linked sites. If you choose to visit the linked sites, you do so at your own risk, and you will be subject to such sites' terms of use and privacy policies, over which AQR.com has no control. In no event will AQR be responsible for any information or content within the linked sites or your use of the linked sites.

 

The information contained herein is only as current as of the date indicated, and may be superseded by subsequent market events or for other reasons. The views and opinions expressed herein are those of the author and do not necessarily reflect the views of AQR Capital Management, LLC, its affiliates or its employees. This information is not intended to, and does not relate specifically to any investment strategy or product that AQR offers. It is being provided merely to provide a framework to assist in the implementation of an investor’s own analysis and an investor’s own view on the topic discussed herein. Past performance is not a guarantee of future results.

 

Hypothetical performance results have many inherent limitations, some of which, but not all, are described herein. Hypothetical performance results are presented for illustrative purposes only.

 

Diversification does not eliminate the risk of experiencing investment loss.

 

Certain publications may have been written prior to the author being an employee of AQR.

This material is intended for informational purposes only and should not be construed as legal or tax advice, nor is it intended to replace the advice of a qualified attorney or tax advisor.