Statistical Modeling

Overview

Covariance and correlation estimation sit behind most risk and portfolio-construction work, and a sample covariance matrix estimated from a short window relative to the number of assets is dominated by estimation noise, not signal.

Core questions

• How many of a sample covariance matrix's eigenvalues carry genuine signal, versus noise consistent with a purely random matrix?
• How does regularizing that noise change a portfolio's out-of-sample risk?

Mathematical formulation

Marchenko–Pastur eigenvalue density

For a random matrix with aspect ratio q = N/T (N assets, T observations), this gives the density of eigenvalues expected from noise alone. Eigenvalues falling inside [λ₋, λ₊] are indistinguishable from a purely random correlation structure at this sample size; only eigenvalues above λ₊ are treated as signal.

Methods we use

• Shrinkage covariance estimation

  Ledoit, O., & Wolf, M. (2004). A well-conditioned estimator for large-dimensional covariance matrices. Journal of Multivariate Analysis, 88(2), 365–411.

• Random matrix theory and eigenvalue clipping

  Marchenko, V. A., & Pastur, L. A. (1967). Distribution of eigenvalues for some sets of random matrices. Mathematics of the USSR-Sbornik, 1(4), 457–483; and Laloux, L., Cizeau, P., Bouchaud, J.-P., & Potters, M. (1999). Noise dressing of financial correlation matrices. Physical Review Letters, 83(7), 1467–1470.

• Eigenvalue concentration as a systemic-risk measure

  Kritzman, M., Li, Y., Page, S., & Rigobon, R. (2011). Principal components as a measure of systemic risk. Journal of Portfolio Management, 37(4), 112–126.

Open problems

• What shrinkage target is appropriate when the number of assets exceeds the estimation window by a wide margin?
• How stable is an estimated correlation structure across a volatility-regime change?