Predicting neural scaling-law exponents from properties of the data

For a deep network trained in the feature-learning regime on natural data, the test loss follows $L($$N$, $D$$) = L_{\infty} + A N^{-\beta_N} + B D^{-\beta_D}$ over several decades, with N parameters and D training examples or tokens. Derive $\beta_N, \beta_D$ and the compute-optimal allocation exponent from measurable statistics of the data distribution (for example the power-law decay exponent of the data covariance spectrum or of the target function's spectral decomposition). An answer is a theory that predicts measured exponents on a held-out dataset and architecture within quoted errors, or a proof that no data statistic of this kind fixes them.

A theory whose predicted $\beta_{N}$ and $\beta_{D}$ match exponents measured on several independent datasets and architectures trained in the feature-learning regime.