On a phase transition in general order spline regression
share
In the Gaussian sequence model Y= θ_0 + ε in R^n, we study the fundamental limit of approximating the signal θ_0 by a class Θ(d,d_0,k) of (generalized) splines with free knots. Here d is the degree of the spline, d_0 is the order of differentiability at each inner knot, and k is the maximal number of pieces. We show that, given any integer d≥ 0 and d_0∈{-1,0,...,d-1}, the minimax rate of estimation over Θ(d,d_0,k) exhibits the following phase transition: inf_θsup_θ∈Θ(d,d_0, k)E_θθ - θ^2 _d kloglog(16n/k), 2≤ k≤ k_0, klog(en/k), k ≥ k_0+1. The transition boundary k_0, which takes the form (d+1)/(d-d_0) + 1, demonstrates the critical role of the regularity parameter d_0 in the separation between a faster loglog(16n) and a slower log(en) rate. We further show that, once encouraging an additional 'd-monotonicity' shape constraint (including monotonicity for d = 0 and convexity for d=1), the above phase transition is eliminated and the faster kloglog(16n/k) rate can be achieved for all k. These results provide theoretical support for developing ℓ_0-penalized (shape-constrained) spline regression procedures as useful alternatives to ℓ_1- and ℓ_2-penalized ones.
READ FULL TEXT