Uncommon penalties for common problems

Penalties are great tools to steer parametric and nonparametric models in desired directions. Familiar examples are the ridge and least absolute shrinkage and selection operator penalty to stabilize regression and roughness penalties to fit smooth trends. The purpose of this paper is to showcase some uncommon extensions to these penalties that help to solve common problems. Switching from the usual sum of squares in the penalty to sums of smaller powers can achieve very sparse results. Asymmetric penalties make it easy to enforce shape constraints. In some cases it is effective to work with a sum of 2 signals, each having its own “tailored” penalty to adapt it to specific properties of the data. Many examples, in 1 and 2 dimensions, mostly based on real data, illustrate the (moderate) theory.

• View at Publisher
• View at Scopus