1 Answers
In constrained least squares one solves a linear least squares problem with an additional constraint on the solution. I.e., the unconstrained equation X β = y {\displaystyle \mathbf {X} {\boldsymbol {\beta }}=\mathbf {y} } must be fit as closely as possible while ensuring that some other property of β {\displaystyle {\boldsymbol {\beta }}} is maintained.
There are often special-purpose algorithms for solving such problems efficiently. Some examples of constraints are given below:
If the constraint only applies to some of the variables, the mixed problem may be solved using separable least squares by letting X = {\displaystyle \mathbf {X} =} and β T = {\displaystyle \mathbf {\beta } ^{\rm {T}}=} represent the unconstrained and constrained components. Then substituting the least-squares solution for β 1 {\displaystyle \mathbf {\beta _{1}} } , i.e.
back into the original expression gives an equation that can be solved as a purely constrained problem in β 2 {\displaystyle \mathbf {\beta } _{2}}.