World Library  
Flag as Inappropriate
Email this Article

Isotonic regression


Isotonic regression

An example of isotonic regression

In numerical analysis, isotonic regression (IR) involves finding a weighted least-squares fit x\in \Bbb{R}^n to a vector a\in \Bbb{R}^n with weights vector w\in \Bbb{R}^n subject to a set of non-contradictory constraints of the kind x_i \ge x_j.

Such constraints define partial order or total order and can be represented as a directed graph G=(N,E), where N is the set of variables involved, and E is the set of pairs (i, j) for each constraint x_i \ge x_j. Thus, the IR problem corresponds to the following quadratic program (QP):

\min \sum_{i=1}^n w_i (x_i - a_i)^2 \text{subject to }x_i\ge x_j~ \text{ for all } (i,j)\in E.

In the case when G=(N,E) is a total order, a simple iterative algorithm for solving this QP is called the pool adjacent violators algorithm (PAVA). Best and Chakravarti (1990) have studied the problem as an active set identification problem, and have proposed a primal algorithm in O(n), the same complexity as the PAVA, which can be seen as a dual algorithm.[1]

IR has applications in statistical inference, for example, to fit of an isotonic curve to mean experimental results when an order is expected. A benefit of isotonic regression is that it does not assume any form for the target function, such as linearity assumed by linear regression.

Another application is nonmetric multidimensional scaling,[2] where a low-dimensional embedding for data points is sought such that order of distances between points in the embedding matches order of dissimilarity between points. Isotonic regression is used iteratively to fit ideal distances to preserve relative dissimilarity order.

Isotonic regression is also sometimes referred to as monotonic regression. Correctly speaking, isotonic is used when the direction of the trend is strictly increasing, while monotonic could imply a trend that is either strictly increasing or strictly decreasing.

Isotonic regression under the L_p for p>0 is defined as follows:

\min \sum_{i=1}^n w_i |x_i - a_i|^p \mathrm{subject~to~}x_i\ge x_j~ \text{ for all } (i,j)\in E.

Simply ordered case

To illustrate the above, let x_1 \leq x_2 \leq \ldots \leq x_n, and f(x_1) \leq f(x_2) \leq \ldots \leq f(x_n), and w_i \geq 0 .

The isotonic estimator, g^*, minimizes the weighted least squares-like condition:

\min_g \sum_{i=1}^n w_i (g(x_i) - f(x_i))^2

Where g is the unknown function we are estimating, and f is a known function.

Software has been developed in the R statistical package for computing isotone (monotonic) regression.[3]


  1. ^ Best, M.J.; & Chakravarti N. (1990). "Active set algorithms for isotonic regression; a unifying framework". Mathematical Programming 47: 425–439.  
  2. ^  
  3. ^ De Leeuw, Jan; K. Hornik; P. Mair (2009). "Isotone Optimization in R: Pool-Adjacent-Violators Algorithm (PAVA) and Active Set Methods". Journal of statistical software 32 (5): 1. 

Further reading

  • Robertson, T.; Wright, F. T.; Dykstra, R. L. (1988). Order restricted statistical inference. New York: Wiley.  
  • Barlow, R. E.; Bartholomew, D. J.; Bremner, J. M.; Brunk, H. D. (1972). Statistical inference under order restrictions; the theory and application of isotonic regression. New York: Wiley.  
  • Shively, T.S., Sager, T.W., Walker, S.G. (2009). "A Bayesian approach to non-parametric monotone function estimation".  
  • Wu, W. B.; Woodroofe, M.; & Mentz, G. (2001). "Isotonic regression: Another look at the changepoint problem". Biometrika 88 (3): 793–804.  
This article was sourced from Creative Commons Attribution-ShareAlike License; additional terms may apply. World Heritage Encyclopedia content is assembled from numerous content providers, Open Access Publishing, and in compliance with The Fair Access to Science and Technology Research Act (FASTR), Wikimedia Foundation, Inc., Public Library of Science, The Encyclopedia of Life, Open Book Publishers (OBP), PubMed, U.S. National Library of Medicine, National Center for Biotechnology Information, U.S. National Library of Medicine, National Institutes of Health (NIH), U.S. Department of Health & Human Services, and, which sources content from all federal, state, local, tribal, and territorial government publication portals (.gov, .mil, .edu). Funding for and content contributors is made possible from the U.S. Congress, E-Government Act of 2002.
Crowd sourced content that is contributed to World Heritage Encyclopedia is peer reviewed and edited by our editorial staff to ensure quality scholarly research articles.
By using this site, you agree to the Terms of Use and Privacy Policy. World Heritage Encyclopedia™ is a registered trademark of the World Public Library Association, a non-profit organization.

Copyright © World Library Foundation. All rights reserved. eBooks from World Library are sponsored by the World Library Foundation,
a 501c(4) Member's Support Non-Profit Organization, and is NOT affiliated with any governmental agency or department.