Isotonic regression

Isotonic regression

belongs to the family of regression algorithms. Formally isotonic regression is a problem where given a finite set of real numbers Y=y1,y2,...,yn representing observed responses and X=x1,x2,...,xn the unknown response values to be fitted finding a function that minimizes

with respect to complete order subject to x1≤x2≤...≤xn where wi are positive weights. The resulting function is called isotonic regression and it is unique. It can be viewed as least squares problem under order restriction. Essentially isotonic regression is a monotonic function best fitting the original data points.

import org.apache.spark.ml.regression.IsotonicRegression
// Loads data.
val dataset = spark.read.format("libsvm")
.load("file:///opt/spark/data/mllib/sample_libsvm_data.txt")
// Trains an isotonic regression model.
val ir = new IsotonicRegression()
val model = ir.fit(dataset)
println(s"Boundaries in increasing order: ${model.boundaries}n")
println(s"Predictions associated with the boundaries: ${model.predictions}n")
// Makes predictions.
model.transform(dataset).show(3)

/*
Output:
Boundaries in increasing order: [0.0,0.0]n
Predictions associated with the boundaries: [0.0,1.0]n
+-----+--------------------+----------+
|label|            features|prediction|
+-----+--------------------+----------+
|  0.0|(692,[127,128,129...|       0.0|
|  1.0|(692,[158,159,160...|       0.0|
|  1.0|(692,[124,125,126...|       0.0|
+-----+--------------------+----------+
only showing top 3 rows


*/