A Gaussian Mixture Model
A Gaussian Mixture Model represents a composite distribution whereby points are drawn from one of k Gaussian sub-distributions, each with its own probability. The spark.ml implementation uses the expectation-maximization algorithm to induce the maximum-likelihood model given a set of samples.
GaussianMixture is implemented as an Estimator and generates a GaussianMixtureModel as the base model.
A Gaussian Mixture Model represents a composite distribution whereby points are drawn from one of k Gaussian sub-distributions, each with its own probability. The spark.ml implementation uses the expectation-maximization algorithm to induce the maximum-likelihood model given a set of samples.
import org.apache.spark.ml.clustering.GaussianMixture // Loads data val dataset = spark.read.format("libsvm").load("/opt/spark/data/mllib/sample_kmeans_data.txt") // Trains Gaussian Mixture Model val gmm = new GaussianMixture() .setK(2) val model = gmm.fit(dataset) // output parameters of mixture model model for (i <- 0 until model.getK) { println("weight=%fnmu=%snsigma=n%sn" format (model.weights(i), model.gaussians(i).mean, model.gaussians(i).cov)) }
1
import org.apache.spark.ml.clustering.GaussianMixture
2
// Loads data
3
val dataset = spark.read.format("libsvm").load("file:///opt/spark/data/mllib/sample_kmeans_data.txt")
4
// Trains Gaussian Mixture Model
5
val gmm = new GaussianMixture()
6
.setK(2)
7
val model = gmm.fit(dataset)
8
// output parameters of mixture model model
9
for (i <- 0 until model.getK) {
10
println("weight=%fnmu=%snsigma=n%sn" format
11
(model.weights(i), model.gaussians(i).mean, model.gaussians(i).cov))
12
}
13
​
14
/*
15
Output:
16
weight=0.500000nmu=[0.10000000000001552,0.10000000000001552,0.10000000000001552]nsigma=n0.006666666666806454 0.006666666666806454 0.006666666666806454
17
0.006666666666806454 0.006666666666806454 0.006666666666806454
18
0.006666666666806454 0.006666666666806454 0.006666666666806454 n
19
weight=0.500000nmu=[9.099999999999984,9.099999999999984,9.099999999999984]nsigma=n0.006666666666812185 0.006666666666812185 0.006666666666812185
20
0.006666666666812185 0.006666666666812185 0.006666666666812185
21
0.006666666666812185 0.006666666666812185 0.006666666666812185 n
22
​
23
*/
24
​
Copied!
​
Last modified 1yr ago
Copy link