Graphx

Basic components of Graph computing:

Graph essentially consistes Vertex, Edge and Triplet, each is an implementation of RDD, therefore, methods that are applied to a RDD can be applied to Vertex RDD, Edge RDD and Triplet RDD. If you want, for example, you can convert Vetex RDD, Edge RDD and Triplet RDD to Spark SQL dataframe and use SQL to manipulate them.

Vetex, Edge and Triplet

Vertex is a 2 element tuple: (vertex id, attribute)

1
graph.vertices.take(5).foreach(println)
2
​
3
/*
4
output:
5
(34,34.0)
6
(52,52.0)
7
(96,96.0)
8
(4,4.0)
9
(16,16.0)
10
​
11
*/
Copied!

Edge is a case class, or structure (like a struct in C): Edge(source vertex id, destination vertex id, Edge attribute)

1
graph.edges.take(5).foreach(println)
2
/*
3
Output:
4
Edge(0,2,1)
5
Edge(0,3,1)
6
Edge(0,6,1)
7
Edge(0,7,1)
8
Edge(0,13,1)
9
*/
Copied!

Triplet is a 3 element nested tuple (tuple in tuple): (source Vertex, destination Vertex, Edge attribute)

1
graph.triplets.take(5).foreach(println)
2
/*
3
Output:
4
((0,0.0),(2,2.0),1)
5
((0,0.0),(3,3.0),1)
6
((0,0.0),(6,6.0),1)
7
((0,0.0),(7,7.0),1)
8
((0,0.0),(13,13.0),1)
9
*/
Copied!

Graphx contains the classes and others in below packages:

Last modified 1yr ago