union(otherStream)

union(otherStream)

Return a new DStream that contains the union of the elements in the source DStream and other DStream.
1
import org.apache.spark._
2
import org.apache.spark.streaming._
3
import org.apache.spark.streaming.StreamingContext._ // not necessary since Spark 1.3
4
​
5
// Create a local StreamingContext with two working thread and batch interval of 1 second.
6
// The master requires 2 cores to prevent from a starvation scenario.
7
​
8
val conf = new SparkConf().setMaster("local[2]").setAppName("NetworkWordCount")
9
sc.stop
10
val ssc = new StreamingContext(conf, Seconds(1))
11
val lines = ssc.socketTextStream("localhost", 9999)
12
val lines_another = ssc.socketTextStream("10.0.0.202", 9999)
13
val combo_lines=lines.union(lines_another)
14
​
Copied!
​
Last modified 1yr ago
Copy link