repartition(numPartitions)

Changes the level of parallelism in this DStream by creating more or fewer partitions.

//Default partition is number of CPU "cores (threads really)" in the machine that runs Spark
getSomeData.rdd.getNumPartitions
val rePartition = getSomeData.repartition(numPartitions = 6)

Previousfilter(func)Nextunion(otherStream)

Last updated 4 years ago