Bucketing, Sorting and Partitioning

For file-based data source, it is also possible to bucket and sort or partition the output. Bucketing and sorting are applicable only to persistent tables:
peopleDF.write.bucketBy(42, "name").sortBy("age").saveAsTable("people_bucketed")
while partitioning can be used with both save and saveAsTable when using the Dataset APIs.
It is possible to use both partitioning and bucketing for a single table:
.bucketBy(42, "name")
partitionBy creates a directory structure as described in the Partition Discovery section. Thus, it has limited applicability to columns with high cardinality.
bucketBy distributes data across a fixed number of buckets and can be used when a number of unique values is unbounded.