For the complete documentation index, see llms.txt. This page is also available as Markdown.

SparkSession

The entry point into all functionality in Spark SQL is the SparkSession class. If you use spark-shell, SparkSession is automatically created for you.

Otherwise, you will need to create SparkSesison:

import org.apache.spark.sql.SparkSession
val spark = SparkSession
    .builder
    .appName("SQL example")
    .master("local[*]")
    .config("spark.sql.warehouse.dir", "file:///d:/tmp")
    .getOrCreate()

SparkSession is required for runing Spark SQL query.

Last updated