# Common Spark command line

#### $SPARK\_HOME/bin/spark-submit

command to submit Spark driver program (application) to run on Spark cluster

```
 $SPARK_HOME/bin/spark-submit --master spark://10.0.0.202:7077 --class org.apache.spark.examples.SparkPi $SPARK_HOME/examples/jars/spark-examples_2.11-2.4.4.jar
20/04/29 16:05:15 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Pi is roughly 3.1450957254786274

```

#### $SPARK\_HOME/bin/spark-shell

Command line interface to run Scala code with Spark.  It automatically creates a SparkSession called spark and SparkContext called sc after launching. To exit spark-shell, use command :quit

```
spark-shell
20/04/29 15:32:41 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Spark context Web UI available at http://master.hadoop.lan:4040
Spark context available as 'sc' (master = local[*], app id = local-1588199577814).
Spark session available as 'spark'.
Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /___/ .__/\_,_/_/ /_/\_\   version 2.4.4
      /_/

Using Scala version 2.11.12 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_191)
Type in expressions to have them evaluated.
Type :help for more information.

scala>

```

#### $SPARK\_HOME/bin/pyspark

Command line interface to run Python code with Spark, upon launching, it automatically creates SparkSession called spark and SparkContext called sc, it is Python interface, exit with quit()

```
Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /__ / .__/\_,_/_/ /_/\_\   version 2.4.4
      /_/

Using Python version 3.7.4 (default, Aug 13 2019 20:35:49)
SparkSession available as 'spark'.

```

#### $SPARK\_HOME/bin/spark-sql

A SQL client used to run SQL queries against HIVE tables through HIVE thrift server.

```
spark-sql
20/04/29 15:48:41 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Spark master: local[*], Application Id: local-1588200525594
spark-sql> show schemas;
default
jentekllc
Time taken: 3.895 seconds, Fetched 2 row(s)
spark-sql> select 1;
1
Time taken: 1.802 seconds, Fetched 1 row(s)
spark-sql>

```

#### $SPARK\_HOME/bin/run-example

Run built in scala example in $SPARK\_HOME/examples/src/main/scala/org/apache/spark/examples

```
$SPARK_HOME/bin/run-example SparkPi
20/04/29 15:55:03 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Pi is roughly 3.1335356676783386

```


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://george-jen.gitbook.io/data-science-and-apache-spark/spark-command-line.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.