index
If you are using windows, and you do not have winzip or winrar installed, there are free alternatives of decompressing software that can expand tgz compressed file, that is needed to unpack Spark downloaded tgz file.
For windows, it is needed to setup for Hadoop, by download winutils.exe
Specifically, you can just download below exe file
By now, you have downloaded and extracted Spark, and winutils.exe (Hadoop utility for windows), next task is to:
Set up SPARK_HOME environment variable to point to home dir of Spark, in our case:
SPARK_HOME=c:\spark\spark
Append %SPARK_HOME%\bin to the PATH environment variable
Set up HADOOP_HOME environment variable to point to Hadoop home dir, in our case:
HADOOP_HOME=c:\winutils
Append %HADOOP_HOME%\bin to the PATH environment variable
Set up default /tmp/hive directory that Spark needs. This means, in Windows, for example, you need to create a folder for example
Open a cmd command window as administrator
mkdir c:\tmp\hive
Then set permission by
%HADOOP_HOME\bin\winutils.exe chmod –R 777 c:\tmp
Also point %TEMP% and %TMP% to c:\tmp
Then you are done with Spark setup.
Last updated
