Dev environment setup, task list

Dev environment setup, task list

Install Java devilment toolkit 1.8 (required, higher version Java may not work properly with Spark, we found out jdk1.8.0_191 works best, therefore, recommend jdk1.8.0_191. (Do NOT install JDK 9 or above, Spark is not currently compatible with JDK 9 or above.)
Download and install Anaconda Python and create virtual environment with Python 3.6 (work best with most of the deep learning libraries)
Download and install Spark
Download and install Scala (optional, as Spark has scala)
Setup Spylon-kernel on jupyter-notebook, we will be using jupyter-notebook for both Python and Scala
Setup Elipcse Scala IDE

I have detailed video presentation on how to the developer workstation on Windows 10

Last modified 1yr ago
Copy link