Development environment on MacOS

Pr-requisites:

If there is connection refused error, you need to properly address the port connection issue by configure firewall under

system preferences -> security and privacy->firewall

Install XCode

If your MacOS is latest release, you can install XCode from App Store; otherwise, you can download and install from developer site for older version of XCode that is compatible to your MacOS

https://stackoverflow.com/questions/44390162/download-install-xcode-version-without-premium-developer-account

Install homebrew (or brew)

Open the Terminal, enter below

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Then follow the message on the screen

Download and install JDK 8

same process for Windows, except you need to download the dmg file for the MacOS

https://www.oracle.com/technetwork/java/javase/downloads/java-archive-javase8-2177648.html

After install, you can find JAVA_HOME on your Mac by below command line

$(dirname $(readlink $(which javac)))/java_home

On my MacBook, it displays java home path

/Library/Java/JavaVirtualMachines/jdk1.8.0_202.jdk/Contents/Home

Append into ~/.bash_profile on my Mac

export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0_202.jdk/Contents/Home

Download setup Eclipse IDE with Scala plug in

Same process as Windows install

Install download utility wget

brew install wget

Download and install Anaconda Python 3

Same process as windows

Download and install Hadoop

Same process as Linux install

Download and install HIVE

Same process as Linux install

Download and install Spark

Same process as Linux install

Install Python module findspark and Jupyter Spylon-kernel for Scala

Same process as Windows Install