Development environment on MacOS

Pr-requisites:

If there is connection refused error, you need to properly address the port connection issue by configure firewall under
system preferences -> security and privacy->firewall

Install XCode

If your MacOS is latest release, you can install XCode from App Store; otherwise, you can download and install from developer site for older version of XCode that is compatible to your MacOS

Install homebrew (or brew)

Open the Terminal, enter below
1
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
Copied!
Then follow the message on the screen

Download and install JDK 8

same process for Windows, except you need to download the dmg file for the MacOS
After install, you can find JAVA_HOME on your Mac by below command line
1
$(dirname $(readlink $(which javac)))/java_home
Copied!
On my MacBook, it displays java home path
1
/Library/Java/JavaVirtualMachines/jdk1.8.0_202.jdk/Contents/Home
Copied!
Append into ~/.bash_profile on my Mac
1
export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0_202.jdk/Contents/Home
Copied!

Download setup Eclipse IDE with Scala plug in

Install download utility wget

1
brew install wget
Copied!

Download and install Anaconda Python 3

Download and install Hadoop

Download and install HIVE

Download and install Spark

Install Python module findspark and Jupyter Spylon-kernel for Scala

​
Last modified 1yr ago
Copy link