# Development environment on MacOS

### Pr-requisites:

*If there is connection refused error, you need to properly address the port connection issue by configure firewall under* 

*system preferences -> security and privacy->firewall*

#### Install XCode

If your MacOS is latest release, you can install XCode from App Store; otherwise, you can download and install from developer site for older version of XCode that is compatible to your MacOS

<https://stackoverflow.com/questions/44390162/download-install-xcode-version-without-premium-developer-account>

#### Install homebrew (or brew)

Open the Terminal, enter below

```
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
```

Then follow the message on the screen

#### Download and install JDK 8

same process for Windows, except you need to download the dmg file for the MacOS

<https://www.oracle.com/technetwork/java/javase/downloads/java-archive-javase8-2177648.html>

After install, you can find JAVA\_HOME on your Mac by below command line

```
$(dirname $(readlink $(which javac)))/java_home
```

On my MacBook, it displays java home path

```
/Library/Java/JavaVirtualMachines/jdk1.8.0_202.jdk/Contents/Home
```

Append into \~/.bash\_profile on my Mac

```
export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0_202.jdk/Contents/Home
```

#### Download setup Eclipse IDE with Scala plug in

[Same process as Windows install](https://george-jen.gitbook.io/data-science-and-apache-spark/scala-ide)

#### Install download utility wget

```
brew install wget
```

#### Download and install Anaconda Python 3

[Same process as windows](https://george-jen.gitbook.io/data-science-and-apache-spark/conda_setup6)

#### Download and install Hadoop

[Same process as Linux install](https://george-jen.gitbook.io/data-science-and-apache-spark/hadoop-configuration)

#### Download and install HIVE

[Same process as Linux install](https://george-jen.gitbook.io/data-science-and-apache-spark/install-hive)

#### Download and install Spark

[Same process as Linux install](https://george-jen.gitbook.io/data-science-and-apache-spark/setup-apache-spark)

#### Install Python module findspark and Jupyter Spylon-kernel for Scala

[Same process as Windows Install](https://george-jen.gitbook.io/data-science-and-apache-spark/install-findspark-add-spylon-kernel-for-scala)


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://george-jen.gitbook.io/data-science-and-apache-spark/development-environment-on-macos.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.