Docker deployment of Spark Cluster

Docker deployment of Spark Cluster

Docker Install, for our purpose, we only need to install Docker Community Edition (CE)

Get Docker CE for CentOS:

https://docs.docker.com/v17.09/engine/installation/linux/docker-ce/centos/docs.docker.com

Get Docker CE for Debian:

https://docs.docker.com/v17.09/engine/installation/linux/docker-ce/debian/docs.docker.com

Get Docker CE for Ubuntu:

https://docs.docker.com/v17.09/engine/installation/linux/docker-ce/ubuntu/#set-up-the-repositorydocs.docker.com

Docker for Windows 10:

https://docs.docker.com/v17.09/docker-for-windows/install/#download-docker-for-windowsdocs.docker.com

Docker for Mac:

https://docs.docker.com/v17.09/docker-for-mac/install/docs.docker.com

For others:

https://docs.docker.com/v17.09/engine/installation/#serverdocs.docker.com

Non Windows:

After Docker is installed, need to do the following to allow non root user to run Docker container

sudo groupadd docker

sudo usermod -aG docker $USER

exit and log in again

Install docker-compose:

Install Docker ComposeDocker Documentation