Launch custom built Docker container with docker-compose

Launch custom built Docker container with docker-compose

1
docker-compose up --scale spark-worker=3
Copied!
It is better to run as background task to free up the terminal by:
1
nohup docker-compose up --scale spark-worker=3 &
Copied!
​
The above commands starts up Spark cluster docker container with 1 spark master node and 3 spark worker nodes
The worker nodes are the ones that run the Spark jobs
Launch a web browser, connect to Master node <IP address>:8080
The IP address can be the localhost or the IP address of the server that runs the docker
For all the docker containers used in the class, they are available for download
Last modified 1yr ago
Copy link