Entering Docker Container
Once the docker is running, you can show docker container info by:
docker ps
1
docker ps
2
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
3
bb5da52bef7d spark_lab/spark:latest "/start-worker.sh" 18 minutes ago Up 18 minutes 0.0.0.0:32772->8080/tcp docker_dir_spark-worker_2
4
0a16636c3895 spark_lab/spark:latest "/start-worker.sh" 18 minutes ago Up 18 minutes 0.0.0.0:32773->8080/tcp docker_dir_spark-worker_1
5
9753b4389785 spark_lab/spark:latest "/start-master.sh" 18 minutes ago Up 18 minutes 0.0.0.0:7077->7077/tcp, 0.0.0.0:8080->8080/tcp spark-master
6
​
Copied!
You can enter the container by:
docker exec -it [container id] bash
1
#You are in host machine
2
docker exec -it 9753b4389785 bash
3
#You are in container, exit to return to host
4
bash-4.4# ls
5
bin home mnt root spark start-worker.sh usr
6
dev lib opt run srv sys var
7
etc media proc sbin start-master.sh tmp
8
​
9
bash-4.4# spark/bin/spark-shell
10
20/04/23 01:18:00 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
11
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
12
Setting default log level to "WARN".
13
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
14
Spark context Web UI available at http://spark-master:4040
15
Spark context available as 'sc' (master = local[*], app id = local-1587604698244).
16
Spark session available as 'spark'.
17
Welcome to
18
____ __
19
/ __/__ ___ _____/ /__
20
_\ \/ _ \/ _ `/ __/ '_/
21
/___/ .__/\_,_/_/ /_/\_\ version 3.0.0-preview2
22
/_/
23
​
24
Using Scala version 2.12.10 (OpenJDK 64-Bit Server VM, Java 1.8.0_212)
25
Type in expressions to have them evaluated.
26
Type :help for more information.
27
​
28
scala>
29
​
Copied!
​
Last modified 1yr ago
Copy link