Jupyter-notebook server
If you have a Linux VM, you can setup Jupyter-notebook server, then you can connect to the Jupyter-notebook running on the VM with a web browser and run Python and Scala code on the VM.
First, you need to use putty or ssh to the VM, you should already done below. If not, do it now from below link without skipping steps.
This is to assume you already have Anaconda python 3 installed on the VM and have created virtual environment spark as outlined here. If not, follow below link to install Anaconda Python 3 and create virtual environment spark, exactly instructed without skipping any steps:
To create Jupyter-notebook server, run below command:
1
jupyter notebook --generate-config
Copied!
Then
1
vi ~/.jupyter/jupyter_notebook_config.py
Copied!
search below parameter and set the value on the right side of the =, and remove # comment symbol at the beginning of the line, save and exit vi
1
c.NotebookApp.ip='0.0.0.0' #Permit external computer to connect
2
c.NotebookApp.notebook_dir = '' #Local notebook visit directory
3
c.NotebookApp.open_browser = False # Upon start, will not auto open a browser
4
c.NotebookApp.port = 8888 #Specify port
5
c.NotebookApp.allow_origin = '*'#Allow visit from anywhere
6
​
Copied!
Then set the password of jupyter-notebook server, run below command, enter your password, and confirm password
1
jupyter notebook password
2
​
Copied!
Then launch jupyter-notebook server by:
1
nohup jupyter-notebook &
Copied!
The port of jupyter-notebook server is 8888, as you have configured previously:
1
c.NotebookApp.port = 8888
Copied!
You need to setup port forwarding on the virtualbox, in my example, make host port 33 forwarding to guest port 8888
Finally, launch web-browser to connect to the Jupyter-notebook server at port 33, first time, you will be asked to enter jupyter password you have set previously
You now have successfully setup jupyter-notebook server, and connect to jupyter-notebook remotely with a web browser.
Next task is to add a spylon kernel to this jupyter-notebook server:
which in short, do below in a terminal window
1
conda activate spark
2
pip install spylon-kernel
3
#Create a kernel spec for Jupyter notebook
4
python -m spylon_kernel install --user
Copied!
Now restart jupyter-notebook in the terminal, if it is already running, you need to stop it first
1
nohup jupyter-notebook &
Copied!
Then open a web browser to point to host machine Virtualbox port forwarding port, in our case, port 33, which forwards to port 8888 of the guest machine that Jupyter-notebook server is listening to, click the notebook "New" for the drop down list, you will see Spylon-Kernel to run Scala code interactively
for example:
http://localhost:33
To stop jupyter-notebook server
jupyter-notebook server is essentially jupyter-notebook process running in background and there is no graceful way to stop it. Only way to stop it is by Linux command "kill -9"
First find out the process id (PID) by
1
ps -ef | grep jupyter
Copied!
Note down the <PID>s for all jupyter processes, then for each <PID>
1
kill -9 <PID>
Copied!
​
Last modified 1yr ago
Copy link