Setup TensorFlow, Keras, Theano, Pytorch/torchvision on the CentOS VM
We need to use Keras with TensorFlow as backend for some deep learning projects in the Spark platform we practice data science. The platform we have been using is a CentOS VM running on software based hypervisor called VirtualBox.
Additionally, for future deep learning projects that may use the dataset with Apache Spark, we need to install Theano, Pytorch and Torchvision.
We already have Python 3.6 under virtual environment on Anaconda. Here are the steps:
Enter virtual environment spark
1
conda activate spark
Copied!
Upgrade pip
1
pip install pip --upgrade
2
​
3
#Collecting pip
4
# Downloading pip-20.1.1-py2.py3-none-any.whl (1.5 MB)
5
# |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1.5 MB 2.5 MB/s
6
#Installing collected packages: pip
7
# Attempting uninstall: pip
8
# Found existing installation: pip 20.0.2
9
# Uninstalling pip-20.0.2:
10
# Successfully uninstalled pip-20.0.2
11
#Successfully installed pip-20.1.1
Copied!
Install TensorFlow
1
conda install -c conda-forge tensorflow
2
​
3
#Collecting package metadata (current_repodata.json): done
4
#Solving environment: done
5
#
6
## Package Plan ##
7
#.....
8
# environment location: /opt/hadoop/anaconda3/envs/spark
9
​
10
# added / updated specs:
11
# - tensorflow
12
#....
13
#Preparing transaction: done
14
#Verifying transaction: done
15
#Executing transaction: done
Copied!
Verify TensorFlow installed successfully
1
python -c "import tensorflow"
Copied!
Install Keras
1
conda install -c conda-forge keras
2
#Collecting package metadata (current_repodata.json): done
3
#Solving environment: done
4
​
5
## Package Plan ##
6
​
7
# environment location: /opt/hadoop/anaconda3/envs/spark
8
​
9
# added / updated specs:
10
# - keras
11
​
12
​
13
# The following packages will be downloaded:
14
​
15
# package | build
16
# ---------------------------|-----------------
17
# keras-2.4.3 | py_0 30 KB conda-forge
18
# ------------------------------------------------------------
19
# Total: 30 KB
20
​
21
#The following NEW packages will be INSTALLED:
22
​
23
# keras conda-forge/noarch::keras-2.4.3-py_0
24
​
25
​
26
# Proceed ([y]/n)? y
27
​
28
​
29
#Downloading and Extracting Packages
30
#keras-2.4.3 | 30 KB | ################################################################################################################## | 100%
31
#Preparing transaction: done
32
#Verifying transaction: done
33
#Executing transaction: done
Copied!
Verify Keras is installed successfully
1
python -c "import keras"
Copied!
Install Theano
1
conda install -c anaconda theano
2
#Collecting package metadata (current_repodata.json): done
3
#Solving environment: done
4
​
5
## Package Plan ##
6
​
7
# environment location: /opt/hadoop/anaconda3/envs/spark
8
​
9
# added / updated specs:
10
# - theano
11
​
12
#...
13
​
14
#Preparing transaction: done
15
#Verifying transaction: done
16
#Executing transaction: done
Copied!
Install Pytorch
1
conda install -c anaconda pytorch
2
#Collecting package metadata (current_repodata.json): done
3
#Solving environment: done
4
​
5
## Package Plan ##
6
​
7
# environment location: /opt/hadoop/anaconda3/envs/spark
8
​
9
# added / updated specs:
10
# - pytorch
11
​
12
#...
13
#Preparing transaction: done
14
#Verifying transaction: done
15
#Executing transaction: done
Copied!
Install Torchvision
1
conda install -c torchvision
2
#Collecting package metadata (current_repodata.json): done
3
​
4
## Package Plan ##
5
​
6
# environment location: /opt/hadoop/anaconda3/envs/spark
7
​
8
# added / updated specs:
9
# - torchvision
10
​
11
#...
12
#Preparing transaction: done
13
#Verifying transaction: done
14
#Executing transaction: done
Copied!
Verify all installation
1
python
2
Python 3.6.10 |Anaconda, Inc.| (default, Mar 25 2020, 23:51:54)
3
[GCC 7.3.0] on linux
4
Type "help", "copyright", "credits" or "license" for more information.
5
>>> import tensorflow
6
>>> import keras
7
>>> import theano
8
>>> import torch
9
>>> import torchvision
10
>>> quit()
Copied!
Last modified 1yr ago
Copy link