Issue might happen import PyArrow

Depending on your Linux OS, especially with Ubuntu, you might run into issue like, when you installed pyarrow by conda install
>>> import pyarrow
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/dv6/anaconda3/envs/spark/lib/python3.6/site-packages/pyarrow/__init__.py", line 54, in <module>
from pyarrow.lib import cpu_count, set_cpu_count
ImportError: /home/dv6/anaconda3/envs/spark/lib/python3.6/site-packages/pyarrow/lib.cpython-36m-x86_64-linux-gnu.so: undefined symbol: _ZNK5arrow5F.... B5cxx11Ev
>>> quit()
(spark) dv6@dv6:~$ conda uninstall pyarrow
Collecting package metadata (repodata.json): done
Solving environment: done
​
I have avoided the issue with pip install pyarrow --user, after conda uninstall
(spark) dv6@dv6:~$ conda uninstall pyarrow
Collecting package metadata (repodata.json): done
Solving environment: done
​
​
==> WARNING: A newer version of conda exists. <==
current version: 4.8.2
latest version: 4.8.3
​
Please update conda by running
​
$ conda update -n base -c defaults conda
​
​
​
## Package Plan ##
​
environment location: /home/dv6/anaconda3/envs/spark
​
removed specs:
- pyarrow
​
​
The following packages will be REMOVED:
​
pyarrow-0.11.1-py36hbbcf98d_1002
python_abi-3.6-1_cp36m
​
The following packages will be SUPERSEDED by a higher-priority channel:
​
ca-certificates conda-forge::ca-certificates-2020.4.5~ --> pkgs/main::ca-ce rtificates-2020.1.1-0
certifi conda-forge::certifi-2020.4.5.1-py36h~ --> pkgs/main::certi fi-2020.4.5.1-py36_0
openssl conda-forge::openssl-1.1.1f-h516909a_0 --> pkgs/main::opens sl-1.1.1f-h7b6447c_0
​
​
Proceed ([y]/n)? y
​
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
(spark) dv6@dv6:~$ pip install pyarrow --user
Collecting pyarrow
Downloading pyarrow-0.16.0-cp36-cp36m-manylinux2014_x86_64.whl (63.1 MB)
|████████████████████████████████| 63.1 MB 25 kB/s
Requirement already satisfied: numpy>=1.14 in ./anaconda3/envs/spark/lib/python3 .6/site-packages (from pyarrow) (1.18.1)
Requirement already satisfied: six>=1.0.0 in ./anaconda3/envs/spark/lib/python3. 6/site-packages (from pyarrow) (1.14.0)
Installing collected packages: pyarrow
Successfully installed pyarrow-0.16.0
(spark) dv6@dv6:~$ python -c "import pyarrow"
(spark) dv6@dv6:~$ python
Python 3.6.10 |Anaconda, Inc.| (default, Jan 7 2020, 21:14:29)
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
​