Skip to content

Latest commit

 

History

History
87 lines (75 loc) · 3.14 KB

INSTALL_NewRiver.md

File metadata and controls

87 lines (75 loc) · 3.14 KB

NewRiver

TensorFlow

  1. Connect to a GPU node.
  2. Install Anaconda python of your choice.
  3. module purge
  4. module load cuda/8.0.44
  5. Download cudnn from here.
  6. Add LD_LIBRARY_PATH to your .bashrc file. Jinwoo's example:
# set the server name
serv_name=$(hostname)

if [[ $serv_name == *"hu"* ]];
then
    # for PowerAI (huckleberry)
    # added by Miniconda2 4.3.14 installer
    export PATH="/home/jinchoi/pkg/miniconda2/bin:$PATH"
else
    # for newriver
    # added by Anaconda2 4.4.0 installer
    export PATH="/home/jinchoi/pkg/anaconda2_nr/bin:$PATH"
    export LD_LIBRARY_PATH=/home/jinchoi/lib/cuda/lib64:$LD_LIBRARY_PATH
fi
  1. source ~/.bashrc so that the os can locate your cudnn directory.
  2. Follow the official TensorFlow installation procedure provided here.
  3. Enjoy!

Install Custom TensorFlow

  1. (Optional) Create a virtual environment, assume that you have installed Anaconda/Miniconda
# Create the environment
conda create python=$VERSION -n $NAME

# Enter the environment
source activate $NAME

replace $VERSION with 2.7 or 3.6, and set a $NAME for your environment

  1. Add the following lines to your ~/.bashrc file to load CUDA and CUDNN
#TF version >= 1.5.0
NVCC=/opt/apps/cuda/9.0.176/bin
export LD_LIBRARY_PATH=/opt/apps/cuda/9.0.176/lib64:$LD_LIBRARY_PATH
CUDA_PATH=/opt/apps/cudnn/7.1
export LD_LIBRARY_PATH=/opt/apps/cudnn/7.1/lib64:$LD_LIBRARY_PATH

## If you want to use older version, you can use the CUDA 8.0
#NVCC=/opt/apps/cuda/8.0.61/bin
#export LD_LIBRARY_PATH=/opt/apps/cuda/8.0.61/lib64:$LD_LIBRARY_PATH
## I am not sure if this cudnn corresponds to CUDA 8.0. If not, you can download it and set the path
#CUDA_PATH=/opt/apps/cudnn/6.0
#export LD_LIBRARY_PATH=/opt/apps/cudnn/6.0/lib64:$LD_LIBRARY_PATH
  1. Install TensorFlow in the login (CPU) node
pip install tensorflow-gpu

if you want other version than 1.11.0, just specify the version

  1. Sanity check
# Log in to the GPU node via interactive mode
interact -q p100_dev_q -lnodes=1:ppn=10:gpus=1 -A vllab_01 -l walltime=2:00:00

source ~/.bashrc

python
import tensorflow as tf

Caffe2 and Detectron

  • This instruction requires Anaconda/Miniconda, please install it first
  • Compiling may need GPU, while GPU node cannot get access to the Internet. So please open another terminal when you need to git clone or pip install or conda install; while use the GPU node when you are building/compiling GPU related things
  • The Caffe2 has been merged to the Pytorch, the installation instruction with the old Caffe2 repository does not work anymore. Here is the installation instruction of the new Caffe2 without any sudo access: New Caffe2 installation instruction

OpenCV

  1. Install Anaconda and make your conda environment
  2. Just do pip install opencv-python
$python
>>import cv2

If you don't see any errors, you are good to go.