System Setup with GPU

Please follow the instructions to setup a new GPU system with cuda, opencv, torch, carla and alive repository # For E2O https://hackmd.io/@ohV_DuXMSFyEZOTxMOIgmw/Hy2YnX8N2 # Outline - Be on Ubuntu **20.04** with added compilation libraries - Nvidia drivers 510 + Cuda 11.6 + Cudnn 8.4 (only with GPU) - OpenCV 4.5.5 (compiled with cuDNN, only with GPU) - LibTorch - 1.12.0 (only with GPU) - ROS noetic full - Carla - 0.9.11 (and carla_ros_bridge) - Compiling alive-dev repository - Add JOSM to edit lanelet maps ## Additional libraries installation to Ubuntu ```bash! sudo apt-get update && sudo apt-get install -y \ automake autoconf libpng-dev nano vim wget curl zip unzip libtool swig zlib1g-dev pkg-config git-all xz-utils python3-mock libpython3-dev libpython3-all-dev python3-pip g++ gcc make pciutils cpio liblapack-dev liblapacke-dev locales cmake unzip openssh-server python-lxml python-is-python3 libgeographic-dev lsb-release xterm pip3 install empy sudo apt-get install -y linux-headers-$(uname -r) ``` ## NVIDIA drivers, CUDA-11.6 and CuDNN 8.4.0 ### 1. NVIDIA drivers > Prefered method: Go to software and updates -> drivers, choose nvidia-drivers-510 reboot ```bash! # verify by running nvidia-smi lsmod | grep "nvidia" ``` ### 2. CUDA-11.6: ```bash! sudo apt install cuda-toolkit-11-6 sudo apt install nvidia-gds-11-6 sudo reboot reboot # add to bash export PATH=/usr/local/cuda-12.0/bin${PATH:+:${PATH}} sudo apt install libglfw3-dev zlib1g # needed to compile cuda samples / cudnn ```  #### Post installation add to ~/.bashrc ```bash! export PATH=/usr/local/cuda-11.6/bin${PATH:+:${PATH}} # if using runfile export LD_LIBRARY_PATH=/usr/local/cuda-11.6/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}} ``` to check the installation: ```bash! source ~/.bashrc nvcc --version ``` ### 2. Get CuDNN 8.4.0.27 > reference [here](https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html) [Download](https://drive.google.com/drive/folders/1cKxDWH_ygsRGGG5JotdWbs_Xfbj4umrG?usp=sharing) CuDNN headers' folder. ```bash! cd cudnn-linux-x86_64-8.4.0.27_cuda11.6-archive/ sudo cp lib/libcudnn* /usr/local/cuda/lib64 sudo cp -P include/cudnn* /usr/local/cuda/include sudo chmod a+r /usr/local/cuda/lib64/libcudnn* ``` ## OpenCV - 4.5.5 1. [Download OpenCV zip](https://github.com/opencv/opencv/tree/4.5.5) 2. [Download OpenCV-Contrib zip](https://github.com/opencv/opencv_contrib/tree/4.5.5) 3. Unzip in the same folder 4. Follow the commands ```bash! cd opencv-4.5.5/ mkdir build && cd build cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D WITH_TBB=ON -D WITH_CUDA=ON -DWITH_CUDNN=ON -D OPENCV_DNN_CUDA=ON -D ENABLE_FAST_MATH=1 -D CUDA_FAST_MATH=1 -D WITH_CUBLAS=1 -D WITH_V4L=ON -D WITH_QT=ON -D WITH_OPENGL=ON -D WITH_GSTREAMER=ON -D OPENCV_GENERATE_PKGCONFIG=ON -D OPENCV_PC_FILE_NAME=opencv.pc -D OPENCV_EXTRA_MODULES_PATH=../../opencv_contrib-4.5.5/modules -D BUILD_SHARED_LIBS=ON -D PYTHON_EXECUTABLE=/usr/bin/python3 .. make -j$(nproc) sudo make install ``` ## LibTorch 1.12 - [Download LibTorch 1.12 - CUDA 11.6 - (cxx11 ABI)](https://download.pytorch.org/libtorch/cu116/libtorch-cxx11-abi-shared-with-deps-1.12.0%2Bcu116.zip) - Unzip ```bash! unzip libtorch-cxx11-abi-shared-with-deps-1.12.0+cu116.zip ``` - Place in ~ ```bash! mv libtorch ~/ ``` - Set Torch_DIR in .bashrc ```bash! echo 'export Torch_DIR=$HOME/libtorch' >> ~/.bashrc ``` ## ROS Noetic 1. Follow full desktop-full from [here](http://wiki.ros.org/noetic/Installation/Ubuntu) ```bash! sudo sh -c 'echo "deb http://packages.ros.org/ros/ubuntu $(lsb_release -sc) main" > /etc/apt/sources.list.d/ros-latest.list' curl -s https://raw.githubusercontent.com/ros/rosdistro/master/ros.asc | sudo apt-key add - sudo apt-get update sudo apt install -y ros-noetic-desktop-full python3-rosdep python3-rosinstall python3-rosinstall-generator python3-wstool build-essential python3-catkin-tools echo "source /opt/ros/noetic/setup.bash" >> ~/.bashrc source ~/.bashrc sudo rosdep init rosdep update ``` ## Carla & carla_ros_bridge - 0.9.11 ### Option 1: get CARLA from repo and source ros-bridge 1. Get Carla 0.9.11 ```bash! sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 1AF1527DE64CB8D9 sudo add-apt-repository "deb [arch=amd64] http://dist.carla.org/carla $(lsb_release -sc) main" sudo apt-get update sudo apt-get install carla-simulator=0.9.11 sudo apt-mark hold carla-suimulator # to prevent accidental upgrades ``` ```bash! # In ~/.bashrc export CARLA_ROOT=/opt/carla-simulator export PYTHONPATH=$PYTHONPATH:${CARLA_ROOT}/PythonAPI/carla/dist/carla-0.9.11-py2.7-linux-x86_64.egg source ~/.bashrc ``` 2. Get carla-ros-bridge 0.9.11 from tag on git ```bash! sudo apt install ros-$ROS_DISTRO-ackermann-msgs ros-$ROS_DISTRO-derived-object-msgs mkdir -p ~/carla-ros-bridge/catkin_ws/src cd ~/carla-ros-bridge git clone --recurse-submodules https://github.com/carla-simulator/ros-bridge.git catkin_ws/src/ros-bridge cd catkin_ws/src/ros-bridge && git checkout 0.9.11 # patch to fix actor creation sync (is pushed upstream) wget https://github.com/carla-simulator/ros-bridge/pull/576.patch git am 576.patch sudo ./install_dependencies.sh cd ../../ && catkin build source devel/setup.bash ``` ### Option 2: Use Docker 1. Get docker - [installation instructions](https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository) 2. If already using rootless mode: uninstall rootless docker: [here](https://docs.docker.com/engine/security/rootless/) Rootless security prevents ros nodes from communicating. 4. Get carla-simulator 0.9.11 docker - Setup nvidia-ctk: https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#setting-up-nvidia-container-toolkit - Get docker image: ```bash! sudo docker pull carlasim/carla:0.9.11 xhost +local:docker # to run on system with display sudo docker run --name carla-docker --privileged --gpus all --net=host -e DISPLAY=$DISPLAY -e SDL_VIDEODRIVER=x11 -v /tmp/.X11-unix:/tmp/.X11-unix:rw carlasim/carla:0.9.11 /bin/bash ./CarlaUE4.sh -vulkan # for server/headless: https://carla.readthedocs.io/en/latest/adv_rendering_options/#off-screen-mode ``` 3. Get carla-ros-bridge docker. Only useful if normal carla-ros-bridge is buggy ```bash! git clone --recurse-submodules https://github.com/carla-simulator/ros-bridge.git catkin_ws/src/ros-bridge cd catkin_ws/src/ros-bridge && git checkout 0.9.11 cd ~/carla-ros-bridge/catkin_ws/src/ros-bridge/docker sudo ./build.sh -r noetic -c 0.9.11 sudo ./run.sh -i carla-ros-bridge -t noetic apt install vim vim /etc/hosts # check if localhost is pointing to loopback > localhost 127.0.0.1 # [Commit changes](https://docs.docker.com/engine/reference/commandline/commit/#commit-a-container) in docker image to preserve the changes. # outside docker container wget https://github.com/carla-simulator/ros-bridge/pull/576.patch git am 576.patch docker ps docker commit <commit-sha> carla-ros-bridge:complete # then run docker with new tag ./run.sh -i carla-ros-bridge -t complete ``` ## ALiVe Dependencies - Install additional packages: ```bash! sudo apt-get install -y ros-$ROS_DISTRO-mrt-cmake-modules ros-$ROS_DISTRO-carla-msgs ros-$ROS_DISTRO-hector-trajectory-server ros-${ROS_DISTRO}-mavros ros-${ROS_DISTRO}-mavros-extras ros-${ROS_DISTRO}-mavros-msgs ros-${ROS_DISTRO}-grid-map ros-$ROS_DISTRO-realsense2* ros-$ROS_DISTRO-jsk-rviz-plugins ros-$ROS_DISTRO-navigation libompl-dev libgoogle-glog-dev libgflags-dev libatlas-base-dev libeigen3-dev libsuitesparse-dev libzmq3-dev libboost-dev wget https://raw.githubusercontent.com/mavlink/mavros/master/mavros/scripts/install_geographiclib_datasets.sh sudo bash ./install_geographiclib_datasets.sh ``` - Install [ceres-solver](http://ceres-solver.org/installation.html) for floam: ```bash! git clone https://ceres-solver.googlesource.com/ceres-solver cd ceres-solver && git checkout 2.1.0 && cd .. mkdir ceres-bin cd ceres-bin cmake ../ceres-solver make -j$(nproc) sudo make install ``` - Install [alglib-cmake](https://github.com/S-Dafarra/alglib-cmake) for structured_planner ```bash! git clone https://github.com/S-Dafarra/alglib-cmake.git cd alglib-cmake mkdir build && cd build cmake .. make -j$(nproc) sudo make install ```  - Install nlopt for velocity_profile ```bash! cd src/velocity_profile/include tar -xvf nlopt-2.7.1.tar.gz cd nlopt-2.7.1 mkdir build && cd build cmake .. make -j$(nproc) sudo make install ``` - Edit config for lanelet_visualisation - follow instructions [here](https://github.com/KIT-MRT/mrt_cmake_modules/issues/27) OR - type ```bash! echo "qtbase5-dev: components: [Core Gui Network] include_dirs: [QT_INCLUDES] libraries: [QT_LIBRARIES] name: MrtQt5" | sudo tee -a /opt/ros/$ROS_DISTRO/share/mrt_cmake_modules/yaml/cmake.yaml ``` ## Clone and compile alive-dev 1. Submodule fetch and get leftover dependencies ```bash! git clone --recurse-submodules git@bitbucket.org:alive_iiitd/alive-dev.git cd alive-dev # refer to readme for branch-specific build instructions catkin init catkin build lanelet* velodyne* ``` 2. If running objection detection: [Download](https://drive.google.com/drive/folders/1qfFAZnGiz7t5qzPcoDY0M5kAr98ztcaQ?usp=sharing) the required torch models in the folder - ```object_detection/src/models```. 3. OpenCV (needed for estimator) For Melodic: ```bash # this needs to be done because object_detection runs on OpenCV 4, so cv_bridge should also be compiled with OpenCV 4 and default melodic cv_bridge uses OpenCV 3.2 git submodule update --init --recursive cd src/vision_opencv git apply ../../vision_opencv_melodic.patch # can do git diff --stat to check the changes ``` For Noetic ```bash rm -rf src/vision_opencv ``` 4. Useful Configure and build options: ```bash! #RECOMMENDED catkin config --cmake-args -DCMAKE-BUILD-TYPE=Release catkin build vision_opencv velodyne lanelet* catkin config --whitelist alive_msgs car_data cascaded_pid floam velocity_profile structured_planner map_publisher estimator can e2o virtual_obs_pose_pub point_cloud_transformer localisation_fusion catkin build # when you want to to debug a package with gdb catkin build hybrid_astar --no-deps --cmake-args -DCMAKE_BUILD_TYPE=Debug ``` ## Install [Flycapture SDK](https://www.flir.com/support-center/iis/machine-vision/downloads/spinnaker-sdk-flycapture-and-firmware-download/) If you want to use **Pointgrey Cameras** -- [Download from here](https://flir.app.boxcn.net/v/Flycapture2SDK) then unzip: ```bash! # Install the necessary dependencies required sudo apt-get install -y libraw1394-11 libgtkmm-2.4-1v5 libglademm-2.4-1v5 libgtkglextmm-x11-1.2-dev libgtkglextmm-x11-1.2 libusb-1.0-0 tar -xvzf <flycapture_path> && cd <flycapture_path> sudo sh install_flycapture.sh ``` ## Install JOSM If you want to edit lanelet_maps 1. Install JOSM from [here](https://josm.openstreetmap.de/wiki/Download) ```bash! echo "deb [signed-by=/usr/local/share/keyrings/josm-apt.gpg] https://josm.openstreetmap.de/apt $(lsb_release -sc) universe" | sudo tee /etc/apt/sources.list.d/josm.list > /dev/null # Create the directory for manually downloaded keys if it was not already created sudo mkdir -p /usr/local/share/keyrings # Download the key wget -q https://josm.openstreetmap.de/josm-apt.key -O- | sudo gpg --dearmor -o /usr/local/share/keyrings/josm-apt.gpg # You may need to install ssl support for apt in advance: sudo apt-get install apt-transport-https # Refresh sources sudo apt-get update # For the tested version sudo apt-get install josm ``` 2. Setup JOSM for use with lanelets from [here](https://github.com/fzi-forschungszentrum-informatik/Lanelet2/tree/master/lanelet2_maps) ```bash! roscd lanelet2_maps && cd josm # The josm folder of this package contains styles and presets for simple set-up, unzip the images in style_images.zip unzip style_images.zip ``` 3. Open JOSM - Under Preferences->Map Settings -> Map Paint styles -> add (+) select lanelets.mapcss and lines.mapcss to add the lanelet2 styles to JOSM. - Under Preferences->Map Settings -> Tagging Presets -> add (+) select laneletpresets.xml to add the lanelet2 tagging style to JOSM