# Documentation on Camera/Lidar to Point cloud
Documentation for the following things:
* ROS
* Basler Camera (Front)
* LSD-SLAM
* ORB-SLAM
* DeepTAM
* Breezy-SLAM
* Hector-SLAM
* IMU
# Projects
## ROS(Robot Operating System)
### Cheat Sheet
http://github.com/ros/cheatsheet/releases/download/0.0.1/ROScheatsheet_catkin.pdf
### ROS Workspace Environment
#### Overview
Defines context for the current workspace
#### Load default workspace
```
source /opt/ros/indigo/setup.bash
```
#### Overlay your *catkin* workspace
```
cd ~/catkin_ws
source devel/setup.bash
```
#### Check workspace
```
echo $ROS_PACKAGE_PATH
```
### ROS Master
#### Overview
* Manages the communication between nodes
* Every node registers at startup with the master
#### Start Master
```
roscore
```
### ROS Nodes
#### Overview
* Single purpose, executable program
* Individually compiled, executed and managed
* Organized in packages
#### Run a node
```
rosrun package_name node_name
```
#### See active nodes
```
rosnode list
```
#### Retrieve information about a node
```
rosnode info node_name
```
### ROS Topics
#### Overview
* Nodes communicate over topics
* Nodes can publish or subscribe to a topic
* Topic is a name for a stream of messages
#### List active topics
```
rostopic list
```
#### Subscribe and print contents of a topic
```
rostopic echo /topic
```
#### Show information about a topic
```
rostopic info /topic
```
#### Analayze the frequency
```
rostopic hz /topic
```
### ROS Messages
#### Overview
* Data structure defining the *type* of a topic
* Compromised of a nested structure of integers, floats, booleans, strings etc. and arrays of objects
* Defined in *.msg files
#### See type of a topic
```
rostopic type /topic
```
#### Publish a message to a topic
```
rostopic pub /topic type args
```
### catkin Build System
```
catkin_make
```
#### Navigate to catkin workspace
```
cd ~/catkin_ws
```
#### Build package
```
catkin build package_name
```
#### !! If you build a new package -> update your environment !!
```
source devel/setup.bash
```
#### Clean entire build and devel space
```
catkin clean
```
#### Check catkin workspace setup
```
catkin config
```
### ROS Launch
#### Overview
* *launch* is a tool for launching multiple nodes
* are written in SML as *.launch files
* launch starts automatically a roscore, if it is not running yet
#### Start a launch file
```
roslaunch file_name.launch
```
#### Start launch file from a package
```
roslaunch package_name file_name.launch
```
#### See content of the launch file
```
rosed file_name.launch
```
#### File Structur
Example
```
<launch>
<node name="node_name" pkg="package_name" type="listener/talker" output="screen"/>
<node name="node_name" pkg="package_name" type="listener/talker" output="screen"/>
</launch>
```
Explanation
* launch
root element of the launch file
* node
each < node > tag specifies a node to be launched
* name
name of the node
* pkg
package containing the node
* type
type of the node, there must be a corresponding executable with the same name
* output
specifies where to output log messages(screen: console, log: log file)
#### Arguments
```
<arg name="arg_name" default="default_value"/>
```
Example
```
<arg name="use_sim_time" default="true"/>
```
##### Use arguments in launch a file
```
$(arg arg_name)
```
Example
```
if="$(arg use_sim_time)"
```
##### Launching arguments can be set with
```
roslaunch launch_file.launch arg_name:=value
```
#### Including Other Launch Files
##### Include other launch files
```
<include file="package_name"/>
```
##### Find the system path to other packages
```
$(find package_name)
```
##### Pass arguments to the included file
```
<arg name="arg_name" value="value"/>
```
### ROS Packages
* ROS software is organized into packages, which can contain source code, launch files, configuartion files, message definitions, data and documentation
* A packge that builds up on/requires other packages, declares these as dependencies
```
catkin_create_pkg package_name {dependencies}
```
#### CMakeLists.xml
```
cmake_minimum_required(VERSION 2.8.3)
project(ros_package_template)
## Use C++11
add_definitions(--std=c++11)
## Find catkin macros and libraries
find_package(catkin REQUIRED
COMPONENTS
roscpp
sensor_msgs
)
catkin_package(
INCLUDE_DIRS include
#LIBRARIES
CATKIN_DEPENDS roscpp sensor_msgs
#DEPENDS
)
include_directories(include ${catkin_INCLUDE_DIRS})
add_executable(${PROJECT_NAME} src/${PROJECT_NAME}_node.cpp
src/name.cpp)
target_link_libraries(${PROJECT_NAME} ${catkin_LIBRARIES})
```
* ```project(ros_package_template)```
Use the same name as in the package.xml
* ```add_definitions(--std=c ++ 11)```
use C ++ 11 by default
* ```find_package(catkin REQUIRED COMPONENTS roscpp sensor_msgs)```
List the packages that your package requires to build (have to be listed in package.xml)
* ```catkin_package( INCLUDE_DIRS include CATKIN_DEPENDS roscpp sensor_msgs)```
Specify build export information
INCLUDE_DIRS -> Directories with header files
LIBRARIES -> Libraries created in this project
CATKIN_DEPENDS -> Packages depent projects alsoneed
DEPENDS -> System dependencies dependent projects also need (have to be listed in package.xml)
* ```include_directories(include ${catkin_INCLUDE_DIRS})```
Specify locations of header files
* ```add_executable(${PROJECT_NAME} src/${PROJECT_NAME}_node.cpp src/name.cpp)```
Declare a C++ executable
* ```target_link_libraries(${PROJECT_NAME} ${catkin_LIBRARIES})```
Specify libraries to link the execuble against
### Easy ROS C++ Example (Hello World)
```
#include <ros/ros.h>
```
ROS main header file include
```
int main(int argc, char** argv)
{
ros::init(argc, argv, "hello_world");
```
ros::init(...) hasto be calle before calling other ROS functions
```
ros::NodeHandle nodeHandle;
```
The node handle is the access point for communications with the ROS system(topics services parameters)
```
ros::Rate loopRate(10);
```
ros::Rate is a helper class to run loops at a desired frequency
```
unsigned int count = 0;
while(ros::ok()){
```
ros::ok() check if a node should continue running
*Returns false if SIGINT is received(Ctrl + C) or ros::shutdown() has been called*
```
ROS_INFO_STREAM("Hello World" << count);
```
ROS_INFO() logs messages to the filesystem
```
ros::spinOnce();
```
ros::spinOnce() processes incoming messages via callbacks
```
loopRate.sleep();
count++;
}
return 0;
}
```
### Node Handles (Recommended)
* Default (public) node handle:
nh_ = ros::NodeHandle();
/namespace/topic
* Private node handle:
nh_private_ = ros::NodeHandle("~");
/namespace/node/topic
* Namespaced node hanle:
nh_eth_ = ros::NodeHandle("eth");
/namespace/eth/topic
### Logging
```
/rosout topic
```
In the program:
```
ROS_INFO("Result: %d", result);
ROS_INFO_STREAM("Result: "<< result);
```
To see it in the console, set the output configuration to *screen* in the launch file
```
<launch>
<node name="node_name" output="screen"/>
</launch>
```
### Subscriber
```
#include "ros/ros.h"
#include "std_msgs/String.h"
void chatterCallback(const std_msgs::Sring::ConstPtr& msg)
{
ROS_INFO("message: [%s]", msg->data.c_str());
}
int main(int argc, char **argv)
{
ros::init(argc, argv, "listener");
ros::NodeHandle node;
ros::Subscriber subscriber = node.subscribe("chatter",10, chatterCallback);
ros::spin();
return 0;
}
```
* ```ros::Subscriber subscriber = node.subscribe("chatter",10, chatterCallback);```
Start listeninig to a topic by calling the method subscribe() of the node handle
ros:: Subscriber subscriber = nodeHandle.subscribe(topic, queue_size, callback_function);
* ```void chatterCallback(const std_msgs::Sring::ConstPtr& msg){ROS_INFO("message: [%s]", msg->data.c_str()); }```
When a message is received -> callback function is called with the contents of the message as argument
* ```ros::spin();```
ros::spin() processes callbacks and will not return until the node has been shutdown
### Publisher
```
#include <ros/ros.h>
#include <std_msgs/String.h>
int main(int argc, char **argv){
ros::init(argc, argv, "name");
ros::NodeHandle nh;
ros::Publisher chatterPublisher = nh.advertise<std_msgs::String>("chatter",1);
ros::Rate loopRate(10);
unsigned int count = 0;
while(ros::ok()){
std_msgs::String message;
message.data = "hello world" + std::to_string(count);
ROS_INFO_STREAM(message.data);
chatterPublisher.publish(message);
ros::spinOnce();
loopRate.sleep();
count++;
}
return 0;
}
```
* Create a publisher with help of the node handle
```
ros::Publisher publisher = nh.advertise<message_type>(topic, queue_size);
```
* ``` message.data = "hello world" + std::to_string(count);```
Create the message contents
### ROS Parameter Service
#### Overview
* Nodes use the *parameter server* to store and retrieve parameters at runtime
* Best used for static data such as configuration parameters
* Parameters can be defined in launch files or separate YAML files
* Parameters can be defined in lauch files or separate YAML files
```
<launch>
<node name="name" pkg="package" type="node_type">
<rosparam command="load" file="$(find package)/config/config.yaml"/>
</node>
</launch>
```
#### List all parameters
```
rosparam list
```
#### Get value of a parameter
```
rosparam get parameter_name
```
#### Set value of a parameter
```
rosparam set parameter_name value
```
### C++ API
* Get a parameter in C++
```
nodeHandle.getParam(parameter_name, variable)
```
This method returns *true* if parameter was found, *false* otherwise
* Global and relative parameter access
* Global parameter name with preceding /
```
nodeHandle.getParam("/package/camera/left/exposure", variable)
```
* Relative parameter name (relative to the node handle)
```
nodeHandle.getParam("camera/left/exposure", variable)
```
* For parameters, typically use the private node handle
```
ros::NodeHandle("~")
```
```
ros::NodeHandle nodeHandle("~");
std::string topic;
if(!nodeHandle.getParam("topic_name", topic0)){
ROS_ERROR("Could not find topic parameter!")
}
```
### ERRORS
* #### RLException: [file_name.launch] is neither a launch file in package [package_name] nor is [package_name] a launch file name
The traceback for the exception was written to the log file
**Solution:**
source ~/.bashrc
* ####[gazebo-2] process has died [pid < pidnr >, exit code 255, cmd /opt/ros/melodic/lib/gazebo_ros/gzserver -e ode /home/name/mybot_ws/src/mybot_gazebo/worlds/mybot.world __name:=gazebo __log:=/home/name/.ros/log/3aa819c6-a2f5-11e9-a3ef-080027af97c9/gazebo-2.log].
log file: /home/name/.ros/log/3aa819c6-a2f5-11e9-a3ef-080027af97c9/gazebo-2*.log
**Solution**
killall gzserver
kilall gzclient
(and with ctrl+C in the terminal)
* #### Add the installation prefix of "tf2_sensor_msgs" to CMAKE_PREFIX_PATH or
set "tf2_sensor_msgs_DIR" to a directory containing one of the above files.
**Solution**
sudo apt-get install ros-melodic-tf2-sensor-msgs
* #### RLException: roscore cannot run as another roscore/master is already running.
Please kill other roscore/master processes before relaunching.
The ROS_MASTER_URI is http://miriam-HP-ProBook-470-G4:11311/
The traceback for the exception was written to the log file
**Solution**
killall -9 roscore
killall -9 rosmaster
* #### Could not find a package configuration file provided by "joy" with any of
the following names:
joyConfig.cmake
joy-config.cmake
**Solution**
$ sudo apt-get install ros-melodic-joy*
## Basler Camera (Front)
### Information
The ADAS 2018 Car uses the [daA1280-54uc (S-Mount) - Basler dart ](https://www.baslerweb.com/de/produkte/kameras/flaechenkameras/dart/daa1280-54uc-s-mount/)
### Camera drivers
The Basler front camera requires a special node to work. The Node requires that the [pylonSDK](https://www.baslerweb.com/de/support/downloads/downloads-software/) is installed on the system.
### Camera Node
The node that published the video feed is called [pylon_camera](https://wiki.ros.org/pylon_camera) and should be installed as mentioned in the wiki.
When started it should publish these things:
> `image_rect` only shows up when calibration file is loaded
#### Calibration
Calibration for the camera was done by using [camera_calibration](https://wiki.ros.org/camera_calibration) which generated a `.yaml` file that gets passed onto `pylon_camera` by the launchfile.
##### Example Launch File:
```xml=
<?xml version="1.0"?>
<launch>
<arg name="respawn" default="false" />
<arg name="debug" default="false" />
<arg name="node_name" default="front_camera_node" />
<arg name="config_file" default="$(find saamcar)/config/basler_camera_config.yaml" />
<arg unless="$(arg debug)" name="launch_prefix" value="" />
<arg if="$(arg debug)" name="launch_prefix" value="gdb -ex run --args" />
<node name="$(arg node_name)" pkg="pylon_camera" type="pylon_camera_node" output="screen"
respawn="$(arg respawn)" launch-prefix="$(arg launch_prefix)">
<rosparam command="load" file="$(arg config_file)" />
</node>
</launch>
```
##### Example Calibration:
```yaml=
image_width: 1280
image_height: 960
camera_name: basler_camera
camera_matrix:
rows: 3
cols: 3
data: [658.685133, 0, 633.21133, 0, 666.044003, 430.496099, 0, 0, 1]
distortion_model: plumb_bob
distortion_coefficients:
rows: 1
cols: 5
data: [-0.253962, 0.042064, 0.002441, -0.004375, 0]
rectification_matrix:
rows: 3
cols: 3
data: [1, 0, 0, 0, 1, 0, 0, 0, 1]
projection_matrix:
rows: 3
cols: 4
data: [439.611725, 0, 618.08749, 0, 0, 550.546692, 410.861304, 0, 0, 0, 1, 0]
```
## LSD-SLAM
### Info
Repo that was working on ROS-Melodic on the 25^th^ July 2019 can be found [HERE](https://github.com/MrMinemeet/lsd_slam). But here is a guide too.
On this [LSD-SLAM Knowledge Base](/7Lb4cDYFTgCCR8L7H6O9Sg) you can find all errors we encoutered getting LSD-SLAM runnhttps://wiki.ros.org/orb_slam2_rosing.
### Installation Guide
1. Install the required packages
```
sudo apt-get install ros-melodic-libg2o ros-melodic-cv-bridge liblapack-dev libblas-dev freeglut3-dev libqglviewer-dev-qt4 libsuitesparse-dev libx11-dev ros-melodic-catkin cmake
```
Link QGLViewer to another place as ROS is looking there
```
sudo ln -s /usr/lib/x86_64-linux-gnu/libQGLViewer-qt4.so /usr/lib/x86_64-linux-gnu/libQGLViewer.so
```
2. Download & install G2O
A special version of G2O is required so please use the one provided in the guide.
```
git clone https://github.com/felixendres/g2o.git
cd g2o
mkdir build
cd build
cmake ..
sudo make install
```
3. Download Eigen
When compiling LSD-SLAM you'll need a special version of Eigen.
Download `Eigen 3.2.5` from [THIS](https://bitbucket.org/eigen/eigen/downloads/?tab=tags) website.
4. Download & install LSD-SLAM
First start with creating the workspace and cloning the git-repository
```
mkdir ~/lsd_ws
cd lsd_ws
mkdir src
cd src
git clone https://github.com/MrMinemeet/lsd_slam.git
```
next step is to open the `CMakeList.txt` in the `lsd_slam_core` folder and replace
`{PATH TO EIGEN THAT YOU JUT DOWNLOADED}` with the path to your eigen folder.
That line should be above
`include_directories(...`
at the end you go to your `src`-folder and execute `catkin_make`
### How to start it
1. Start the viewer to show the pointcloud
```bash
rosrun lsd_slam_viewer viewer
```
2. Start the LSD-SLAM Node
```
rosrun lsd_slam_core live_slam image:=/image_raw camera_info:=/camera_info
```
### I/O
#### Subscribe
* `image` for camera feed
* `camera_info` for camera information
#### Publish
* `Pointcloude` which can be safed as a `.ply` file
### Notes for good results
* Recommended resolution is **640x480**. Higher resolutions need more computing power.
* Global shutter since rolling shutter leads to inferior results
* A Framerate of **30FPS** or higher recommended
* LSD-SLAM generates different results when running it multiple times
* A wide **Field-of-View** (130°) is recommended
* Rotating the camera on the same spot doesn't work that well. You'll have to move it too
## ORB-SLAM
[ORB-SLAM](https://wiki.ros.org/orb_slam2_ros) worked pretty much out of the box.
The only thing is that ORB is a basic SLAM wich isn't the best.
### Installation
1. Getting Code
```
git clone https://github.com/appliedAI-Initiative/orb_slam_2_ros.git
```
2. Installing Eigen3
```
sudo apt install libeigen3-dev
```
3. Building ORB
Run `catkin build` in workspace folder
### I/O
#### Subscribe
* `image_raw` for monocamera feed
* `rgb/image_raw` for RGBD camera image and `depth_registered/image_raw` for depth information
* `image_left/image_color_rect` and `image_right/image_color_rect` for a stereocamera
#### Publish
* `PointCloud2` containing all all keypoints of the map
* `PoseStamped` with the current position of the camera
* `ìmage` a live image of the scene with the found key points
* `tf` from the pointcloud frame id to the camera frame id (the position)
## DeepTAM
On this page [DeepTAM Knowledgebase](/zdVLLCCBRGKvm-9lCSzb0A) you can find solutions for errors we encountered.
Some of them may not work or break other stuff.
We are working on it :+1:
## Breezly-SLAM
## Hector-SLAM
## IMU
___
by:
Miriam Forstinger *(HTL-Leonding)*
Alexander Gaisbauer *(HTL-Leonding)*
Simon Moharitsch *(HTL-Braunau)*
[Alexander Voglsperger](https://github.com/MrMinemeet/) *(HTL-Braunau)*
Sign in with Wallet
Connect another wallet