# PCL/OpenNI tutorial 1: Installing and testing
This tutorial is going to guid you install PCL and openni on ubuntu step by step. There are 2 ways to install PCL : one is the istall the binary file using the **precompiled PCL** which the version of pPCL is older 1.69 and the other is **build from source** which you could get the lastest version of PCL. ex. CUDA applications and KINECT fusion.

### Reference :
1. [PCL/OpenNI tutorial 1: Installing and testing](http://robotica.unileon.es/index.php/PCL/OpenNI_tutorial_1:_Installing_and_testing)
2. [Compiling PCL from source on POSIX compliant systems](http://www.pointclouds.org/documentation/tutorials/compiling_pcl_posix.php)

### Experiment OS
- Operating System: Ubuntu 14.04.5 LTS
- Kernel: Linux 4.4.0-31-generic
- Architecture: x86_64
- Name : Howard Chen
- Date : 2017/1/16

## Precompiled PCL for Ubuntu

If you have a valid installation of ROS (through their repository), you do not have to install anything. Both OpenNI and PrimeSense drivers, as well as PCL, should be already installed. You can check it with:

sudo apt-get install libpcl-1.7-all libpcl-1.7-all-dev libopenni-dev libopenni-sensor-primesense-dev

The previous command should state that all packages are already installed (change the PCL version number as needed), install them if not. If you get an error about overwriting some file, check this.

In case you do not want ROS, there is a PPA (Personal Package Archive, a private repository) which has everything we need. Add it to your sources, and install everything:

sudo add-apt-repository ppa:v-launchpad-jochen-sprickerhof-de/pcl
sudo apt-get update
sudo apt-get install build-essential cmake libpcl1.7 libpcl-dev pcl-tools

Trying to mix ROS and PCL repositories and packages can cause some errors, so choose one of them and stick with it. Check the PCL/OpenNI troubleshooting page because your Kinect may not work by default in 32 bits.

## Compiling PCL from source

For Linuxes without a precompiled version of PCL, you will need to compile it yourself. This has an advantage, actually: you can customize the build options and choose what you want. Also, the resulting binaries and libraries should be a bit faster. And you always get the latest version! The instructions are here, but the steps are easy so I will show them to you.
Installing the dependencies

Some of PCL dependencies can be installed via the package manager. Others will require additional work.

sudo apt-get install build-essential cmake cmake-curses-gui libboost-all-dev libeigen3-dev libflann-dev libvtk5-dev libvtk5-qt4-dev libglew-dev libxmu-dev libsuitesparse-dev libqhull-dev libpcap-dev libxmu-dev libxi-dev libgtest-dev libqt4-opengl-dev

**Notes: If you met some dependancy problem, just add those lib into install list**

The trunk (1.8) version of PCL adds support for VTK 6 and Qt 5, so if you want to use them, you must install the following packages (say yes if you are asked to remove VTK 5 and Qt 4 packages):

sudo apt-get install libvtk6-dev libqt5opengl5-dev

PCL will not build if you mix them, e.g., using VTK 5 with Qt 5 will give you an error halfway through the compilation.

## JDK

OpenNI requires Sun's official JDK (Java Development Kit), which is no longer available on official apt repositories. You can use unofficial ones, or do it manually. For the last method, go to the Java SE downloads page [SE means Standard Edition](www.oracle.com/technetwork/java/javase/downloads/index.html) and download the latest version (e.g., ```jdk-8u73-linux-x64.tar.gz```). Extract it, then move the contents to ```/usr/lib/jvm/ ```so it is available system-wide:

- **We use jdk-8u73-linux-x64.tar.gz as exapmle :**

sudo mkdir -p /usr/lib/jvm/
sudo cp -r jdk1.8.0_73/ /usr/lib/jvm/

Then, make it the default choice to compile and run Java programs. Remember to change the version number as needed!

sudo update-alternatives --install "/usr/bin/java" "java" "/usr/lib/jvm/jdk1.8.0_73/bin/java" 1
sudo update-alternatives --install "/usr/bin/javac" "javac" "/usr/lib/jvm/jdk1.8.0_73/bin/javac" 1
sudo update-alternatives --install "/usr/bin/jar" "jar" "/usr/lib/jvm/jdk1.8.0_73/bin/jar" 1

To be sure, use the following commands to manually select the correct option, in case there is more than one choice:

sudo update-alternatives --config java
sudo update-alternatives --config javac
sudo update-alternatives --config jar

If you are still not sure, run them and display the version, making sure it is the one you installed:

java -version
javac -version

Sun's JDK is now installed.

## OpenNI

PCL uses OpenNI and the PrimeSense drivers to get data from devices like Kinect or Xtion. It is optional, but it would not make much sense not to install it, would it? If you are using Ubuntu, add the PPA and install libopenni-dev and libopenni-sensor-primesense-dev, which should be done already. Otherwise, fetch the OpenNI and PrimeSense Sensor sources from GitHub (download them as .zip, the link is on the right). Extract them, and install the dependencies:

sudo apt-get install python libusb-1.0-0-dev freeglut3-dev doxygen graphviz

Go to the directory where you extracted OpenNI (OpenNI-master/ for me), and open a terminal in the ```Platform/Linux/CreateRedist/ subdirectory```. Issue:

./RedistMaker

When it finishes, and if there are no errors (check the PCL/OpenNI troubleshooting page if you get any), go to ```Platform/Linux/Redist/OpenNI-Bin-Dev-Linux-x64-v1.5.7.10/``` (or your equivalent), and install (you must be root):

sudo ./install.sh

Now, go to the directory where you extracted the PrimeSense drivers (```Sensor-master/``` for me), and repeat the exact same procedure (go to ```Platform/Linux/CreateRedist/```, issue ```./RedistMaker```, go to ```Platform/Linux/Redist/Sensor-Bin-Linux-x64-v5.1.6.6/```, issue ```sudo ./install.sh```). Congratulations, you have now installed OpenNI.

## CUDA

Like OpenNI, nVidia CUDA is an optional dependency, that will allow PCL to use your GPU (Graphics Processing Unit, that is, your graphics card) for certain computations. This is mandatory for tools like KinFu (do not bother unless you have at least a series 400 card with 1.5 GB of VRAM).
You have to download the .deb file of CUDA from the [offical site](https://developer.nvidia.com/cuda-downloads)

sudo dpkg -i cuda-repo-ubuntu1404-8-0-local_8.0.44-1_amd64.deb`
sudo apt-get update`
sudo apt-get install cuda`

The environment variables need to be changed so your system can find CUDA's libraries and binaries. Open /etc/ld.so.conf:

sudo nano /etc/ld.so.conf

And append one of these two lines:

/usr/local/cuda/lib64 # Add this on 64-bit only.
/usr/local/cuda/lib # Add this on 32-bit only.

Save with Ctrl+O and Enter, exit with Ctrl+X. Reload the cache of the dynamic linker with:

sudo ldconfig

Now, append CUDA's bin directory to your PATH. Do this by editing your local .bashrc file:

vim ~/.bashrc

And append this line:

export PATH=$PATH:/usr/local/cuda/bin

CUDA is now installed.
## Getting the source

Every dependency is installed. Time to download PCL's source code. First, you must choose whether to install the stable or the experimental branch of PCL. The stable branch is the latest official release and it is guaranteed to work without problems. The experimental branch may give you a compiling error seldomly, but you can find some interesting features that stable users will have to wait some months for. Apart from that, both are built the same way.

To get the stable version, go to the downloads page, get pcl-pcl-1.7.2.tar.gz or whatever the latest release is, and extract it somewhere. For the experimental version, use Git:

sudo apt-get install git
git clone https://github.com/PointCloudLibrary/pcl 

The compiled trunk version of PCL will take up more than 8 GB. So make sure you put the source folder in a partition with enough free space!
Compiling

Go the the PCL source directory, and create a new subdirectory to keep the build files in:

mkdir build
cd build

Now it is time to configure the project using CMake. We will tell it to build in Release (fully optimized, no debug capabilities) mode now, and customize the rest of the options later:
Run the CMake build system using the default options:

cmake ..

Or change them (uses cmake-curses-gui):

ccmake ..

Please note that cmake might default to a debug build. If you want to compile a release build of PCL with enhanced compiler optimizations, you can change the build target to “Release” with “-DCMAKE_BUILD_TYPE=Release”:

cmake -DCMAKE_BUILD_TYPE=Release ..

If you are done configuring, it is time to build:

make -j2

NOTE: Additionally, you can append the parameter -jX to speed up the compilation, X being the number of cores or processors of your PC, plus one.

**Remember that, at any time, you can manually force the project to be reconfigured and built from scratch by emptying the ```build/``` directory with:**

rm -rf ./*

## Installing

It will take some time to compile PCL (up to a few hours if your PC is not powerful enough). When it is finished, install it system-wide with:

sudo make install

And you should reboot and proceed to the next section, to see if your computer now recognizes (and uses) your Kinect device.
Testing (OpenNI viewer)

We are going to write a simple example program that will fetch data from the Kinect or Xtion and present it to the user, using the PCL library. It will also allow to save the current frame (as point cloud) to disk. If you feel lazy, you can download it here, and skip the next two sections. Otherwise, create a new directory anywhere in your hard disk and proceed.

### **CMakeLists.txt**

Inside that directory, create a new text file named CMakeLists.txt. PCL-based programs use the CMake build system, too. Open it with any editor and paste the following content:

cmake_minimum_required(VERSION 2.8 FATAL_ERROR)
 
project(PCL_openni_viewer)
 
find_package(PCL 1.7 REQUIRED)
 
include_directories(${PCL_INCLUDE_DIRS})
link_directories(${PCL_LIBRARY_DIRS})
add_definitions(${PCL_DEFINITIONS})
 
set(PCL_BUILD_TYPE Release)
 
file(GLOB PCL_openni_viewer_SRC
    "src/*.h"
    "src/*.cpp"
)
add_executable(openniViewer ${PCL_openni_viewer_SRC})
 
target_link_libraries (openniViewer ${PCL_LIBRARIES})

CMake syntax is quite self-explanatory. We ask for a CMake version 2.8 installation, minimum. We declare a new project named "openni_PCL_viewer". We tell CMake to check for the presence of PCL library development files, version 1.7. If our system can not meet the CMake and PCL version requirement, the process will fail.

Next, we feed the compiler and linker the directories where PCL includes and libraries can be found, and the defined symbols. We tell CMake to use the "Release" build type, which will activate certain optimizations depending on the compiler we use. Other build types are available, like "Debug", "MinSizeRel", and "RelWithDebInfo".

Finally, we create a variable, "opennipclviewer_SRC", that will store a list of files to be compiled (though we will only have one). We create a new binary to be compiled from these source files, and we link it with the PCL library.

### main.cpp

We told CMake it could find the source files in a src/ subdirectory, so let's keep to out word and create it. Then, add a new main.cpp file inside and paste the following lines:

// Original code by Geoffrey Biggs, taken from the PCL tutorial in
// http://pointclouds.org/documentation/tutorials/pcl_visualizer.php

// Simple OpenNI viewer that also allows to write the current scene to a .pcd
// when pressing SPACE.
#include <pcl/io/openni_grabber.h>
#include <pcl/io/pcd_io.h>
#include <pcl/visualization/cloud_viewer.h>
#include <pcl/console/parse.h>

#include <iostream>

using namespace std;
using namespace pcl;

PointCloud<PointXYZRGBA>::Ptr cloudptr(new PointCloud<PointXYZRGBA>); // A cloud that will store color info.
PointCloud<PointXYZ>::Ptr fallbackCloud(new PointCloud<PointXYZ>);    // A fallback cloud with just depth data.
boost::shared_ptr<visualization::CloudViewer> viewer;                 // Point cloud viewer object.
Grabber* openniGrabber;                                               // OpenNI grabber that takes data from the device.
unsigned int filesSaved = 0;                                          // For the numbering of the clouds saved to disk.
bool saveCloud(false), noColor(false);                                // Program control.

void
printUsage(const char* programName)
{
	cout << "Usage: " << programName << " [options]"
		 << endl
		 << endl
		 << "Options:\n"
		 << endl
		 << "\t<none>     start capturing from an OpenNI device.\n"
		 << "\t-v FILE    visualize the given .pcd file.\n"
		 << "\t-h         shows this help.\n";
}

// This function is called every time the device has new data.
void
grabberCallback(const PointCloud<PointXYZRGBA>::ConstPtr& cloud)
{
	if (! viewer->wasStopped())
		viewer->showCloud(cloud);

	if (saveCloud)
	{
		stringstream stream;
		stream << "inputCloud" << filesSaved << ".pcd";
		string filename = stream.str();
		if (io::savePCDFile(filename, *cloud, true) == 0)
		{
			filesSaved++;
			cout << "Saved " << filename << "." << endl;
		}
		else PCL_ERROR("Problem saving %s.\n", filename.c_str());

		saveCloud = false;
	}
}

// For detecting when SPACE is pressed.
void
keyboardEventOccurred(const visualization::KeyboardEvent& event,
					  void* nothing)
{
	if (event.getKeySym() == "space" && event.keyDown())
		saveCloud = true;
}

// Creates, initializes and returns a new viewer.
boost::shared_ptr<visualization::CloudViewer>
createViewer()
{
	boost::shared_ptr<visualization::CloudViewer> v
	(new visualization::CloudViewer("OpenNI viewer"));
	v->registerKeyboardCallback(keyboardEventOccurred);

	return (v);
}

int
main(int argc, char** argv)
{
	if (console::find_argument(argc, argv, "-h") >= 0)
	{
		printUsage(argv[0]);
		return -1;
	}

	bool justVisualize(false);
	string filename;
	if (console::find_argument(argc, argv, "-v") >= 0)
	{
		if (argc != 3)
		{
			printUsage(argv[0]);
			return -1;
		}

		filename = argv[2];
		justVisualize = true;
	}
	else if (argc != 1)
	{
		printUsage(argv[0]);
		return -1;
	}

	// First mode, open and show a cloud from disk.
	if (justVisualize)
	{
		// Try with color information...
		try
		{
			io::loadPCDFile<PointXYZRGBA>(filename.c_str(), *cloudptr);
		}
		catch (PCLException e1)
		{
			try
			{
				// ...and if it fails, fall back to just depth.
				io::loadPCDFile<PointXYZ>(filename.c_str(), *fallbackCloud);
			}
			catch (PCLException e2)
			{
				return -1;
			}

			noColor = true;
		}

		cout << "Loaded " << filename << "." << endl;
		if (noColor)
			cout << "This cloud has no RGBA color information present." << endl;
		else cout << "This cloud has RGBA color information present." << endl;
	}
	// Second mode, start fetching and displaying frames from the device.
	else
	{
		openniGrabber = new OpenNIGrabber();
		if (openniGrabber == 0)
			return -1;
		boost::function<void (const PointCloud<PointXYZRGBA>::ConstPtr&)> f =
			boost::bind(&grabberCallback, _1);
		openniGrabber->registerCallback(f);
	}

	viewer = createViewer();

	if (justVisualize)
	{
		if (noColor)
			viewer->showCloud(fallbackCloud);
		else viewer->showCloud(cloudptr);
	}
	else openniGrabber->start();

	// Main loop.
	while (! viewer->wasStopped())
		boost::this_thread::sleep(boost::posix_time::seconds(1));

	if (! justVisualize)
		openniGrabber->stop();
}

Save and close.

## Compiling

Follow the same steps you used to build PCL. That is, create a new ```build/``` subdirectory next to the ```src/``` one. Open a terminal there and issue:

cmake -DCMAKE_BUILD_TYPE=Release ..
make

## Executing

Still from the same terminal, run the compiled example program:

./openniViewer

After some seconds, the main window will appear and the application will start grabbing frames from the device. You can inspect the current point cloud using the mouse, holding the left button to rotate, the right one (or the mouse wheel) to zoom, and the middle one to pan the camera around. At first, you may see only a black screen, or some big colored axes, but no cloud. Try zooming out, to see the whole scene. Another useful key is R, which will reset the camera parameters when pressed. Use it whenever you notice that zooming has gotten slow after some camera movement, or if you still can not see the cloud. See the PCLVisualizer tutorial for additional controls and features.

Whenever you feel ready, press the SPACE key. The program will pause for a fraction of a second and the output "Saved inputCloud0.pcd." will appear on the console. Check the current folder to see that file inputCloud0.pcd has indeed been written. You can now close the program with Q or Alt+F4.

Next, run it again giving the following parameter:

./openniViewer -v inputCloud0.pcd

This will tell the program not to take data from the device, but from the saved point cloud file instead. After it loads, you will realize that you are presented the same scene you saved to disk.

**NOTE: PCD data is saved relative to the sensor. No matter how much you have manipulated the view, it will reset to default when you load the file.**
## Conclusion

At this point, your Kinect device should be working and getting depth data for you. There is a collection of excellent tutorials for PCL in the official webpage. I encourage you to finish them all before proceeding your experiments with the Kinect sensor. You can also find a good introduction/tutorial at the PCL library here.

If you use an ASUS Xtion PRO device instead, you should have gotten everything to work without problems or additional steps (except maybe for this one).

## Trouble shooting

### 1. Device Id not set, using first device.

terminate called after throwing an instance of 'pcl::IOException'  
  what():  virtual void pcl::OpenNIGrabber::start() in /home/schwarmcyc/PCL-trunk/io/src/openni_grabber.cpp @ 241 : Could not start streams. Reason: virtual void openni_wrapper::OpenNIDevice::startImageStream() @ /home/schwarmcyc/PCL-trunk/io/src/openni_camera/openni_device.cpp @ 551 : starting image stream failed. Reason: Bad Parameter sent to the device!  
Aborted (core dumped)

### Solutions
1. **kinect driver not install well**

step1. Install libfreenect(which is a new driver for kinect1)

git clone https://github.com/OpenKinect/libfreenect
or use what already download

cd libfreenect 
mkdir build 
cd build 
cmake -L .. 
make

### 2. Kinfu problem

Warning: USB events thread - failed to set priority. This might cause loss of data...  
Error: out of memory        /home/schwarmcyc/PCL-trunk/gpu/containers/src/device_memory.cpp:260 
schwarmcyc@schwarmcyc:~/PCL-trunk/build/bin$ pcl_kinfu_app  
[pcl::gpu::printShortCudaDeviceInfo] : Device 0:  "GeForce GTX 650"  1023Mb, sm_30, 384 cores, Driver/Runtime ver.7.50/7.0  
Segmentation fault (core dumped)  

solution: restart terminal

### 3. Problem

terminate called after throwing an instance of 'pcl::IOException'
  what():  void pcl::OpenNIGrabber::setupDevice(const string&, const pcl::OpenNIGrabber::Mode&, const pcl::OpenNIGrabber::Mode&) in /home/schwarmcyc/pcl-master/io/src/openni_grabber.cpp @ 359 : No matching device found. openni_wrapper::OpenNIDevice::OpenNIDevice(xn::Context&, const xn::NodeInfo&, const xn::NodeInfo&, const xn::NodeInfo&, const xn::NodeInfo&) @ /home/schwarmcyc/pcl-master/io/src/openni_camera/openni_device.cpp @ 129 : creating depth generator failed. Reason: Xiron OS failed to wait on event!
Aborted (core dumped)

- sol:
sudo chmod +x ./install.sh
http://www.pcl-users.org/Can-t-use-Kinect-in-Ubuntu14-04-td4033666.html

### 4. Problem
**Note: if you get the error:**

InitFromXml failed: Failed to set USB interface!
the solution is to remove the gspca_linect kernel module:

sudo rmmod gspca_kinect

http://mitchtech.net/ubuntu-kinect-openni-primesense/

### 5.Problem
Device: PrimeSense/SensorV2/5.1.0.41: The device is not connected!

chmod +x install.sh

sudo ./install.sh
http://www.20papercups.net/programming/kinect-on-ubuntu-with-openni/