乐视体感astra pro深度摄像头在ros系统获取深度图像彩色图像无色彩点云数据彩色点云数据

1.astra pro深度摄像头介绍
2.astra pro驱动安装
3.astra pro获取深度图像无色彩pointCloud2
4.astra pro获取彩色图像带彩色的pointCloud2

1.astra pro深度摄像头介绍

摄像头有四个镜头,从左到右的镜头分别是1 ir发射源 2 最小距离传感器 3 彩色摄像头 4 ir接收器

2.astra pro驱动安装

2.1下载驱动及依赖

ubuntu16 kinetic及ubuntu18 melodic下载以下驱

OpenNI-Linux-x64-2.3.tar.bz2

$sudo apt-get install build-essential freeglut3 freeglut3-dev
$ldconfig -p | grep libudev.so.1
$cd /lib/x86_64-linux-gnu
$sudo ln -s libudev.so.x.x.x libudev.so.1

2.2编译驱动

解压下载的OpenNI-Linux-x64-2.3驱动

$ cd OpenNI-Linux-x64-2.3
$ sudo chmod a+x install.sh

安装

$ sudo chmod a+x install.sh
$ sudo ./install.sh

插拔摄像头,source环境

$ source OpenNIDevEnvironment

编译

$ cd Samples/SimpleViewer
$ make

连接设备，执行示例

$ cd Bin/x64-Release
$ ./SimpleViewer

编译安装libuvc

$ cd ~
$ git clone https://github.com/ktossell/libuvc
$ cd libuvc
$ mkdir build
$ cd build
$ cmake ..
$ make && sudo make install
$ sudo cp /usr/include/libusb-1.0/libusb.h  /usr/local/include/libuvc/

2.3 安装ros的官方astra_launch astra_camera libuvc rgbd包， ubuntu melodic的需要下载源码安装这些包，因为apt-get不到

如果提示源码安装找不到包，到ros wiki上更新一下本地的ros软件源

$ sudo apt-get install ros-kinetic-astra-camera 
$ sudo apt-get install ros-kinetic-astra-launch  
$ sudo apt-get install ros-kinetic-libuvc-*
$ sudo apt-get install ros-kinetic-rgbd-launch

3.astra pro获取深度图像无色彩pointCloud2

$ roslaunch astra_launch astra.launch

打开rqt_image_view查看图像话题

$ rqt_image_view

打开rviz查看无彩色点云数据

$ rviz

4.astra pro获取彩色图像带彩色的pointCloud2

因为astra的驱动无发布彩色图像，导致点云数据也无彩色数据.

有三种思路取解决：

1.修改官方驱动

2.修改ros的底层astra_camera 和 rgbd_launch 的适配

3.另外新开一个节点监听无彩色pointCloud2数据和彩色数据，用彩色数据的rgb通道数值填充pointCloud2的rgbd的通道，在重新发布出来就是彩色的点云数据

这里采用第二种做法，第一做法难度大，第三做法耗电脑资源

4.1获取彩色图像

这里采用ros的uvc_camera包来获取彩色图像

$ sudo apt-get install uvc_camera

新建一个launch文件来启动uvc_camera

<launch><node pkg="uvc_camera" type="uvc_camera_node" name="uvc_camera" output="screen"><param name="device" type="string" value="/dev/video1" /> <param name="width" type="int" value="640" /><param name="height" type="int" value="480" /><param name="fps" type="int" value="30" /><param name="frame_id" type="string" value="camera_rgb_optical_frame" /></node>
</launch>

/dev/video1时摄像头的设备号,查看一下摄像头的设备号

$ ls /dev

启动上面的launch文件

$ roslauch uvc.launch

使用rqt_image_view查看

当然也可以采用usb_cam来获取彩色图像，参数ros wiki上有

4.2 校准彩色摄像头和深度摄像头

获取彩色点云数据需要对相机进行校准(个人感觉比kinect校准难，校准了好多遍)

校准深度摄像头

$ roslaunch astra_launch astrapro.launch
$ rosrun camera_calibration cameracalibrator.py image:=/camera/ir/image camera:=/camera/ir --size 8x6 --square 0.024

把校准得到的yaml文件的camera_name值设为 camera_name: depth_Astra_Orbbec

校准彩色摄像头

$ rosrun uvc_camera uvc_camera_node device:=/dev/video1 
$ rosrun camera_calibration cameracalibrator.py image:=/image_raw camera:=/ --size 8x6 --square 0.024

把校准得到的yaml文件的camera_name值设为 camera_name: rgb_Astra_Orbbec

创建launch文件

<launch><!-- "camera" should uniquely identify the device. All topics are pushed downinto the "camera" namespace, and it is prepended to tf frame ids. --><arg name="camera" default="camera" /><arg name="rgb_frame_id"   default="$(arg camera)_rgb_optical_frame" /><arg name="depth_frame_id" default="$(arg camera)_depth_optical_frame" /><!-- device_id can have the following formats:"#n"            : the nth device found, starts from 1"2@n"           : the nth device on USB bus 2, n starts from 1"2bc5/0401@1/6" : uri in format <vendor ID>/<product ID>@<bus number>/<device number>"15120410023"   : serial number --><arg name="device_id" default="#1" /><arg name="bootorder" default="0" /><arg name="devnums" default="1" /><!-- By default, calibrations are stored to file://${ROS_HOME}/camera_info/${NAME}.yaml,where ${NAME} is of the form "[rgb|depth]_[serial#]", e.g. "depth_B00367707227042B".See camera_info_manager docs for calibration URL details. --><!--<arg name="rgb_camera_info_url"   default="" /><arg name="depth_camera_info_url" default="" />--><arg name="rgb_camera_info_url"   default="file://$(find drive)/rgb.yaml" /><arg name="depth_camera_info_url" default="file://$(find drive)/ir.yaml" /><!-- Hardware depth registration --><arg name="depth_registration" default="false" /><!-- Driver parameters --><arg name="color_depth_synchronization"     default="false" /><arg name="auto_exposure"                   default="true" /><arg name="auto_white_balance"              default="true" /><!-- Arguments for remapping all device namespaces --><arg name="rgb"              default="rgb" /><arg name="ir"               default="ir" /><arg name="depth"            default="depth" /><!-- Optionally suppress loading the driver nodelet and/or publishing the default tftree. Useful if you are playing back recorded raw data from a bag, or aresupplying a more accurate tf tree from calibration. --><arg name="load_driver" default="true" /><arg name="publish_tf" default="true" /><!-- Processing Modules --><arg name="rgb_processing"                  default="true"  /><arg name="debayer_processing"              default="false" /><arg name="ir_processing"                   default="false" /><arg name="depth_processing"                default="true" /><arg name="depth_registered_processing"     default="true" /><arg name="disparity_processing"            default="false" /><arg name="disparity_registered_processing" default="false" /><arg name="hw_registered_processing"        default="true" if="$(arg depth_registration)" /><arg name="sw_registered_processing"        default="false" if="$(arg depth_registration)" /><arg name="hw_registered_processing"        default="false" unless="$(arg depth_registration)" /><arg name="sw_registered_processing"        default="true" unless="$(arg depth_registration)" /><!-- Disable bond topics by default --><arg name="respawn" default="false" /><!-- Worker threads for the nodelet manager --><arg name="num_worker_threads" default="4" /><!-- Push down all topics/nodelets into "camera" namespace --><group ns="$(arg camera)"><!-- Start nodelet manager --><arg name="manager" value="$(arg camera)_nodelet_manager" /><arg name="debug" default="false" /> <!-- Run manager in GDB? --><include file="$(find rgbd_launch)/launch/includes/manager.launch.xml"><arg name="name" value="$(arg manager)" /><arg name="debug" value="$(arg debug)" /><arg name="num_worker_threads"  value="$(arg num_worker_threads)" /></include><!-- Load driver --><include file="$(find astra_launch)/launch/includes/device.launch.xml" if="$(arg load_driver)"><arg name="manager"                         value="$(arg manager)" /><arg name="device_id"                       value="$(arg device_id)" /><arg name="bootorder"                       value="$(arg bootorder)" /><arg name="devnums"                         value="$(arg devnums)" /><arg name="rgb_frame_id"                    value="$(arg rgb_frame_id)" /><arg name="depth_frame_id"                  value="$(arg depth_frame_id)" /><arg name="rgb_camera_info_url"             value="$(arg rgb_camera_info_url)" /><arg name="depth_camera_info_url"           value="$(arg depth_camera_info_url)" /><arg name="rgb"                             value="$(arg rgb)" /><arg name="ir"                              value="$(arg ir)" /><arg name="depth"                           value="$(arg depth)" /><arg name="respawn"                         value="$(arg respawn)" /><arg name="depth_registration"              value="$(arg depth_registration)" /><arg name="color_depth_synchronization"     value="$(arg color_depth_synchronization)" /><arg name="auto_exposure"                   value="$(arg auto_exposure)" /><arg name="auto_white_balance"              value="$(arg auto_white_balance)" /></include><!-- Load standard constellation of processing nodelets --><include file="$(find rgbd_launch)/launch/includes/processing.launch.xml"><arg name="manager"                         value="$(arg manager)" /><arg name="rgb"                             value="$(arg rgb)" /><arg name="ir"                              value="$(arg ir)" /><arg name="depth"                           value="$(arg depth)" /><arg name="respawn"                         value="$(arg respawn)" /><arg name="rgb_processing"                  value="$(arg rgb_processing)" /><arg name="debayer_processing"              value="$(arg debayer_processing)" /><arg name="ir_processing"                   value="$(arg ir_processing)" /><arg name="depth_processing"                value="$(arg depth_processing)" /><arg name="depth_registered_processing"     value="$(arg depth_registered_processing)" /><arg name="disparity_processing"            value="$(arg disparity_processing)" /><arg name="disparity_registered_processing" value="$(arg disparity_registered_processing)" /><arg name="hw_registered_processing"        value="$(arg hw_registered_processing)" /><arg name="sw_registered_processing"        value="$(arg sw_registered_processing)" /></include><node pkg="uvc_camera" type="uvc_camera_node" name="uvc_camera" output="screen"><param name="device"  value="/dev/video1" /> <param name="camera_info_url"  value="$(arg rgb_camera_info_url)" /><param name="camera_name" value="rgb_Astra_Orbbec"/><param name="width"  value="640" /><param name="height"  value="480" /><param name="fps" value="30" /><param name="frame_id"  value="camera_rgb_optical_frame" /><remap from="/camera/camera_info" to="/camera/rgb/camera_info"/><remap from="/camera/image_raw" to="/camera/rgb/image_raw"/></node></group> <!-- camera --><!-- Load reasonable defaults for the relative pose between cameras --><include file="$(find astra_launch)/launch/includes/astra_frames.launch" if="$(arg publish_tf)"><arg name="camera" value="$(arg camera)" /></include></launch>

自行修改ir 和 rgb两个文件的路径，启动这个launch文件打开rviz订阅/camera/depth_registered/points就可以看到彩色点云了