PyOpenNI ROS Node (Subscriber)

PyOpenNI is a Python integration of OpenNI, a computer vision library that supports body and hand gesture recognition and tracking. However, in order to access the information in ROS, a ROS node must publish the data captured from the sensor. This tutorial will explain how to subscribe to the ROS PyOpenNI node as well as how to format your project to use the PyOpenNI node.

Node Initializiation

rospy.Subscriber('rgb', String, callback_rgb)
rospy.Subscriber('depth', String, callback_depth)
rospy.Subscriber('gesture', String, callback_gest)
rospy.Subscriber('skeleton', Skeleton, callback_skeleton)
rospy.spin()

Instantiate the ROS subscribers for RGB image, depth, gestures, and skeleton. Link the subscribers to each of their respective callback methods.

RGB and Depth Conversion

def callback_rgb(data):
    frame = np.fromstring(data.data, dtype=np.uint8).reshape(480, 640, 3)
def callback_depth(data):
    frame = np.fromstring(data.data, dtype=np.uint8).reshape(480, 640)

The RGB and Depth data is passed in the form of strings because of ROS restrictions on message datatypes. However, in order to use the data, they must be converted to some other useful form. Thus, in the callback methods, we convert the string into numpy arrays. The numpy.fromstring method converts the string data into an array of ints, which is reformatted into a multi-dimensional array with the reshape method.

Numpy arrays are compatible with all OpenCV methods, making them convenient to use. To display the array, simply use cv2.imshow and cv2.waitKey methods, which is much easier than using pygame.

Gestures

def callback_gest(data):
  print data

Compared to the RGB and depth callbacks, the gesture callback is incredibly simple. The name of the gesture is basically passed in the form of a string. For instance, if the user waved, the publisher would pass “wave” as its string.

Skeletons

Skeleton callbacks are a bit more complicated. First of all, notice that we are using a custom message type Skeleton. The custom message allows multiple information in skeleton to get passed at the same time. Here is the definition of Skeleton in rosmsg show:

[ros_py_openni/Skeleton]:
int8 id
string data

The id represents the user’s ID generated by the publisher, and the data are the points of their skeletons in order. Notice that this data is again passed in the form of a string. This is due to the ROS limitation that prevents python lists from being passed. In order to use it, however, we must convert the string back to a python list.

import ast

...

def callback_skeleton(data):

    newpos_skeleton = ast.literal_eval(data.data)

Through ast.literal_eval, we can convert the string back to a list of its appropriate datatype (meaning if the list originally contained floats, literal_eval will convert string back to a list of floats instead of a list of strings). What is convenient about literal_eval is that it can also recognize and convert nested lists, so the original list structure of skeleton is not distorted in any way.

To integrate the custom Skeleton message into your program, however, requires a few more steps. First, make a directory named msg inside your project directory. Open that directory and create a new text file named Skeleton.msg. Then inside that message file, add the following two lines:

int8 id
string data

Next, go into your package.xml and add these two lines:

<build_depend>message_generation</build_depend>

<run_depend>message_runtime</run_depend>

Then, go into your CMakeLists.txt and make sure that your find_package includes at least the following:

find_package(catkin REQUIRED COMPONENTS
  roscpp
  rospy
  std_msgs
  message_generation
)

After that, scroll down and uncomment the add_message_files lines. Delete the sample message files under FILES and instead put Skeleton.msg as your file name. Your add_message_files should look like this:

add_message_files(
  FILES
  Skeleton.msg
)

Uncomment the generate_messages lines:

generate_messages(
  DEPENDENCIES
  std_msgs  # Or other packages containing msgs
)

Finally, uncomment the catkin_package line and make sure that it at least says CATKIN_DEPENDS message_runtime. Others may be optional:

catkin_package(
#  INCLUDE_DIRS include
#  LIBRARIES beginner_tutorials
  CATKIN_DEPENDS message_runtime roscpp rospy std_msgs
#  DEPENDS system_lib
)

After you save your package.xml and CMakeLists.txt, make sure to do a catkin_make and source your setup.bash file again. Now, your project will have the OpenNI node fully integrated.

Helpful Links:

http://wiki.ros.org/ROS/Tutorials/CreatingMsgAndSrv: This tutorial from ROS explains in detail how to make a custom message type.

http://wiki.ros.org/msg: Here are some documentations of the ROS message type that include what are/are not allowed as ROS messages.

Share this:

Like this: