3D Camera

There are different kinds of cameras to obtain the depth, categorized according to the used technique:

  • Depth sensor: There are two elements which must always be present for depth sense: An IR (Infra-Red) projector, and an IR camera. Some depth sensors have an RGB (Red Green Blue) camera, some don't. The IR projector projects a pattern of IR light which falls on objects around it like a sea of dots. We can't see the dots because the light is projected in the Infra-Red color range. But the IR camera can see the dots. An IR camera is essentially the same as a regular RGB camera except that the images it captures are in the Infra-Red color range. So nothing too fancy going on there, still no actual depth sense. The camera sends it's video feed of this distorted dot pattern into the depth sensor's processor, and the processor works out depth from the displacement of the dots. On near objects the pattern is spread out, on far objects the pattern is dense.

  • Stereo camera: A type of 3d camera with two or more lenses with a separate image sensor or film frame for each lens. This allows the camera to simulate human binocular vision, and therefore gives it the ability to capture three-dimensional images.

  • ToF camera: It is a range imaging camera system that resolves distance based on the known speed of light, measuring the time-of-flight of a light signal between the camera and the subject for each point of the image. The time-of-flight camera is a class of scannerless LIDAR, in which the entire scene is captured with each laser or light pulse, as opposed to point-by-point with a laser beam such as in scanning LIDAR systems.
HRIM defines a 3D camera as
An optical instrument that provides a 3D reading about the environment.

So, apart from the common requirements (ID, Status, Power, Specs and Simulation, detailed in HRIM component model section), all 3D cameras will contain a topic referring to publishing a depth information of the environment.

As there is a lot of technological difference between the three types defined above, the information has been divided with the aim of making the section more understandable. The first table is common for all, however the rest collect the specific information that each type should include.

HRIM component model: 3DCamera
/hrim_sensor_3dcamera_<instance_id>/id hrim_generic_msgs/msg/ID.msg topic (Sub) M Component identification.
/hrim_sensor_3dcamera_<instance_id>/status hrim_generic_msgs/msg/Status.msg topic (Sub) M How the module is working.
/hrim_sensor_3dcamera_<instance_id>/power hrim_generic_msgs/msg/Power.msg topic (Sub) M Describes the power supply type and give the information about the module power consumption.
/hrim_sensor_3dcamera_<instance_id>/specs_comm hrim_generic_msgs/msg/SpecsCommunication.msg topic (Sub) M Published the capabilities in term of communicaction that the component offers.
/hrim_sensor_3dcamera_<instance_id>/state_comm hrim_generic_msgs/msg/StateCommunication.msg topic (Sub) M Published the resources that the component is using at the moment.
/hrim_sensor_3dcamera_<instance_id>/module_3d hrim_generic_msgs/msg/Simulation3D.msg topic (Sub) M Ask for the 3D model of the HRIM component.
/hrim_sensor_3dcamera_<instance_id>/module_urdf hrim_generic_msgs/msg/SimulationURDF.msg topic (Sub) M Ask for the information of, 3D model of the HRIM component.
/hrim_sensor_3dcamera_<instance_id>/specs hrim_sensor_3dcamera_msgs/msg/SpecsDepthSensor.msg
hrim_sensor_3dcamera_msgs/msg/SpecsStereoCamera.msg
hrim_sensor_3dcamera_msgs/msg/SpecsToFCamera.msg
topic (Sub) M 3D camera characteristics. Choose one of them.
/hrim_sensor_3dcamera_<instance_id>/point_cloud hrim_sensor_3dcamera_msgs/msg/PointCloud.msg topic (Sub) M Environment point cloud.
/hrim_sensor_camera_<instance_id>/audio_raw hrim_sensor_microphone_msgs/msg/Audio.msg topic (Sub) O Integrated microphone.
/hrim_sensor_3dcamera_<instance_id>/reconfiguration hrim_sensor_imu_msgs/msg/Reconfiguration.msg topic (Sub) O Dynamic reconfiguration of H-ROS systems. The integration of an IMU sensor is needed.
Parameters
frame_id string M Sets the TF frame from which the 3D camera is publishing.
weight int px M Resolution: weight (weight x height)
height int px M Resolution: height (weight x height)
frame_rate int fps M Refresh rate
image_format string - M Possible values are: ”rgb8”, ”rgba8”, ”rgb16”, ”rgba16”, ”bgr8”, ”bgra8”, ”bgr16”, ”bgra16”, ”mono8”, ”mono16”, ”bayerrggb8”, ”bayerbggr8”, ”bayergbrg8”, ”bayergrbg8”, ”bayerrggb16”, ”bayerbggr16”, ”bayergbrg16”, ”bayergrbg16”.:
frecuency_rate int Hz/s O Related to Audio.msg
Audio sampling frecuency.
NOTE: Mandatory if Audio.msg is added.
auto_calibration bool [true/false] O Enable/disable automatic calibration.
light bool [true/false] O If camera include a integrated light.
Enable/disable the light.
light_brightness float [0-100]% O If camera include a integrated light with the optional to control the brightness.
Adjust the brightness.
flash bool [true/false] O If camera include a integrated flash.
Enable/disable flash.
auto_flash bool [true/false] O If camera include a integrated flash.
Enable/disable flash automatic control.

In depth sensor case the manufacturer will have to include the following content, apart from the common from all the 3D Cameras (described in HRIMComponent model: 3DCamera).

HRIM component model: 3DCamera - Depth sensor
/hrim_sensor_3dcamera_<instance_id>/specs hrim_sensor_3dcamera_msgs/msg/SpecsDepthSensor.msg topic (Sub) M Depth sensor characteristics.
/hrim_sensor_3dcamera_<instance_id>/ir_image_raw hrim_sensor_camera_msgs/msg/Image.msg topic (Sub) M Raw image.
/hrim_sensor_3dcamera_<instance_id>/depth_image_raw hrim_sensor_camera_msgs/msg/Image.msg topic (Sub) M Raw image.
/hrim_sensor_3dcamera_<instance_id>/rgb_image_raw hrim_sensor_camera_msgs/msg/Image.msg topic (Sub) O Raw RGB image.
/hrim_sensor_3dcamera_<instance_id>/rgb_camera_info hrim_sensor_camera_msgs/msg/CameraInfo.msg topic (Sub) O RBG camera calibration and metadata.
/hrim_sensor_3dcamera_<instance_id>/depth_camera_info hrim_sensor_camera_msgs/msg/CameraInfo.msg topic (Sub) O Camera calibration and metadata.
/hrim_sensor_3dcamera_<instance_id>/ir_camera_info hrim_sensor_camera_msgs/msg/CameraInfo.msg topic (Sub) O IR camera calibration and metadata.
/hrim_sensor_3dcamera_<instance_id>/ir_projector_info hrim_sensor_camera_msgs/msg/CameraInfo.msg topic (Sub) O IR projector calibration.
/hrim_sensor_3dcamera_<instance_id>/point_cloud_color hrim_sensor_3dcamera_msgs/msg/PointCloud.msg topic (Sub) O Color point cloud.
/hrim_sensor_3dcamera_<instance_id>/set_camera_info hrim_sensor_camera_srvs/srv/SetCameraInfo.srv service

O Set camera calibration.
/hrim_sensor_3dcamera_<instance_id>/rgb_set_camera_info hrim_sensor_camera_srvs/srv/SetCameraInfo.srv service

O Set RGB camera calibration.
/hrim_sensor_3dcamera_<instance_id>/ir_set_camera_info hrim_sensor_camera_srvs/srv/SetCameraInfo.srv service

O Set IR camera calibration.

To create a stereo camera, the manufacturer will have to include the following content, apart from the common from all the 3D Cameras (described in HRIMComponent model: 3DCamera).

HRIM component model: 3DCamera - Stereo camera
/hrim_sensor_3dcamera_<instance_id>/specs hrim_sensor_3dcamera_msgs/msg/SpecsStereoCamera.msg topic (Sub) M Stereo camera characteristics.
/hrim_sensor_3dcamera_<instance_id>/left_image_raw hrim_sensor_camera_msgs/msg/Image.msg topic (Sub) M Left camera raw image.
/hrim_sensor_3dcamera_<instance_id>/right_image_raw hrim_sensor_camera_msgs/msg/Image.msg topic (Sub) M Right camera raw image.
/hrim_sensor_3dcamera_<instance_id>/left_camera_info hrim_sensor_camera_msgs/msg/CameraInfo.msg topic (Sub) O Left camera calibration and metadata.
/hrim_sensor_3dcamera_<instance_id>/left_image_raw_color hrim_sensor_camera_msgs/msg/Image.msg topic (Sub) O Color unrectified left image.
/hrim_sensor_3dcamera_<instance_id>/left_compressed hrim_sensor_camera_msgs/msg/CompressedImage.msg topic (Sub) O .jpg or .png
/hrim_sensor_3dcamera_<instance_id>/left_image_mono hrim_sensor_camera_msgs/msg/Image.msg topic (Sub) O Monochrome distorted.
/hrim_sensor_3dcamera_<instance_id>/left_image_color hrim_sensor_camera_msgs/msg/Image.msg topic (Sub) O Color distorted.
/hrim_sensor_3dcamera_<instance_id>/left_image_rect hrim_sensor_camera_msgs/msg/Image.msg topic (Sub) O >Monochrome, rectified.
/hrim_sensor_3dcamera_<instance_id>/left_image_rect_color hrim_sensor_camera_msgs/msg/Image.msg topic (Sub) O Color, rectified.
/hrim_sensor_3dcamera_<instance_id>/right_camera_info hrim_sensor_camera_msgs/msg/CameraInfo.msg topic (Sub) O Right camera calibration and metadata.
/hrim_sensor_3dcamera_<instance_id>/right_image_raw_color hrim_sensor_camera_msgs/msg/Image.msg topic (Sub) O Color unrectified right image.
/hrim_sensor_3dcamera_<instance_id>/right_compressed hrim_sensor_camera_msgs/msg/CompressedImage.msg topic (Sub) O Color unrectified right image.
/hrim_sensor_3dcamera_<instance_id>/right_image_mono hrim_sensor_camera_msgs/msg/Image.msg topic (Sub) O Monochrome distorted.
/hrim_sensor_3dcamera_<instance_id>/right_image_color hrim_sensor_camera_msgs/msg/Image.msg topic (Sub) O Color distorted.
/hrim_sensor_3dcamera_<instance_id>/rightt_image_rect hrim_sensor_camera_msgs/msg/Image.msg topic (Sub) O Monochrome, rectified.
/hrim_sensor_3dcamera_<instance_id>/right_image_rect_color hrim_sensor_camera_msgs/msg/Image.msg topic (Sub) O Color, rectified.
/hrim_sensor_3dcamera_<instance_id>/set_camera_info hrim_sensor_camera_srvs/srv/SetCameraInfo.srv service

O Set camera calibration.
/hrim_sensor_3dcamera_<instance_id>/right_set_camera_info hrim_sensor_camera_srvs/srv/SetCameraInfo.srv service

O Set right camera calibration
/hrim_sensor_3dcamera_<instance_id>/left_set_camera_info hrim_sensor_camera_srvs/srv/SetCameraInfo.srv service

O Set left camera calibration.

To create a ToF camera, the manufacturer will have to include the following content, apart from the common from all the 3D Cameras (described in HRIMComponent model: 3DCamera).

HRIM component model: 3DCamera - ToF camera
/hrim_sensor_3dcamera_<instance_id>/specs hrim_sensor_3dcamera_msgs/msg/SpecsToFCamera.msg topic (Sub) M ToF camera characteristics.
/hrim_sensor_3dcamera_<instance_id>/depth hrim_sensor_camera_msgs/msg/Image.msg topic (Sub) M Depth values.
/hrim_sensor_3dcamera_<instance_id>/confidence hrim_sensor_camera_msgs/msg/Image.msg topic (Sub) M Confidence values.
/hrim_sensor_3dcamera_<instance_id>/intensity hrim_sensor_camera_msgs/msg/Image.msg topic (Sub) M Intensity values.
/hrim_sensor_3dcamera_<instance_id>/distance hrim_sensor_rangefinder_msgs/msg/Distance.msg topic (Sub) O Distance to an object.
/hrim_sensor_3dcamera_<instance_id>/set_camera_info hrim_sensor_camera_srvs/srv/SetCameraInfo.srv service

O Set camera calibration.
Parameters
time_measurements double s O Related to Distance.msg
Time between measurements in seconds.
distance_min double m O Related to Distance.msg
The minimum distance to return in meters.
distance_max double m O Related to Distance.msg
The maximum distance to return in meters.