Multibeam Sonar | Project DAVE

[utterances-bot]

Multibeam Sonar | Project DAVE

Physics-based modelling and simulation of Multibeam Echosounder perception in Gazebo Classic, Multibeam Sonar Simulation

https://field-robotics-lab.github.io/dave.doc/contents/dave_sensors/Multibeam-Forward-Looking-Sonar/

[GriTRini]

terminate called after throwing an instance of 'pcl::UnorganizedPointCloudException'
what(): : Can't use 2D indexing with an unorganized point cloud
Aborted
[gazebo-2] process has died [pid 2122, exit code 134, cmd /home/jeong/catkin_ws/src/gazebo_ros_pkgs/gazebo_ros/scripts/gzserver -e ode /home/jeong/catkin_ws/src/sensor/worlds/empty.world __name:=gazebo __log:=/home/jeong/.ros/log/99600efe-e48e-11ed-bbd1-00155dfae7ae/gazebo-2.log].
log file: /home/jeong/.ros/log/99600efe-e48e-11ed-bbd1-00155dfae7ae/gazebo-2*.log

I ran blueview_p900_nps_multibeam_ray. But it doesn't run because the error works as shown. And when the blueview_p900_nps_multibeam model is executed, the sonar_image is displayed as a black screen and the same video as seen on YouTube is not displayed. What's the matter?

[beifannaobaowei]

Hello! I run roslaunch nps_uw_multibeam_sonar sonar_tank_blueview_p900_nps_multibeam.launch,but the windows doesn't appear.Could you help me solve it?

[mabelzhang]

@woensug-choi are you aware of this?

[woensug-choi]

any error msg on the console? did everything else worked? multibeam sonar requires specific installations

https://field-robotics-lab.github.io/dave.doc/contents/dave_sensors/Multibeam-Forward-Looking-Sonar/#installation

[huangbintao188]

I also encountered the same problem as beifannaobaowei , the method I used Option B. Install on the Host

[beifannaobaowei]

I also encountered the same problem as beifannaobaowei , the method I used Option B. Install on the Host

you need to install the Dave first！

[Nick-the-remaker]

I got trouble mounting a multibeam sonar to a Rexrov, can someone tell me how to do?

[mabelzhang]

Could you elaborate on what you mean by "trouble"? What exactly did you encounter, is it at development time or runtime, etc.?

[Nick-the-remaker]

Thank you, I wang to use the multibeam sonar sensor on the Rexrov, I notices a file 'Rexrov_oberon7.xacro' contains these comments:

I tried them all, but it didn't work, there's no ROS topic about multibeam sonar, could you please tell me the right way to use a multibeam sonar sensor on a Rexrov?

[Nick-the-remaker]

those comments are:

   <xacro:uuv_forward_looking_sonar
      namespace="${namespace}"
      suffix="down"
      parent_link="${namespace}/base_link"
      topic="forward_sonar"
      mass="0.015"
      update_rate="40"
      samples="100"
        fov="1.54719755"
        width="768"
        height="492">
      <inertia ixx="0.00001" ixy="0.0" ixz="0.0" iyy="0.00001" iyz="0.0" izz="0.00001" />
      <origin xyz="1.15 0 0.4" rpy="0 0.3 0"/>
      <visual>
        <geometry>
          <mesh filename="model://blueview_p900_nps_multibeam/meshes/p900.dae" />
        </geometry>
        <pose xyz="1.15 0 0.4" rpy="0 0.3 0"/>
      </visual>
   </xacro:uuv_forward_looking_sonar>
 
   <xacro:uuv_forward_multibeam_p900
      namespace="${namespace}"
      parent_link="${namespace}/base_link">
      <origin xyz="1.15 0 0.4" rpy="0 0.3 0"/>
   </xacro:uuv_forward_multibeam_p900>

   <xacro:uuv_fl_multibeam_generic
      namespace="${namespace}"
      parent_link="${namespace}/base_link">
      <origin xyz="1.15 0 0.4" rpy="0 0.3 0"/>
   </xacro:uuv_fl_multibeam_generic>

   <!-- creates beam lines in Gazebo view --> 
   <xacro:uuv_forward_multibeam_sonar_m450_130 namespace="$(arg namespace)" parent_link="$(arg namespace)/base_link">
      <origin xyz="1.4 0 0.65" rpy="0 0 0"/>
   </xacro:uuv_forward_multibeam_sonar_m450_130>

[mabelzhang]

I edited the code block for readability.

I think this is a question for @woensug-choi

[AmyPhung]

I followed through the install instructions on this page with a fresh install of Ubuntu 22.04, CUDA 12.3, with a NVIDIA GeForce RTX 3070 card with driver version 535. Everything seems to run without any errors (nvidia-smi and nvcc --version also work as expected), but I just get a black image in the pop-up image window even when I put objects in front of the sonar in Gazebo. This happens for both the raster (roslaunch nps_uw_multibeam_sonar sonar_tank_oculus_m1200d_nps_multibeam.launch) and ray (roslaunch nps_uw_multibeam_sonar sonar_tank_blueview_p900_nps_multibeam_ray.launch) versions. When I put objects in front of the sonar, the blue rays do appear to "hit" them, and the pointcloud shows the objects correctly when viewed in RVIZ. I used option B (git clone manually) for obtaining a copy of the repositories.

While running, I get printouts that look like this in the terminal:

Sonar Ray Summation 5/100 [s]
GPU Window & Correction 1/100 [s]
GPU FFT Calc Time 0/100 [s]
[ INFO] [1707942817.247347951, 102.471000000]: GPU Sonar Frame Calc Time 10/100 [s]

Any pointers for how to proceed? And do I need the full DAVE simulator built and running too, or is the multibeam simulator stand-alone? Looking through the code I didn't see anywhere the multibeam simulator was dependent on the DAVE repos, but I could be wrong

[Nick-the-remaker]

I followed through the install instructions on this page with a fresh install of Ubuntu 22.04, CUDA 12.3, with a NVIDIA GeForce RTX 3070 card with driver version 535. Everything seems to run without any errors (nvidia-smi and nvcc --version also work as expected), but I just get a black image in the pop-up image window even when I put objects in front of the sonar in Gazebo. This happens for both the raster (roslaunch nps_uw_multibeam_sonar sonar_tank_oculus_m1200d_nps_multibeam.launch) and ray (roslaunch nps_uw_multibeam_sonar sonar_tank_blueview_p900_nps_multibeam_ray.launch) versions. When I put objects in front of the sonar, the blue rays do appear to "hit" them, and the pointcloud shows the objects correctly when viewed in RVIZ. I used option B (git clone manually) for obtaining a copy of the repositories.

While running, I get printouts that look like this in the terminal:

Sonar Ray Summation 5/100 [s]
GPU Window & Correction 1/100 [s]
GPU FFT Calc Time 0/100 [s]
[ INFO] [1707942817.247347951, 102.471000000]: GPU Sonar Frame Calc Time 10/100 [s]

Any pointers for how to proceed? And do I need the full DAVE simulator built and running too, or is the multibeam simulator stand-alone? Looking through the code I didn't see anywhere the multibeam simulator was dependent on the DAVE repos, but I could be wrong

The multibeam sonar is stand alone, I edit some parts to combine the Rexrov and multibeam sonar, maybe it will be helpful to you.

in src/dave/urdf/dave_snippets/urdf/sensor_snippets.xacro, add:

<xacro:include filename="$(find nps_uw_multibeam_sonar)/urdf/multibeam_sonar_blueview_p900.xacro" />

in src/dave/urdf/robots/rexrov_description/urdf/rexrov_sensors.xacro, add:

<!-- Sonar sensor -->
<xacro:multibeam_blueview_p900_raster_macro name="blueview_p900" scale="'1 1 1'" sonar_image_topic="sonar_image" debug="false" sonar_image_raw_topic="sonar_image_raw" maxDistance="12" raySkips="10" plotScaler="0" sensorGain="0.4" writeLog="false" writeFrameInterval="5" artificialVehicleVibration="false" constantReflectivity="true" reflectivityDatabaseFile="variationalReflectivityDatabase" robot_link="${namespace}/base_link" ray_visual="true">
    <origin xyz="1.6 0 0.6" rpy="0 0 0"/>
</xacro:multibeam_blueview_p900_raster_macro>

in src/nps_uw_multibeam_sonar/urdf/multibeam_sonar_blueview_p900.xacro, edit this block:

<xacro:macro name="multibeam_blueview_p900_raster_macro"
               params="name scale sonar_image_topic debug
               sonar_image_raw_topic maxDistance raySkips plotScaler sensorGain writeLog
               writeFrameInterval artificialVehicleVibration constantReflectivity
               reflectivityDatabaseFile robot_link ray_visual">

add a '*origin' to this block:

<xacro:macro name="multibeam_blueview_p900_raster_macro"
               params="name scale sonar_image_topic debug
               sonar_image_raw_topic maxDistance raySkips plotScaler sensorGain writeLog
               writeFrameInterval artificialVehicleVibration constantReflectivity
               reflectivityDatabaseFile robot_link ray_visual *origin">

in src/dave/examples/dave_demo_launch/launch/dave_demo.launch, add:

<!-- image_view for sonar image -->
    <node name="image_view_sonar" pkg="image_view" type="image_view"
            args="image:=/rexrov/blueview_p900/sonar_image">
        <param name="window_name" value="blueview_p900" />
        <param name="autosize" value="true" />
        <param name="filename_format" value="/tmp/SonarImage_capture_%04i.jpg" />
    </node>

    <node pkg="tf" type="static_transform_publisher" name="multibeam_sonar_base_link" args="0 0 0 0 0 0 world blueview_p900_link 50" />

Finally, run roslaunch dave_demo_launch dave_demo.launch
good luck

[AmyPhung]

Thanks for the pointers. Unfortunately, I'm still getting the same issue of the sonar image showing up blank, even after installing Dave. Last time I went through the install instructions I used option B (git cloning the repos), and when I ran roslaunch nps_uw_multibeam_sonar sonar_tank_blueview_p900_nps_multibeam.launch a tank with objects didn't show up like they did in the instructions. I wiped everything and re-did the install, using option a (vcs import) instead. This time, when I ran the same command the tank showed up but the sonar is still black. I don't get any errors in the terminal, but I'm still not getting the sonar image to show anything useful. Here's how I'm running the sim:

. ~/rocker_venv_cuda/bin/activate
cd ~/rocker_venv_cuda/dockwater
./run.bash -c dockwater:noetic
source ~/uuv_ws/devel/setup.bash
roslaunch nps_uw_multibeam_sonar sonar_tank_blueview_p900_nps_multibeam.launch

Here's the terminal output:

roslaunch nps_uw_multibeam_sonar sonar_tank_blueview_p900_nps_multibeam.launch
... logging to /home/chaoslab/.ros/log/72a3d392-d1c1-11ee-94af-0242ac110002/roslaunch-265f69df5edc-1957.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://265f69df5edc:34517/

SUMMARY
========

PARAMETERS
 * /gazebo/enable_ros_network: True
 * /image_view_sonar/autosize: True
 * /image_view_sonar/filename_format: /tmp/SonarImage_c...
 * /image_view_sonar/window_name: blueview_p900
 * /rosdistro: noetic
 * /rosversion: 1.16.0
 * /use_sim_time: True

NODES
  /
    gazebo (gazebo_ros/gzserver)
    gazebo_gui (gazebo_ros/gzclient)
    image_view_sonar (image_view/image_view)
    multibeam_sonar_base_link (tf/static_transform_publisher)

auto-starting new master
process[master]: started with pid [1965]
ROS_MASTER_URI=http://localhost:11311

setting /run_id to 72a3d392-d1c1-11ee-94af-0242ac110002
process[rosout-1]: started with pid [1975]
started core service [/rosout]
process[gazebo-2]: started with pid [1982]
process[gazebo_gui-3]: started with pid [1987]
process[image_view_sonar-4]: started with pid [1992]
process[multibeam_sonar_base_link-5]: started with pid [1993]

(blueview_p900:1992): dbind-WARNING **: 15:32:08.463: Couldn't connect to accessibility bus: Failed to connect to socket /run/user/1000/at-spi/bus_1: No such file or directory
[ INFO] [1708633928.835523646]: Finished loading Gazebo ROS API Plugin.
[ INFO] [1708633928.836481996]: waitForService: Service [/gazebo/set_physics_properties] has not been advertised, waiting...
[ INFO] [1708633928.904529434]: Finished loading Gazebo ROS API Plugin.
[ INFO] [1708633928.905264577]: waitForService: Service [/gazebo_gui/set_physics_properties] has not been advertised, waiting...
amdgpu: os_same_file_description couldn't determine if two DRM fds reference the same file description.
If they do, bad things may happen!
[ INFO] [1708633929.265721333]: waitForService: Service [/gazebo/set_physics_properties] is now available.
[ INFO] [1708633929.275588218]: Physics dynamic reconfigure ready.
[ INFO] [1708633931.234422903]: 
[ INFO] [1708633931.234468017]: ==================================================
[ INFO] [1708633931.234477234]: ============   SONAR PLUGIN LOADED   =============
[ INFO] [1708633931.234484849]: ==================================================
[ INFO] [1708633931.234491982]: ============      RASTER VERSION     =============
[ INFO] [1708633931.234498855]: ==================================================
[ INFO] [1708633931.234515326]: Maximum view range  [m] = 10
[ INFO] [1708633931.234526767]: Distance resolution [m] = 0.0501672
[ INFO] [1708633931.234534541]: # of Beams = 512
[ INFO] [1708633931.234542847]: # of Rays / Beam (Elevation, Azimuth) = (228, 1)
[ INFO] [1708633931.234550722]: Calculation skips (Elevation) = 10
[ INFO] [1708633931.234557554]: # of Time data / Beam = 399
[ INFO] [1708633931.234571721]: Reflectivity method : Constant
[ INFO] [1708633931.234582872]: ==================================================
[ INFO] [1708633931.234594774]: 
[ INFO] [1708633931.234614471]: Raw data at /tmp/SonarRawData_{numbers}.csv
[ INFO] [1708633931.234631833]: every 5 frames
[ INFO] [1708633931.234647122]: 
[ INFO] [1708633931.239816571]: Camera Plugin: The 'robotNamespace' param did not exit
[ INFO] [1708633931.242668940]: Camera Plugin (ns = )  <tf_prefix_>, set to ""
GPU Sonar Computation Time 146/100 [s]
Sonar Ray Summation 5/100 [s]
GPU Window & Correction 1/100 [s]
GPU FFT Calc Time 0/100 [s]
[ INFO] [1708633933.090993722, 1.842000000]: GPU Sonar Frame Calc Time 153/100 [s]

GPU Sonar Computation Time 2/100 [s]
Sonar Ray Summation 5/100 [s]
GPU Window & Correction 1/100 [s]
GPU FFT Calc Time 0/100 [s]
[ INFO] [1708633933.382715677, 2.134000000]: GPU Sonar Frame Calc Time 9/100 [s]

GPU Sonar Computation Time 2/100 [s]
Sonar Ray Summation 5/100 [s]
GPU Window & Correction 1/100 [s]
GPU FFT Calc Time 0/100 [s]
[ INFO] [1708633933.561362956, 2.312000000]: GPU Sonar Frame Calc Time 10/100 [s]

GPU Sonar Computation Time 2/100 [s]
Sonar Ray Summation 5/100 [s]
GPU Window & Correction 1/100 [s]
GPU FFT Calc Time 0/100 [s]
[ INFO] [1708633933.745047807, 2.496000000]: GPU Sonar Frame Calc Time 10/100 [s]

GPU Sonar Computation Time 2/100 [s]
Sonar Ray Summation 5/100 [s]
GPU Window & Correction 1/100 [s]
GPU FFT Calc Time 0/100 [s]
[ INFO] [1708633933.926459492, 2.677000000]: GPU Sonar Frame Calc Time 10/100 [s]

And here's what shows up:

I followed @Nick-the-remaker 's instructions to add the multibeam to Dave, which also runs without errors but results in the same issue

I suspected there might be something wrong with CUDA or my NVIDIA drivers, but as far as I can tell those seem to be working properly. From inside the container: nvcc --version produces:

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Wed_Nov_22_10:17:15_PST_2023
Cuda compilation tools, release 12.3, V12.3.107
Build cuda_12.3.r12.3/compiler.33567101_0

and from the container, nvidia-smi produces:

nvidia-smi
Thu Feb 22 15:38:27 2024       
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.154.05             Driver Version: 535.154.05   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce RTX 3070 ...    Off | 00000000:01:00.0 Off |                  N/A |
| N/A   39C    P0              27W /  80W |      8MiB /  8192MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
                                                                                         
+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
+---------------------------------------------------------------------------------------+

Any additional pointers for getting the multibeam sim to run would be appreciated!