Added documents for migration to URP/HDRP, and other improvements. #188

Bob-Eric · 2025-01-30T15:10:07Z

Rewrote the Depth Camera Shader by Shader Graph to better support URP/HDRP. And added documents for migration. However, versions prior to Unity 2022 with built-in render pipeline need to use the legacy depth material.
The depth value of the depth camera means the perpendicular distance from the camera plane, not a straight line from the object to the camera lens. So it is corrected.
Changed asmdef to exclude Editor scripts when building.
Loaded the shader via a material reference and create the corresponding material. This means there is no longer need to add custom shaders to the Always Included Shaders when building.
Merged branch beta/v2.x.x. It is now possible to encode colour images in RGB8, depth images in 32FC1 and 16UC1 formats.
Added raycast layermask option for raycast-based lidar sensors.
Fixed a bug in TF sensor. Added TF and Clock example scene.
Updated to Unity 2023.2.

* Switch compressed image to raw image * Remove 'compressed' from camera topic name in camera prefab

…cs-Japan#182) * Update to test unity2020.3.x to 2022.3.x * Delete unnecessary files * Fix to delete unnecessary file copy

…pth camera support UPR/HDRP; load the shader via a material reference and create the corresponding material; rename the Color2Depth shader to DepthBufferLidar shader and use it for DepthBufferLidar; Change asmdef to exclude Editor scripts when packaging

… null references in RGBDCamera_ros prefab; adjusted texture format in RGBCameraSensor; updated ImageMsgSerializer to support raw texture data serialization .

…encoding

…scripts; added GPU read error logging to DepthCameraSensor; updated RaycastCommand API in IUpdateRaycastCommandsJob; added UnitySensorsROS to Some sensor prefabs; added Raycast LayerMask in RaycastLiDARSensor.

…laces the PointUtilities class and updates the associated prefabs; remove redundant Usings

…chCount in Job to improve performance; added logging of GPU read errors; removed unsafe code in PointCloud2MsgSerializer.

…re data into a single array; disabled HUD display for ROS TCP Connector to avoid lag caused by frequent GC at high frame rates; removed redundant AlwaysIncludedShaders

…mera frame_id format; fixed TFMessageMsgSerializer multiple TransformStampedMsg references to the same HeaderMsg

…isualizer error

RyodoTanaka · 2025-03-03T15:15:00Z

@Bob-Eric
This PR is awesome !!
Honestly, we are not good at shader, but you almost fix every problems !
Thank you so much.

I will check your update step by step.

RyodoTanaka · 2025-03-03T16:37:42Z

Check list

Rewrote the Depth Camera Shader by Shader Graph to better support URP/HDRP. And added documents for migration. However, versions prior to Unity 2022 with built-in render pipeline need to use the legacy shader.
The depth value of the depth camera means the perpendicular distance from the camera plane, not a straight line from the object to the camera lens. So it is corrected.
Changed asmdef to exclude Editor scripts when building.
Loaded the shader via a material reference and create the corresponding material. This means there is no longer need to add custom shaders to the Always Included Shaders when building.
Merged branch beta/v2.x.x. It is now possible to encode colour images in RGB8, depth images in 32FC1 and 16UC1 formats.
Added raycast layermask option for raycast-based lidar sensors.
Fixed a bug in TF sensor. Added TF and Clock example scene.
Updated to Unity 2023.2.

RyodoTanaka · 2025-03-04T04:45:43Z

I found those problems.

Environment

Unity : 2022.3.49f1
Branch : pr188/dev/upstream

Depth Buffer Lidar has incorrect images.

As following, this PR still has a problem on Depth Buffer Lidars.
With 360 deg Lidar, they will return not circle, but lined observation that mentioned on #114 .
Progress from the #144 is, the data looks not be bended comparing with #144 representation.
This issue can represent with Assets/UnitySensorsROS/Samples/LiDAR/Velodyne/Demo_VLP_128_with_DepthBuffer_ros.unity.

Depth Camera Color mistake

I found that with Legacy Shader, the blue and red colors are replaced as showing on following.
This issue can represent with Assets/UnitySensorsROS/Samples/Camera/Demo_RGBDCamera_ros.unity , after you change to the legacy shader.

@Bob-Eric
Do you have any idea to fix those ... ?

RyodoTanaka · 2025-03-04T04:48:20Z

@Bob-Eric
I have other question, to test your latest shader, which Unity version is necessary, 2023.2.x, right ?

Bob-Eric · 2025-03-04T04:56:48Z

@Bob-Eric I have other question, to test your latest shader, which Unity version is necessary, 2023.2.x, right ?

If you want to use URP or HDRP, 2022, 2023 are okay, but if you want to use built in pileline in unity 2022, you need to use the legacy depth material.

Bob-Eric · 2025-03-04T08:01:17Z

Depth Buffer Lidar has incorrect images

I didn't change the logic of the Depth Buffer lidar shader, so it should behave the same as before!
It's actually still bended, if you move the lidar up a little bit...

Also, I think it's possible to get the coordinates of the pixel point in the camera system directly in the ShaderGraph via the Position node.

Depth Camera Color mistake

As for the Depth Camera Color mistake, I think there was an error in the ROS message serialization part, I published the red point cloud directly and it showed blue in rviz...

The easiest way to do this is to flip the R and B channels in Assets/UnitySensors/Runtime/Scripts/Sensors/Camera/RGBDCamera/ITextureToColorPointsJob.cs
And you will get the correct color...

RyodoTanaka · 2025-03-10T04:30:16Z

@Bob-Eric
Thank you for your reply !

Depth Buffer Lidar has incorrect images

I see the modification is not changed, and this PR should be accepted, I think.

But we still need to fix this bending problem in other development.
So, if you have any idea to fix that, I really appreciate to know it and get PR.

Depth Camera Color mistake

I understand that the ROS software has mistake...
I think we should create some PR to them.
I do not switch R / G implementation.

So, I will accept all of this PR.

RyodoTanaka · 2025-03-03T15:27:48Z

Assets/UnitySensors/Runtime/Scripts/Sensors/IMU/IMUSensor.cs

This part, I will create Issue and we should deal with those FIXME topics.
#166

Update PR #188 example codes

alexandrefch and others added 17 commits October 7, 2024 18:27

Fix camera limitation due to jpg encoding (Field-Robotics-Japan#171)

5b73f20

* Switch compressed image to raw image * Remove 'compressed' from camera topic name in camera prefab

Update to test unity2020.3.x to 2022.3.x for beta/2.x.x (Field-Roboti…

739fdb7

…cs-Japan#182) * Update to test unity2020.3.x to 2022.3.x * Delete unnecessary files * Fix to delete unnecessary file copy

Update to latest master README (Field-Robotics-Japan#184)

d2b31f1

fixed depth camera texture pixel value in Color2DepthCamera.shader

c4e56f0

Add documents and related images migrated to URP/HDRP

e4f1c15

TODO: fix depth publish in ROS

ba45371

Merge branch 'beta/v2.x.x' into dev/fix_depth_ROS

39448e1

Added Unity scenes for Demo_RGBDCamera and Demo_RGBDCamera_ros; fixed…

37205d6

… null references in RGBDCamera_ros prefab; adjusted texture format in RGBCameraSensor; updated ImageMsgSerializer to support raw texture data serialization .

Updated ImageMsgSerializer to flip images and support rgb8 and 32fc1 …

560495a

…encoding

Removed distance inversion from DepthBufferLidar shader and related …

29e0c9a

…scripts; added GPU read error logging to DepthCameraSensor; updated RaycastCommand API in IUpdateRaycastCommandsJob; added UnitySensorsROS to Some sensor prefabs; added Raycast LayerMask in RaycastLiDARSensor.

Holds the point cloud material reference in PointUtilitiesSO and rep…

4e31334

…laces the PointUtilities class and updates the associated prefabs; remove redundant Usings

Added ImageEncodeJob to speed up Msg encoding; updated Schedule's Bat…

bbcb79c

…chCount in Job to improve performance; added logging of GPU read errors; removed unsafe code in PointCloud2MsgSerializer.

Refactored imageEncodeJob to update namespaces and merge target textu…

eb29f5a

…re data into a single array; disabled HUD display for ROS TCP Connector to avoid lag caused by frequent GC at high frame rates; removed redundant AlwaysIncludedShaders

Updated several prefabs to add PointUtilitiesSO references; fixed ca…

a2b6711

…mera frame_id format; fixed TFMessageMsgSerializer multiple TransformStampedMsg references to the same HeaderMsg

Add conditional compilation to TFMessageMsgSerializer for ROS2

58c7e1d

Added tf and clock examples

3e9cf14

Bob-Eric marked this pull request as ready for review January 30, 2025 15:17

Bob-Eric and others added 3 commits February 27, 2025 21:55

feat: add legacy depth camera shader; fix: depth camera point cloud v…

a2e8dff

…isualizer error

fix: use RGBA texture to support opengl

b7a4c2c

Merge branch 'master' into dev/upstream

a8c2290

RyodoTanaka self-requested a review March 3, 2025 15:13

RyodoTanaka mentioned this pull request Mar 4, 2025

After adding lights, the sensors cannot properly scan objects after packaging. #114

Open

Bob-Eric closed this Mar 4, 2025

Bob-Eric reopened this Mar 4, 2025

RyodoTanaka approved these changes Mar 10, 2025

View reviewed changes

RyodoTanaka merged commit 5bd4a0d into Field-Robotics-Japan:master Mar 10, 2025
0 of 4 checks passed

RyodoTanaka mentioned this pull request Mar 10, 2025

Fix for compiling Unity Player executable #190

Closed

Bob-Eric deleted the dev/upstream branch March 10, 2025 07:34

RyodoTanaka added a commit that referenced this pull request Mar 11, 2025

Merge pull request #192 from Field-Robotics-Japan/pr188/dev/upstream

e4f63c9

Update PR #188 example codes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added documents for migration to URP/HDRP, and other improvements. #188

Added documents for migration to URP/HDRP, and other improvements. #188

Bob-Eric commented Jan 30, 2025 •

edited

Loading

RyodoTanaka commented Mar 3, 2025

RyodoTanaka commented Mar 3, 2025 •

edited

Loading

RyodoTanaka commented Mar 4, 2025 •

edited

Loading

RyodoTanaka commented Mar 4, 2025 •

edited

Loading

Bob-Eric commented Mar 4, 2025 •

edited

Loading

Bob-Eric commented Mar 4, 2025 •

edited

Loading

RyodoTanaka commented Mar 10, 2025

RyodoTanaka Mar 3, 2025

Added documents for migration to URP/HDRP, and other improvements. #188

Added documents for migration to URP/HDRP, and other improvements. #188

Conversation

Bob-Eric commented Jan 30, 2025 • edited Loading

RyodoTanaka commented Mar 3, 2025

RyodoTanaka commented Mar 3, 2025 • edited Loading

RyodoTanaka commented Mar 4, 2025 • edited Loading

Depth Buffer Lidar has incorrect images.

Depth Camera Color mistake

RyodoTanaka commented Mar 4, 2025 • edited Loading

Bob-Eric commented Mar 4, 2025 • edited Loading

Bob-Eric commented Mar 4, 2025 • edited Loading

Depth Buffer Lidar has incorrect images

Depth Camera Color mistake

RyodoTanaka commented Mar 10, 2025

Depth Buffer Lidar has incorrect images

Depth Camera Color mistake

RyodoTanaka Mar 3, 2025

Choose a reason for hiding this comment

Bob-Eric commented Jan 30, 2025 •

edited

Loading

RyodoTanaka commented Mar 3, 2025 •

edited

Loading

RyodoTanaka commented Mar 4, 2025 •

edited

Loading

RyodoTanaka commented Mar 4, 2025 •

edited

Loading

Bob-Eric commented Mar 4, 2025 •

edited

Loading

Bob-Eric commented Mar 4, 2025 •

edited

Loading