Apollo car does not move

[FitashUlHaq]

Hi,

I have configured apollo 6.0 with lgsvl, all the other items are working but the control command is not working I have attached an image.

any work around for this?

[FitashUlHaq]

Update: Controls are now receiving data but car is still not moving

Apollo controls:

[ntutangyun]

looks like there is no planed trajectory. you may use cyber_monitor to check the planning module's output.

[FitashUlHaq]

Thank you for the reply, i am facing a new error when i restarted apollo and lgsvl.

it seems there is communication error betweeen apollo and lgsvl, I can see simulator image on apollo dreamview

but nothing else (location and movement of ego vehicle).

and the localisation show this error "Localization: ERROR: ERROR: Both GNSS and Lidar Localization Is Invalid"

[ntutangyun]

from the spawning point, seems that you're using the old version of the Borregas Ave map.

You could download the latest map here

remember to update the map files in apollo's folder as well.

use cyber_monitor to check the actual odometry data sent from the LGSVL simulator, and check what went wrong.

for the ego to move, it needs to have the perception module working or taking ground truth perception from the simulator.

from the first pic in the last reply, perception is not working properly.

[FitashUlHaq]

can it be due to using RTX 2080 ti?

[FitashUlHaq]

Thank you the reply, I will download and use the latest map, the same setup was working yesterday. I didnt installed or changed anything

[ntutangyun]

not sure about the GPU. you could try manually running the perception module and see if it crashes or not.

[FitashUlHaq]

can you please guide me how can I run the perception module? if their is any docs available please provide me the link

[ntutangyun]

you could try cyber_launch start modules/perception/production/launch/perception.launch in one apollo's docker terminal.

[FitashUlHaq]

following is the output I get from running this command

[cyber_launch_14379] INFO Launch file [/apollo/modules/perception/production/launch/perception.launch]
[cyber_launch_14379] INFO ========================================================================================================================
[cyber_launch_14379] INFO Load module [perception] library: [lidar_perception] [CYBER_DEFAULT] conf: [/apollo/modules/perception/production/dag/dag_streaming_perception.dag] exception_handler: []
[cyber_launch_14379] INFO Start process [lidar_perception] successfully. pid: 14380
[cyber_launch_14379] INFO ------------------------------------------------------------------------------------------------------------------------
[cyber_launch_14379] INFO Load module [perception_camera] library: [camera_perception] [CYBER_DEFAULT] conf: [/apollo/modules/perception/production/dag/dag_streaming_perception_camera.dag] exception_handler: []
[cyber_launch_14379] INFO Start process [camera_perception] successfully. pid: 14382
[cyber_launch_14379] INFO ------------------------------------------------------------------------------------------------------------------------
[cyber_launch_14379] INFO Load module [motion_service] library: [motion_service] [CYBER_DEFAULT] conf: [/apollo/modules/perception/production/dag/dag_motion_service.dag] exception_handler: []
[cyber_launch_14379] INFO Start process [motion_service] successfully. pid: 14384
[cyber_launch_14379] INFO ------------------------------------------------------------------------------------------------------------------------
[lidar_perception] WARNING: Logging before InitGoogleLogging() is written to STDERR
[lidar_perception] I0107 16:43:04.225636 14380 module_argument.cc:81] []command: mainboard -d /apollo/modules/perception/production/dag/dag_streaming_perception.dag -p lidar_perception -s CYBER_DEFAULT
[lidar_perception] I0107 16:43:04.226130 14380 global_data.cc:155] []host ip: 10.1.2.221
[lidar_perception] I0107 16:43:04.226300 14380 module_argument.cc:57] []binary_name_ is mainboard, process_group_ is lidar_perception, has 1 dag conf
[lidar_perception] I0107 16:43:04.226308 14380 module_argument.cc:60] []dag_conf: /apollo/modules/perception/production/dag/dag_streaming_perception.dag
[camera_perception] WARNING: Logging before InitGoogleLogging() is written to STDERR
[camera_perception] I0107 16:43:04.226500 14382 module_argument.cc:81] []command: mainboard -d /apollo/modules/perception/production/dag/dag_streaming_perception_camera.dag -p camera_perception -s CYBER_DEFAULT
[camera_perception] I0107 16:43:04.227026 14382 global_data.cc:155] []host ip: 10.1.2.221
[camera_perception] I0107 16:43:04.227185 14382 module_argument.cc:57] []binary_name_ is mainboard, process_group_ is camera_perception, has 1 dag conf
[camera_perception] I0107 16:43:04.227193 14382 module_argument.cc:60] []dag_conf: /apollo/modules/perception/production/dag/dag_streaming_perception_camera.dag
[motion_service] WARNING: Logging before InitGoogleLogging() is written to STDERR
[motion_service] I0107 16:43:04.232636 14384 module_argument.cc:81] []command: mainboard -d /apollo/modules/perception/production/dag/dag_motion_service.dag -p motion_service -s CYBER_DEFAULT
[motion_service] I0107 16:43:04.233132 14384 global_data.cc:155] []host ip: 10.1.2.221
[motion_service] I0107 16:43:04.233296 14384 module_argument.cc:57] []binary_name_ is mainboard, process_group_ is motion_service, has 1 dag conf
[motion_service] I0107 16:43:04.233304 14384 module_argument.cc:60] []dag_conf: /apollo/modules/perception/production/dag/dag_motion_service.dag
[motion_service] [NVBLAS] NVBLAS_CONFIG_FILE environment variable is NOT set : relying on default config filename 'nvblas.conf'
[motion_service] [NVBLAS] Cannot open default config file 'nvblas.conf'
[motion_service] [NVBLAS] Config parsed
[motion_service] [NVBLAS] CPU Blas library need to be provided
[camera_perception] [NVBLAS] NVBLAS_CONFIG_FILE environment variable is NOT set : relying on default config filename 'nvblas.conf'
[camera_perception] [NVBLAS] Cannot open default config file 'nvblas.conf'
[camera_perception] [NVBLAS] Config parsed
[camera_perception] [NVBLAS] CPU Blas library need to be provided
[camera_perception] WARNING: The Permute Plugin is being deprecated and will be removed in the next TensorRT release. Please use the TensorRT Shuffle layer for Permute operation
[camera_perception] WARNING: The Permute Plugin is being deprecated and will be removed in the next TensorRT release. Please use the TensorRT Shuffle layer for Permute operation
[camera_perception] WARNING: The Permute Plugin is being deprecated and will be removed in the next TensorRT release. Please use the TensorRT Shuffle layer for Permute operation
[camera_perception] WARNING: The Permute Plugin is being deprecated and will be removed in the next TensorRT release. Please use the TensorRT Shuffle layer for Permute operation
[camera_perception] WARNING: The Permute Plugin is being deprecated and will be removed in the next TensorRT release. Please use the TensorRT Shuffle layer for Permute operation
[camera_perception] WARNING: The Permute Plugin is being deprecated and will be removed in the next TensorRT release. Please use the TensorRT Shuffle layer for Permute operation
[camera_perception] WARNING: The Permute Plugin is being deprecated and will be removed in the next TensorRT release. Please use the TensorRT Shuffle layer for Permute operation
[camera_perception] WARNING: The Permute Plugin is being deprecated and will be removed in the next TensorRT release. Please use the TensorRT Shuffle layer for Permute operation
[camera_perception] WARNING: The Permute Plugin is being deprecated and will be removed in the next TensorRT release. Please use the TensorRT Shuffle layer for Permute operation
[camera_perception] WARNING: The Permute Plugin is being deprecated and will be removed in the next TensorRT release. Please use the TensorRT Shuffle layer for Permute operation
[camera_perception] WARNING: The Permute Plugin is being deprecated and will be removed in the next TensorRT release. Please use the TensorRT Shuffle layer for Permute operation
[camera_perception] WARNING: The Permute Plugin is being deprecated and will be removed in the next TensorRT release. Please use the TensorRT Shuffle layer for Permute operation
[camera_perception] WARNING: The Permute Plugin is being deprecated and will be removed in the next TensorRT release. Please use the TensorRT Shuffle layer for Permute operation
[lidar_perception] [NVBLAS] NVBLAS_CONFIG_FILE environment variable is NOT set : relying on default config filename 'nvblas.conf'
[lidar_perception] [NVBLAS] Cannot open default config file 'nvblas.conf'
[lidar_perception] [NVBLAS] Config parsed
[lidar_perception] [NVBLAS] CPU Blas library need to be provided
[lidar_perception] E0107 16:43:09.697686 14512 node_manager.cc:101] [mainboard]this process will be terminated due to duplicated node[Velodyne128Segmentation], please ensure that each node has a unique name.
[lidar_perception]
[camera_perception] WARNING: The Permute Plugin is being deprecated and will be removed in the next TensorRT release. Please use the TensorRT Shuffle layer for Permute operation
[camera_perception] WARNING: The Permute Plugin is being deprecated and will be removed in the next TensorRT release. Please use the TensorRT Shuffle layer for Permute operation
[camera_perception] WARNING: The Permute Plugin is being deprecated and will be removed in the next TensorRT release. Please use the TensorRT Shuffle layer for Permute operation
[camera_perception] WARNING: The Permute Plugin is being deprecated and will be removed in the next TensorRT release. Please use the TensorRT Shuffle layer for Permute operation
[camera_perception] WARNING: The Permute Plugin is being deprecated and will be removed in the next TensorRT release. Please use the TensorRT Shuffle layer for Permute operation
[camera_perception] WARNING: The Permute Plugin is being deprecated and will be removed in the next TensorRT release. Please use the TensorRT Shuffle layer for Permute operation
[camera_perception] WARNING: The Permute Plugin is being deprecated and will be removed in the next TensorRT release. Please use the TensorRT Shuffle layer for Permute operation
[camera_perception] WARNING: The Permute Plugin is being deprecated and will be removed in the next TensorRT release. Please use the TensorRT Shuffle layer for Permute operation
[camera_perception] WARNING: The Permute Plugin is being deprecated and will be removed in the next TensorRT release. Please use the TensorRT Shuffle layer for Permute operation
[camera_perception] WARNING: The Permute Plugin is being deprecated and will be removed in the next TensorRT release. Please use the TensorRT Shuffle layer for Permute operation
[camera_perception] WARNING: The Permute Plugin is being deprecated and will be removed in the next TensorRT release. Please use the TensorRT Shuffle layer for Permute operation
[camera_perception] WARNING: The Permute Plugin is being deprecated and will be removed in the next TensorRT release. Please use the TensorRT Shuffle layer for Permute operation
[camera_perception] WARNING: The Permute Plugin is being deprecated and will be removed in the next TensorRT release. Please use the TensorRT Shuffle layer for Permute operation

Some times the car starts driving autonomously, sometimes it stays still

[EricBoiseLGSVL]

@FitashUlHaq 2080 is fine with 6.0. Perception is bugged on apollo side at the moment. We are working to find out what the issue is.

[mengen100]

@FitashUlHaq 2080 is fine with 6.0. Perception is bugged on apollo side at the moment. We are working to find out what the issue is.

Any update on this? I'm also facing the same error. Can't make the vehicle move despite I've enabled all modules and also have seen traffic in cyber_monitor.

[EricBoiseLGSVL]

@mengen100 please follow here for updates ApolloAuto/apollo#12916

[marcusvinicius178]

Hi @EricBoiseLGSVL

I am facing the same issue with Apollo 6.0 . All modules were triggered, however the vehicle does not move. From cyber_monitor I have noticed the Perception Module is not working.
I would like to know if the modules in Apollo 5.0 are working?
I recently tried Apollo 5.0 with the SVL simulator in BorregasAve. However the car was not able to localize properly and Dreamview did not spawn the BorregasMap to visualization:

I ask you guys if this is an issue related to the use of SVL Simulator, or due the use of Apollo 5.0 ? Should I use the LGSVL simulator instead the SVL? What is difference between these simulators? Why LG offers these 2 similar simulator types?

I noticed that the LGSVL does not offer the PythonAPI integration with AD Stack simultaneously as SVL, right or wrong?
I mean I have a window area in SVL to insert a Python Script and also use the autopilot bridge:

If I am not wrong in LGSVL I can just use the ONLY PYTHON API option externally the Simualtor (run python scripts but not with simultaneous use of Autopilot ROS/Cyber bridges). Or is it possible?

Are the modules (mainly Perception) working with Apollo 5.0? and are them more robust and functional than the latest Apollo 6.0? And Finally which Simulator must I use to run Apollo, the LGSVL OR THE SVL?

Please let me know. I wish to move the car autonomously.
Thanks in advance.

[mengen100]

@marcusvinicius178 I had tried to enable perception module Apollo 5.0/5.5/6.0, was only able to partially enable it in Apollo 5 with GPU GTX 1070. Before that, I was using GTX 3070 which seems not supported by Apollo.

From my experience, in order to run autonomous driving successfully, you need to meet 2 requirements:

1. Perception module is fully working.
2. Localization module is working.

If any of them is not met, then the vehicle will not move.

If your perception module is also not fully working for some reason like me, there is a way to bypass it, which is provided by LGSVL, called Modular testing. It provides ground truth data of what the perception module outputs, as if the perception module is running. By this way, you do not need to enable perception module, but the perception module is running.

For the localization module, since there are 2 methods provided by Apollo, namely RTK and MSF, I'm not sure which one you're using but by default it's RTK. MSF is trickier and I never got it successfully running (Could be due to GPU compatibility too). But for your case, as I noticed in your screenshot, on the top right corner your map selected is "San Francisco" but in LGSVL you're running "Borregas Ave", probably that's why in dreamview can't localize.

And For LGSVL and LG SVL simulators I thought they are the same thing, aren't they? HAHA I'm not entirely sure..

[lemketron]

Hi @EricBoiseLGSVL

I am facing the same issue with Apollo 6.0 . All modules were triggered, however the vehicle does not move. From cyber_monitor I have noticed the Perception Module is not working.
I would like to know if the modules in Apollo 5.0 are working?
I recently tried Apollo 5.0 with the SVL simulator in BorregasAve. However the car was not able to localize properly and Dreamview did not spawn the BorregasMap to visualization:

In your screen shot it appears that you did not set the Dreamview vehicle, setup, or map menus as described in the docs (and included screen shot) though your screen shot appears to be showing Apollo 6.0 rather than 5.0:
https://www.svlsimulator.com/docs/system-under-test/apollo-master-instructions/#launching-apollo-alongide-the-simulator

Specifically this:

Select the Mkz Lgsvl setup mode, from the menu to the left of the vehicle menu.
Select the Lincoln2017MKZ LGSVL vehicle and Borregas Ave map in the top right corner.
Open the Module Controller tap (on the left bar).

I ask you guys if this is an issue related to the use of SVL Simulator, or due the use of Apollo 5.0 ? Should I use the LGSVL simulator instead the SVL? What is difference between these simulators? Why LG offers these 2 similar simulator types?

"LGSVL Simulator" was the name of the simulator through last summer's release version 2020.06. The simulator was renamed as "SVL Simulator" with the 2021.1 release earlier this year.

I noticed that the LGSVL does not offer the PythonAPI integration with AD Stack simultaneously as SVL, right or wrong?

Not exactly. The Apollo stack integration comes through the dreamview library that was added to PythonAPI. It wasn't included in 2020.06 but you can find it in the preview2 tag of the Python API which is compatible with the 2020.06 LGSVL Simulator:
https://github.com/lgsvl/PythonAPI/commits/preview2

I mean I have a window area in SVL to insert a Python Script and also use the autopilot bridge:
...
If I am not wrong in LGSVL I can just use the ONLY PYTHON API option externally the Simualtor (run python scripts but not with simultaneous use of Autopilot ROS/Cyber bridges). Or is it possible?

LGSVL Simulator 2020.06 works with Python API and can even control Apollo 5.0 using the dreamview library with the preview2 tag as noted above.

Are the modules (mainly Perception) working with Apollo 5.0? and are them more robust and functional than the latest Apollo 6.0? And Finally which Simulator must I use to run Apollo, the LGSVL OR THE SVL?

Perception should work fine in Apollo 5.0 (but may not work on the same GPU/CPU as the simulator as perception and simulator both use a lot of GPU memory. Check nvidia-smi to make sure the GPU is not out of memory and that mainboard processes are not crashing.

Apollo 6.0 perception hasn't worked for a while which is why we documented Modular Testing to allow the simulator to publish ground truth data using the same topic where Apollo looks for output from the perception module.

[lemketron]

@marcusvinicius178 I had tried to enable perception module Apollo 5.0/5.5/6.0, was only able to partially enable it in Apollo 5 with GPU GTX 1070. Before that, I was using GTX 3070 which seems not supported by Apollo.

From my experience, in order to run autonomous driving successfully, you need to meet 2 requirements:

1. Perception module is fully working.
2. Localization module is working.

If any of them is not met, then the vehicle will not move.

If your perception module is also not fully working for some reason like me, there is a way to bypass it, which is provided by LGSVL, called Modular testing. It provides ground truth data of what the perception module outputs, as if the perception module is running. By this way, you do not need to enable perception module, but the perception module is running.

Technically the perception module is not running but the answers are being provided by the ground truth data from the simulator.

For the localization module, since there are 2 methods provided by Apollo, namely RTK and MSF, I'm not sure which one you're using but by default it's RTK. MSF is trickier and I never got it successfully running (Could be due to GPU compatibility too). But for your case, as I noticed in your screenshot, on the top right corner your map selected is "San Francisco" but in LGSVL you're running "Borregas Ave", probably that's why in dreamview can't localize.

Good explanation -- the default is actually MSF unless you choose Mkz Lgsvl in the setup menu in which case you get RTK which is what works with the simulator. MSF is an advanced topic that some may be able to get working whereas RTK is easier and requires only GNSS data from the simulator so it should "just work".

And For LGSVL and LG SVL simulators I thought they are the same thing, aren't they? HAHA I'm not entirely sure..

As noted above: "LGSVL Simulator" was the name of the simulator through last summer's release version 2020.06. The simulator was renamed as "SVL Simulator" with the 2021.1 release earlier this year.

[marcusvinicius178]

Hi @lemketron and @mengen100 Thanks very much for the response!

Hi @EricBoiseLGSVL
I am facing the same issue with Apollo 6.0 . All modules were triggered, however the vehicle does not move. From cyber_monitor I have noticed the Perception Module is not working.
I would like to know if the modules in Apollo 5.0 are working?
I recently tried Apollo 5.0 with the SVL simulator in BorregasAve. However the car was not able to localize properly and Dreamview did not spawn the BorregasMap to visualization:

In your screen shot it appears that you did not set the Dreamview vehicle, setup, or map menus as described in the docs (and included screen shot) though your screen shot appears to be showing Apollo 6.0 rather than 5.0:
I have just copied the PythonAPI image from google images...to show and make a comparison between LG simulators...it is not from my simulation. But you were right regarding the bridge (It was set wrong, once I was using sometimes the same bridge for Apollo6 and Apollo5 release). Now it is fixed and with the Modular perception strategy the car was able to move. However it has not performed an evasive maneuver.
https://www.svlsimulator.com/docs/system-under-test/apollo-master-instructions/#launching-apollo-alongide-the-simulator

Specifically this:

Select the Mkz Lgsvl setup mode, from the menu to the left of the vehicle menu.
Select the Lincoln2017MKZ LGSVL vehicle and Borregas Ave map in the top right corner.
Open the Module Controller tap (on the left bar).

I ask you guys if this is an issue related to the use of SVL Simulator, or due the use of Apollo 5.0 ? Should I use the LGSVL simulator instead the SVL? What is difference between these simulators? Why LG offers these 2 similar simulator types?

"LGSVL Simulator" was the name of the simulator through last summer's release version 2020.06. The simulator was renamed as "SVL Simulator" with the 2021.1 release earlier this year.

I noticed that the LGSVL does not offer the PythonAPI integration with AD Stack simultaneously as SVL, right or wrong?

Not exactly. The Apollo stack integration comes through the dreamview library that was added to PythonAPI. It wasn't included in 2020.06 but you can find it in the preview2 tag of the Python API which is compatible with the 2020.06 LGSVL Simulator:
https://github.com/lgsvl/PythonAPI/commits/preview2

I mean I have a window area in SVL to insert a Python Script and also use the autopilot bridge:
...
If I am not wrong in LGSVL I can just use the ONLY PYTHON API option externally the Simualtor (run python scripts but not with simultaneous use of Autopilot ROS/Cyber bridges). Or is it possible?

LGSVL Simulator 2020.06 works with Python API and can even control Apollo 5.0 using the dreamview library with the preview2 tag as noted above.

Are the modules (mainly Perception) working with Apollo 5.0? and are them more robust and functional than the latest Apollo 6.0? And Finally which Simulator must I use to run Apollo, the LGSVL OR THE SVL?

Perception should work fine in Apollo 5.0 (but may not work on the same GPU/CPU as the simulator as perception and simulator both use a lot of GPU memory. Check nvidia-smi to make sure the GPU is not out of memory and that mainboard processes are not crashing.

Apollo 6.0 perception hasn't worked for a while which is why we documented Modular Testing to allow the simulator to publish ground truth data using the same topic where Apollo looks for output from the perception module.

I am happy that the car was able to move. But I still miss an evasive maneuver or as in Autoware.AI is said the Motion Planning strategy. Apollo detected the static obstacle and stopped infinitely! As you can see

https://youtu.be/sQI9B8kVkW8
I was used to deal with Autoware Planners. I am new with Apollo planning module. Does it has not an "obstacle avoidance" planner? Would you have this info? I have fiound any tutorial in Apollo documentation how to trigger these obstacle avoidance planners. I just read about the global planner, that perform lane changes,etc. Please let me know, if you have some info on this regard :)

Thanks