How to use PythonAPI to make ego vehicle follow waypoints ?

[joeking11829]

Hi LGSVL Teams,

I would like to automatically record some scenario from simulator via PythonAPI.
So I need to make ego vehicle follow waypoints which creates by VSE.

Any ideas to make it works !?
Thanks.

[EricBoiseLGSVL]

This is not currently supported. We are working on a ego class to follow random waypoints currently. Once this releases, maybe we can add this feature next.

Doing this currently is not supported. Here are the needed changes.

1. new dynamics model similar to waypoint following class for npcs (not being worked on)
2. extend VSE to output ego path (working on this now internally)
3. custom runner support (working on this now internally)
4. new dynamics model would need passed waypoints from API (not being worked on)

Why can't you do this with a NPC vehicle that is supported? You could even create custom sensors that attach to an npc that you want using spectator vehicle.

[joeking11829]

Thanks for the reply, these features will help a lot.

Why can't you do this with a NPC vehicle that is supported? You could even create custom sensors that attach to an npc that you want using spectator vehicle.

How could I attach sensors to npc vehicle and make it publish data to bridge ?
Any document for that ?
Thanks.

[EricBoiseLGSVL]

You can do anything you want really with sensor plugins. You could look for the first NPC spawned, create a gameobject with a class that detects/collects data and then pass that data back to the sensor to send over bridge. That is why we created the spectator vehicle so users can focus on NPCs or Pedestrians in a simulation. You could also setup Autoware/Apollo/LGSVLControl to drive the ego but that is more involved. Look to Unity documentation on parenting.

[joeking11829]

You can do anything you want really with sensor plugins. You could look for the first NPC spawned, create a gameobject with a class that detects/collects data and then pass that data back to the sensor to send over bridge.

But I need to add multiple sensors to the ego vehicle, and then use LGSVL webui to set position for each sensor on the ego vehicle.
I want to collect sensor's data from ego vehicle, and make this process automatically.

Thanks.

[EricBoiseLGSVL]

Why would this not work with multiple sensors? What do you mean automatically?

[joeking11829]

Why would this not work with multiple sensors? What do you mean automatically?

Sorry for the confusion.

Because I want to create lots of scenarios of driving vehicle.
But whenever I change the config of ego vehicle, I must to collect the data again.
So I want to create an automatically process of collect the data from the scenario.

I would like to use PythonAPI to create scenario including ego vehicle driving path, pedestrian moving path, npc moving path and collect the data of multiple sensors from the scenario.

Thanks.

[EricBoiseLGSVL]

No worries. It seems you will need to wait until waypoint following is created for VSE. In the meantime, why can't you use Apollo or Autoware to drive to destinations instead?

[joeking11829]

No worries. It seems you will need to wait until waypoint following is created for VSE. In the meantime, why can't you use Apollo or Autoware to drive to destinations instead?

Thank for your suggestion.
I'm not sure that Apollo or Autoware can control the driving path when drive to destinations.

I would like to collect the IVehicleInputs of VehicleController when use KeyBoardControlSensor or WheelControlSensor to drive the ego vehicle.
And then apply the IVehicleInputs to ego vehicle.

Is it a good way ?

@EricBoiseLGSVL
Any suggestion ?
Thank you.

[EricBoiseLGSVL]

You could create a sensor like the Keyboard but publish the data instead. Then subscribe to the topics that return the inputs and apply it to IVehicleInputs.

[joeking11829]

You could create a sensor like the Keyboard but publish the data instead. Then subscribe to the topics that return the inputs and apply it to IVehicleInputs.

I found that the input frequency of KeyboardControlSensor(WheelControlSensor) is controlled by FPS.
Because KeyboardControlSensor.Update() is called per frame.
And IVehicleInputs are also collected when VehicleController.Update() is triggered.

If I collect the inputs from IVehicleInputs, I will get unstable data which depends on FPS of Simulator.

It seems to me that VehicleSMI is the real controller of ego vehicle,
All IVehicleInputs will update to VehicleSMI via VehicleSMI.FixedUpdate().
Because VehicleSMI.FixedUpdate() is triggered by Fixed Timestep.
Maybe I can get stable data here.

But I'm not sure the relation between "VehicleControllder(IAgentController)"、"VehicleSMI(IVehicleDynamices)、"VehicleActions(IVehicleActions)".

Any documents to explain that ?
@EricBoiseLGSVL
Thank you.

[EricBoiseLGSVL]

Yes it is based on Update(). You can definitely use the IVehicleDynamics class.
IAgentController - Inputs, sensors,, AgentManager.cs methods
IVehicleDynamics - Dynamics of the vehicle, gear box, acceleration, etc.
IVehicleActions - Unity calls for animations, lights, wipers, etc.

[joeking11829]

@EricBoiseLGSVL
I implement the record mechanism inside VehicleSMI.GetInput() which used by VehicleSMI.FixedUpdate.

I collect SteerInput, AccellInput, HandBrake, Reverse for the driving recorder.
And then apply the record in the same way.
It did works.

But I got an issue, when I make a sharp turn at high speed. the record was incorrect.
It looks like that something effect the ego vehicle.

Any Ideas ?
Thanks.

[EricBoiseLGSVL]

This is probably issues with Unity's wheel colliders. You might have slipping or bouncing. What speed are you having these issues?

[joeking11829]

This is probably issues with Unity's wheel colliders. You might have slipping or bouncing. What speed are you having these issues?

Thanks for the reply.
Drive over 60 miles/h

But when I make a sharp turn at low speed. ( maybe 5 - 10 miles/h )
And then apply record multiple times, sometime get incorrect result.

[EricBoiseLGSVL]

Hmm, not sure but keep in mind FixedUpdate can return values multiple times per frame. Update doesn't do this so maybe the calculations need to be adjusted for this?

[joeking11829]

I think that the physics motion of ego vehicle is calculated through FixedUpdate ?
So Update will effect the calculation of physics motion ?

[EricBoiseLGSVL]

no the update cycle is once per frame. Fixed update can happen multiple times per frame, so a calc that uses previous frame data may give bad data because of logic happening multiple times in one frame instead of the expected one (commonly input in engines are captured in update). Maybe try to debug more on the frames you get the odd data.
Sorry, without seeing the data it is tough to debug what is incorrect

[joeking11829]

no the update cycle is once per frame. Fixed update can happen multiple times per frame, so a calc that uses previous frame data may give bad data because of logic happening multiple times in one frame instead of the expected one (commonly input in engines are captured in update). Maybe try to debug more on the frames you get the odd data.
Sorry, without seeing the data it is tough to debug what is incorrect

Got it.
Thanks you.

I will do some analysis for the record/apply process.