5.4. VALIDATION METHODOLOGY
157
Perception The perception module can be validated by evaluating the quality of reconstruction of the world model based on a known sequence of measurements. This is discussed in section 4.4.2.
Route planning The implemented route planning is simple enough to be evaluated simply based on human expectations over the output.
Trajectory planning Being the main decision component, this module is fairly critical. All of its decisions are based on an instantaneous estimation of a world model with predictive capabilities (i.e., it includes moving objects). It can then be tested over different predefined scenarios where the input world model is fixed. Given a partial world view it should be verified that the trajectory planner makes a satisfactory choice. The decisions based on a single partial world view are the one that define the behaviour of the robot when a sequence of measures is used to update online the world model. The portion of world that is observed is dependent of the previously planned trajectory. Previous plans affect future plans. Thus it is desirable to also verify the online behaviour of the vehicle using a simulator such as the one described in section 5.5. For this validation test the perception and control modules should be bypassed, using the simulator to provide a perfectly conservative partial world model and perfect trajectory tracking.
Control & Perception When validating the control on its own we assume that the vehicle state estimation is perfect, or respect a defined noise model. This has to be validated with the actual perception module state estimation. To do so, we execute perception and control modules online in the vehicle, but using a fixed predefined trajectory. The vehicle will move and we are able to check that the perceived control error is lower that a defined bound. Any error software during this integration phase will lead to erratic movements of the vehicles. It is then desirable to first run this test in a simulated environment (see section5.5) before engaging in the physical vehicle.
Trajectory planning & Perception In order to check that the integration of this two modules is working as expected, the software can be executed in a manual mode. The vehicle will not follow the defined trajectory but the online effectiveness of perception and trajectory planning modules can be assessed. In order to better emulate the relation between past trajectories and observed world, it is recommended to have the manual driver follow the computed trajectory (playing the role of the control module).