Scenario Generation

Summary

Sensor failure modes can be found by testing sensor software in simulation on scenarios with a variety of sensor configurations and hazardous obstacle types. Such scenarios can be easily generated using the dRISK Edge tool. In this workflow, we walk through how requirements-based scenarios can be generated in dRISK Edge.

1. Define the YAML

1.1 YAML Field Description

The scenario generation tool takes a YAML file as input which defines the paramter to be varied. The tool will then create one scenario for each combination of parameter values within a parameter group. The input YAML file needs to be structured as follows:

work_packages:
  WP1 compare scan patterns 1 and 2:
    parameter_groups:
      - scan_pattern: [scan_pattern_1, scan_pattern_2]
        mount_height: [1.3, 1.4, 1.5]
        pitch_angle: [-0.50,-0.25, 0, 0.25,0.50]
        roll_angle: [-0, -2, -3]
        horizontal_road_curvature: [0]
        vertical_road_curvature: [2000, 1000, 0, -1000, -2000]
        target_config: [highway_target_config]
target_configs:
  highway_target_config:
    small_box_obstacle_on_the_let:
      type: small_box
      lateral_offset: -6
      longitudinal_offset: 250
    child_pedestrian_on_the_right:
      type: child_pedestrian
      lateral_offset: 5.5
      longitudinal_offset: 250

1.2. Going to and from dRISK’s reference system

dRISK adheres to the ISO 8855:2011 standard and thus is aligned with OpenScenario standards. This means that when using dRISK’s tools, make sure to adhere to the right-handed coordinate system, see illustration below:

    # dRISK's right-handed coordinate system: 
    #    X (forward), Y (right), Z (up)
    #
    #     | Z (yaw/heading)
    #     |
    #     |
    #     +-----------> X (roll)
    #    /
    #   /
    #  / Y (pitch)
    # NOTE: rotation's sign is measured with right 
    # hand thumb pointing towards the axis.

If coming from a left-handed coordinate system like the one on the illustration below:

# Left-handed coordinate system:
#  X (forward), Y(left), Z (up), 
# 
#   | Z (yaw/heading)
#   |  / Y (pitch)
#   | /
#   |/
#   +-----------> X (roll)
# NOTE: rotation's sign is measured with left
# hand thumb pointing towards the axis.

This will mean that pitch_angle (rotation along the Y-axis) will be of the same sign as dRISK’s coordinate system, but roll_angle (rotation along X-axis) will not. Make sure to invert the roll_angle sign in the YAML file before generating scenarios, and invert again when going back to a left-handed coordinate system, e.g. when analyzing results on a left-handed coordinate system.

2. Scenario Generation

2.1 Upload YAML configuration to dRISK Edge view

In the dRISK Edge View search bar, open the view “Scenario Factory Workspace”.

Image title — Figure 1: Searching for a view in dRISK Edge.

To upload the YAML file, press the Upload button on the view menu then when prompted, select the YAML file from your local drive.

Once the yaml has been uploaded, the yaml node (label having the file name of the yaml) will appear in the view. This header node is a high-level node containing all data necessary to generate the scenarios, which is also stored deeper in the graph, but to generate scenarios one needs only to use the Generate scenarios command (see section 2.2, immediately below).

2.2. Generate scenarios

To generate the scenarios, right-click on the yaml node, and press Tools -> Generate scenarios.

2.3. Check Progress

Once scenario generation begins, a new view with the title beginning “scenario_factory_view_” will open automatically. The is a unique uuid identifier that we use to keep track of unique scenario generation sessions. To monitor the scenario generation, refresh this new view regularly by right clicking on the view and selecting Refresh.

Once the view is fully loaded, it will initially contain one node on the left, the scenario generation header node (we will refer to this one as header from now on), and the five scenario generation progress nodes. Initially it will have a [SCENARIO GEN. MSG] starting generation message as the node label, as shown in Figure 4 below.

2.3.1. YAML consistency checks

If a combination of sensor parameter values does not exist in the dRISK Knowledge Graph, an error message like the following will appear in the scenario header node label.

[MALFORMED YAML]

If the YAML file itself is malformed, e.g. doesn’t have the right indentation or consistent structure, scenario generation will not be able to start and the error will be displayed on the header node.

[MISSING TARGET]

If a dRISK obstacle of the given type does not exist in the knowledge graph, an error will appear like the following.

[MISSING ROAD NETWORK]

If one of the specified road curvatures does not exist in the dRISK catalogs, an error will be printed as shown in Figure 7 below.

2.3.2. Progress checks

Once scenario generation starts, the scenario nodes will be laid out vertically.

The central nodes which appear vertically distributed as successors of the work package header node are scenario nodes. The number of scenario nodes equals the number of parameter value combinations as defined in the YAML file. These scenario nodes will change color from gray to yellow to green, according to their stage of generation:

GRAY → Placeholder Scenario Node

YELLOW → Scenario is being generated

GREEN → Scenario generation has completed

RED → Scenario generation has been aborted due to error

This information is also encoded in the SG_STATUS taxonomy; each annotation will connect to the scenarios currently at that particular stage of generation. For example, if a sensor mount height for whose ego entry doesn't exist, an error will appear as such. Notice that it will be connected colored red and connected to the FAIL node.

Notice that for any error appearing in an individual scenario node, the parameter variation will be printed in JSON format, so the user can regenerate the specific scenario using the same process.

The completion of the scenario generation is indicated when all scenarios in the view are colored either red or green, and are connected to the SG_STATUS node. Finally, a new black node with label metadata_<yaml_node_id> will appear on the LHS of the scenario factory view. If all scenarios were generated successfully, they will all appear green.

During the scenario generation process, please frequently refresh the view so that the information displayed is up to date. The need for this will soon be removed with an upcoming software update

2.3. Cancel scenario generation

If at any point during the process of scenario generation you wish to cancel the process, simply draw an edge directed from the CANCEL status node to the header node. This is done by clicking right on the edge and holding to drag the edge to the header node.

3. Download Results

To download a metafile containing all relevant details about the newly generated scenarios, right click the header node with label the metadata_<yaml_node_id> header node then Node(s) > Open URL / file. You should see a file with label ending metafile_<yaml_node_id>.csv appear in your browser downloads folder.

To download the scenario OpenScenario files, right click the work package header node with label scenarios_<yaml_node_id>, then select Open Link. You will shortly find a <yaml_node_id>_scenarios.zip file containing all scenario OpenScenario (.xosc) files in your browser downloads folder.

Last update: 2023-10-25