Transformation tool: BaseFloodElevation column values not converted to OED

[niamhomalleyjba]

Issue Description

The BaseFloodElevation column contains values that are not in the valid value range of the OED specification ([-999,-999], [0,1]). Potentially linked to 'Transformation tool: column headers are repeated in column values'?

Steps to Reproduce (Bugs only)

Config file:

transformations:
  loc: # Transformation name
    input_format:
      name: Cede_Location
      version: 10.0.0
    output_format:
      name: OED_Location
      version: 3.0.2
    runner:
      batch_size: 750 # Number of rows to process in a single batch
    extractor:
      type: mssql # Path to the DB connection file. Other options are 'postgres' and 'sqlite'. Defaults to 'csv', when not present.
      options:
        host: myserver, 1450
        port:
        database: MY_DB
        user: foo
        password: bar
        sql_statement: ./cede_location.sql # SQL statement to extract data
    loader:
      options:
        path: ./oed_location.csv # Path to the output file
        quoting: minimal

Command used:

ods_tools transform --config-file /path/to/config.yaml --nocheck

Output is created but the BaseFloodElevation column contains the values from the original CEDE data. They have not been converted to be within the OED range. For example the following values are present -999 370 161 31 14 404 515 6 49 336 28 53 175 16 8 167 7 1451 209 34 159 1069 326 10 66 733 4 33 9 1 27 171 1378 727 5 341 56 11 90 - out of these, it should only be -999 and 1 - the rest should be converted?

Version / Environment information

Name: ods_tools
Version: 3.2.7

Example data / logs

[carlfischerjba]

Looks like it might be the OED spec that is incorrect for BaseFloodElevation. Unclear why it would be restricted to [0,1]. Should it's valid value range instead be [-999,), similar to FloodDefenseHeight or BuildingHeight? Possibly there was some confusion due to the fact that the description mentions a probability.

[carlfischerjba]

Looks like it might be the OED spec that is incorrect for BaseFloodElevation. Unclear why it would be restricted to [0,1]. Should it's valid value range instead be [-999,), similar to FloodDefenseHeight or BuildingHeight? Possibly there was some confusion due to the fact that the description mentions a probability.

I've reported this at OasisLMF/ODS_OpenExposureData#225.

[benhayes21]

linked to this issue OasisLMF/ODS_OpenExposureData#225