Skip to content

Release 2.3.7
Compare
Choose a tag to compare
@awsbuild awsbuild released this 06 Aug 12:08
2.3.7
1b8d539

Oasis Release v2.3.7

Docker Images (Platform)

Docker Images (User Interface)

Components

Changelogs

OasisPlatform Changelog - 2.3.7

  • #1090 - Fixed CVEs from fiona package (backport)
  • #1091 - Fixed build error from worker-controller image
  • #1057, #1092 - Fix/1092 task cancellation issue
  • #1095 - Updated Package Requirements: twisted==24.7.0rc1
  • #1077 - FIx missing exception trace in V1 workers
  • #1076 - Fix return of types of create 'analyses' and 'model' POST in API spec.
  • #1081 - Fix/api responses v2
  • #1085 - Validation errors when posting a run incorrectly update an anaysis state to RUN_ERROR
  • #1071 - Add new custom code hooks into the V2 workflow

OasisLMF Changelog - 2.3.7

  • #1533 - API client names the downloaded output file .tar instead of .tar.gz
  • #1530 - Short flags unexpectedly changed in 2.3.6
  • #1539 - Allow keys files with both amplification_id and model_data columns
  • #1542 - Time and Memory performance issue for RI contract
  • #1531 - fix effective deductible applied in minded calculation for calcrule 19
  • #1532 - make all compute step run command oasis logged
  • #1468 - loss output at intermediate inuring priorities - new features
  • #1535 - add interval mapping to built in function

ODS_Tools Changelog - 3.2.6

  • #128 - Fix allow blank errors with OED v4 spec
  • #105 - Correlation Settings in Model settings
  • #129 - Explicitly select mapping file to be used (ODTF)
  • #133 - pla update
  • #134 - Update test for OED4 - test_field_required_allow_blank
  • #127 - Update readme

Release Notes

OasisPlatform Notes

Fixed CVEs from fiona package (backport) - (PR #1090)

Updated Package Requirements: fiona==1.10b1 for CVE-2023-45853 & CVE-2020-14152

Fixed build error from worker-controller image - (PR #1091)

A new version of setuptools fails with packaging<22.0, added new pin to requirements file.

From issue: pypa/setuptools#4483)

Fixed task cancellation issue when analysis is queued - (PR #1093)

Analyses cancellation silently fails when there are no workers on a model queue.

Example:

  1. POST /analyses/1/run/ ~ auto-scaller starts spinning up workers but none are ready yet, tasks on queue
  2. POST /analyses/1/cancel/ ~ analyses is marked as cancelled but sub-tasks are unrevoked and still on queue
  3. POST /analyses/1/run/ ~ a new set of sub-tasks are placed on queue.
  4. Bugs #1092 and #1057 occur

Fix

  • sub-tasks celery id's are stored by the task controller when a job is created
  • when a task is cancelled these are all marked as REVOKED when a task cancellation is issued.
  • The workers task guard check_task_redelivered has been updated to 'self kill' if a task detects itself as REVOKED
  • When set to a DEBUG log level the worker-controller lists all dispatched tasks, with linked celery id's
start_input_generation_task[-id-]: 'generate_inputs' - canvas dispatched, analyses=1, run_uuid=9b03e7bf29764b798d78abb59b81906b
start_input_generation_task[-id-]: Prepare input generation params = 7c74f4eb-4090-4238-953d-c04d47f8d25d
start_input_generation_task[-id-]: Pre analysis hook = af5feb49-0b62-41c6-a805-fd84ff854c2e                                                                                                                                                                    
start_input_generation_task[-id-]: Prepare keys file 0 = 2f326943-6f6d-412c-b4fb-0d007999e12a
start_input_generation_task[-id-]: Prepare keys file 1 = e6ebc3aa-06b3-4765-a4aa-98acdf3acece
start_input_generation_task[-id-]: Prepare keys file 2 = 5843cebb-9aea-44db-89c6-e4a99ed6d738
start_input_generation_task[-id-]: Prepare keys file 3 = 8d9367ec-7a56-4aab-83ea-b143117bcdc0
start_input_generation_task[-id-]: Prepare keys file 4 = 098e7dfd-e082-446e-918b-62387cf6d150
start_input_generation_task[-id-]: Prepare keys file 5 = e92d3730-46a0-4c5f-978a-ff25183f4186
start_input_generation_task[-id-]: Prepare keys file 6 = ae236118-96a5-4bac-af6f-6a5aa6c7b3aa
start_input_generation_task[-id-]: Prepare keys file 7 = f9842a6d-1b0e-45bf-a96c-1a1f9720832a
start_input_generation_task[-id-]: Prepare keys file 8 = 996344ae-c633-4215-920c-ae45debbc070
start_input_generation_task[-id-]: Prepare keys file 9 = e30240ff-7aa0-4648-9ba0-e9f631501839
start_input_generation_task[-id-]: Collect keys = ce1289f7-9a60-4fee-b019-0f3335c7cc04
start_input_generation_task[-id-]: Write input files = e88db9e3-2d7f-4883-88da-0da7e7bb0a8d
start_input_generation_task[-id-]: Record input files = 873b2872-b51e-4ebb-8524-53f826b0b1c7
start_input_generation_task[-id-]: Cleanup input generation = 7b152ff6-da9f-4118-a514-ee3d90d996bc

Updated Package Requirements: twisted==24.7.0rc1 - (PR #1095)

FIx missing exception trace in V1 workers - (PR #1077)

Fixed logger to show full trace when a V1 worker raises an exception

Fix API spec - return from create & update for 'analyses' and 'model' - (PR #1078)

Corrected the returns from /analyses/ and /model/ so that the settings resources are settings: <URL-to-file> and not settings: <JSON>

Note: This is the same behaviour as release prior to #1067

Fix api responses v2 - (PR #1081)

More fixes to align Analysis with AnalysisList

Fixed sever setting an analyses to an error state when submitting a bad request. - (PR #1086)

Fixed issue where analyses are marked as RUN_ERROR but no reason can be found in logs. This can happen due to issue #1085. Fixed by not setting ab analyses state to RUN_ERROR on validation issues.

Added new hoosk to v2 workflow (backport) - (PR #1087)

Added custom code hooks from OasisLMF/OasisLMF#1524

  • pre-analysis (unchanged) - runs in its own sub-task before keys lookup and modifies the exposure files.
  • post-generation-hook (new) - runs directly after write-oasis-files but before the inputs.tar.gz is created.
  • pre-losses-hook (new) - runs after the runs losses directory has been initialized, but before executing an analysis.
  • post-losses-hook (unchanged) - runs after the output calc has run, but before creating the results file output.tar.gz

OasisLMF Notes

Fix missing short flags from CLI - (PR #1538)

Fix for OasisLMF/OasisLMF#1530, new custom hooks from 2.3.6 caused the CLI to lose some flags.

Fix Cartesian product issue for RI Fac contract - (PR #1541)

With the introduction of all the scope oed column in the filter process of RI, the Fac contract had to have part of it's logic move to the filter level, however contrary to the other type of contract, we only use 1 layer_id for all FAC, this created a Cartesian product during the merge between info and scope.
this PR add some logic to use the risk level column to merge on in addition to layer_id when contracts see to be exact match which is the case for FAC.

Fix issue in calcrule 19 when min deductible is triggered - (PR #1531)

When % of loss deductible is smaller than the minimum deductible, neither the minimum deductible nor the % of loss deductible gets applied. This overstates gross losses.
the deductible passed to the min deductible calculation was incorrect for calcrule 19, as we passed the percentage instead of the actual deductible, this fix the issue by passing the effective deductible for this calcrule.

add oasis log to all compute step run command - (PR #1532)

Allow to have loss output at intermediate RI inuring priorities - (PR #1534)

  • create new rl_outputs and rl_summaries parameters in analysis settings to drive all reinsurance loss perspective outputs

  • relabel the existing reinsurance loss outputs to use the code rl rather than ri, but otherwise keep the same naming convention

  • use existing ri_inuring_priorities parameter in analysis settings to drive new intermediate reinsurance net loss perspectives with output files labelled ri. This involves some genbash work to 'tee' the fmpy ri net stream to files as well as passing it down the pipe to the next RI calculation

As part of this work and to prepare for the UI controls, we are considering making changes to the run dir;

  • move ri_layers.json into the inputs folder (so that the available net perspectives may be read by the UI)

  • move RI_[X] folders from the root run dir to be nested in the inputs dir, and make the necessary changes to genbash to change the relative folder path of the RI_[X] folders in the run_ktools script.

add new built in in lookup build_interval_to_index - (PR #1535)

Allow user to map a float to an index by giving each index interval

ODS_Tools Notes

Fix allow blank errors with OED v4 spec - (PR #128)

Updated validator to work with both v3 spec and v4 (implied allowed blanks).

Model settings schema update for new correlation settings - (PR #130)

hazard_correlation_value and damage_correlation_value added to model settings schema

Update test for OED4 - test_field_required_allow_blank - (PR #134)

For example ContentsTIV is now an optional field, so won't be automatically added with its default value populated

Update readme file - (PR #127)

Added information on optional requirements installation and CLI data transformations in the readme file.

Assets 5
Loading