You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Including validators and mapping in the package can be 'risky', currently the ODTF logic looks for mappings and validation files in the package if not specified locally.
The problem is that if you change the mapping locally, but don't have a validation file locally (which is optional) it will try to map using the local mapping file and then validate with the inbuilt validation file, but some of the fields in the validation file might not be in your mapping
it can cause run failures - if, say, the inbuilt validation file tries to sum buildings TIV, but you don't have a mapping for that, it will fail
but it also opens up the possibility that you'll be using an inbuilt transformation rather than a local one without knowing it
We should think about removing all internal/hidden mappings and validations all together (and the logic to look for paths through multiple transformations) and only use explicit mapping and validation files from the config file.
Summery
Cut down and simplify the ODTF logic where possible
Rethink if ODTF belongs in the ODS-Tools package
Remove the case where users can run using internal files without explicitly asking for them.
The text was updated successfully, but these errors were encountered:
Issue Description
(Comments from Ben)
Including validators and mapping in the package can be 'risky', currently the ODTF logic looks for mappings and validation files in the package if not specified locally.
The problem is that if you change the mapping locally, but don't have a validation file locally (which is optional) it will try to map using the local mapping file and then validate with the inbuilt validation file, but some of the fields in the validation file might not be in your mapping
it can cause run failures - if, say, the inbuilt validation file tries to sum buildings TIV, but you don't have a mapping for that, it will fail
but it also opens up the possibility that you'll be using an inbuilt transformation rather than a local one without knowing it
We should think about removing all internal/hidden mappings and validations all together (and the logic to look for paths through multiple transformations) and only use explicit mapping and validation files from the config file.
Summery
ODS-Tools
packageThe text was updated successfully, but these errors were encountered: