Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rethink and refactor ODTF component of package #166

Open
sambles opened this issue Nov 28, 2024 · 0 comments
Open

Rethink and refactor ODTF component of package #166

sambles opened this issue Nov 28, 2024 · 0 comments
Assignees
Labels
question Further information is requested

Comments

@sambles
Copy link
Contributor

sambles commented Nov 28, 2024

Issue Description

(Comments from Ben)

Including validators and mapping in the package can be 'risky', currently the ODTF logic looks for mappings and validation files in the package if not specified locally.

The problem is that if you change the mapping locally, but don't have a validation file locally (which is optional) it will try to map using the local mapping file and then validate with the inbuilt validation file, but some of the fields in the validation file might not be in your mapping

it can cause run failures - if, say, the inbuilt validation file tries to sum buildings TIV, but you don't have a mapping for that, it will fail
but it also opens up the possibility that you'll be using an inbuilt transformation rather than a local one without knowing it

We should think about removing all internal/hidden mappings and validations all together (and the logic to look for paths through multiple transformations) and only use explicit mapping and validation files from the config file.

Summery

  • Cut down and simplify the ODTF logic where possible
  • Rethink if ODTF belongs in the ODS-Tools package
  • Remove the case where users can run using internal files without explicitly asking for them.
@sambles sambles added the question Further information is requested label Nov 28, 2024
@benhayes21 benhayes21 self-assigned this Jan 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
Status: Todo
Development

No branches or pull requests

3 participants