Add CAG validation to synthesizer.validate #2480

R-Palazzo · 2025-04-24T15:32:41Z

CU-86b4pmjph
Resolve #2470

sdv-team · 2025-04-24T15:32:45Z

Task linked: CU-86b4pmjph SDV - Add CAG validation to synthesizer.validate #2470

codecov · 2025-04-24T15:34:22Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 83.77%. Comparing base (e7fed1c) to head (5b86f37).
Report is 3 commits behind head on feature/single-table-CAG.

❗ There is a different number of reports uploaded between BASE (e7fed1c) and HEAD (5b86f37). Click for more details.

HEAD has 1 upload less than BASE

Flag BASE (e7fed1c) HEAD (5b86f37)

unit 1 0

Additional details and impacted files

@@                      Coverage Diff                      @@
##           feature/single-table-CAG    #2480       +/-   ##
=============================================================
- Coverage                     98.55%   83.77%   -14.78%     
=============================================================
  Files                            68       68               
  Lines                          7045     7051        +6     
=============================================================
- Hits                           6943     5907     -1036     
- Misses                          102     1144     +1042

Flag	Coverage Δ
integration	`83.77% <100.00%> (+0.32%)`	⬆️
unit	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

sdv/cag/base.py

gsheni · 2025-05-08T17:20:16Z

sdv/single_table/base.py

@@ -775,31 +768,51 @@ def _transform_helper(self, data):

        return data

-    def preprocess(self, data):
-        """Transform the raw data to numerical space.
+    def _validate_cags(self, data):


Should this function just call the user-facing validate_cag function (made for synthetic data)?

SDV/sdv/single_table/base.py

Line 719 in 85b506c

def validate_cag(self, synthetic_data):

On second thought, these two functions seem slightly different.

sdv/multi_table/base.py

sdv/single_table/base.py

amontanez24

Looks great! Just one question

amontanez24 · 2025-05-13T17:20:40Z

sdv/multi_table/base.py

@@ -133,6 +134,7 @@ def __init__(self, metadata, locales=['en_US'], synthesizer_kwargs=None):

        self._initialize_models()
        self._fitted = False
+        self._constraints_fitted = False


this parameter is not used at all for the sampling process right? I'm wondering if we need to worry about the backwards compatibility of this

That's a good point. It's not currently used for sampling and I checked that all the tests are passing on enterprise when sdv points to this branch

pvk-developer · 2025-05-13T20:26:28Z

sdv/multi_table/base.py

@@ -233,27 +228,29 @@ def get_metadata(self, version='original'):

        return Metadata.load_from_dict(self.metadata.to_dict())

-    def _transform_helper(self, data):
+    def _validate_transform_constraints(self, data):


If re-fitting a CAG is a possibility I would like to suggest a force_fit argument; Then we would call it always on the validate but not when we preprocess.

Here is why:
If I run synthesizer.fit(data) then I would like to 're-fit' because I realized I fitted with my data but not real_data, the constraints would have already been fitted with the data and won't be re-fitted ever.

That's a very good point, thanks! Done in 5b86f37

R-Palazzo requested review from amontanez24 and frances-h April 24, 2025 15:32

R-Palazzo self-assigned this Apr 24, 2025

Base automatically changed from issue-24XX-move-multi-table-logic to feature/single-table-CAG April 28, 2025 18:41

frances-h force-pushed the issue-2470-validation branch from 202aa7b to d955a9e Compare April 30, 2025 13:40

pvk-developer force-pushed the feature/single-table-CAG branch from 12ec5bc to c2c3060 Compare April 30, 2025 17:42

R-Palazzo force-pushed the feature/single-table-CAG branch from c2c3060 to 258e10a Compare May 4, 2025 08:48

R-Palazzo force-pushed the issue-2470-validation branch 2 times, most recently from b62ec68 to 14d9936 Compare May 5, 2025 16:41

R-Palazzo changed the base branch from feature/single-table-CAG to issue-2484-add-version-parameter-to-single-table-synthesizer-get-metadata May 5, 2025 16:42

R-Palazzo changed the base branch from issue-2484-add-version-parameter-to-single-table-synthesizer-get-metadata to feature/single-table-CAG May 5, 2025 16:43

R-Palazzo changed the base branch from feature/single-table-CAG to issue-2484-add-version-parameter-to-single-table-synthesizer-get-metadata May 5, 2025 16:44

R-Palazzo changed the base branch from issue-2484-add-version-parameter-to-single-table-synthesizer-get-metadata to issue-2484-add-version-parameter-to-single-table May 5, 2025 16:44

Base automatically changed from issue-2484-add-version-parameter-to-single-table to feature/single-table-CAG May 6, 2025 19:17

R-Palazzo force-pushed the issue-2470-validation branch from 14d9936 to 6e7813e Compare May 7, 2025 11:02

R-Palazzo marked this pull request as ready for review May 7, 2025 11:57

R-Palazzo requested a review from a team as a code owner May 7, 2025 11:57

R-Palazzo force-pushed the issue-2470-validation branch from 74f045e to 85b506c Compare May 8, 2025 16:31

R-Palazzo commented May 8, 2025

View reviewed changes

sdv/cag/base.py Outdated Show resolved Hide resolved

gsheni reviewed May 8, 2025

View reviewed changes

amontanez24 reviewed May 9, 2025

View reviewed changes

R-Palazzo requested review from amontanez24 and gsheni May 12, 2025 12:13

frances-h force-pushed the feature/single-table-CAG branch from 9eba42f to 7347cc6 Compare May 12, 2025 15:16

R-Palazzo force-pushed the issue-2470-validation branch from 157d027 to 61cb10e Compare May 12, 2025 16:31

amontanez24 reviewed May 12, 2025

View reviewed changes

sdv/single_table/base.py Outdated Show resolved Hide resolved

R-Palazzo added 2 commits May 13, 2025 13:59

refactor _validate_primary_key

a181b11

def single table

d7a83ec

R-Palazzo added 11 commits May 13, 2025 13:59

tests

4e94ab1

def + test multi-table

01f2367

change 1 04-05-2025

0e441b5

cleaning

ce5de77

fit cag + transform data

bd25e75

transform data multi-table

99d6b4e

fix tests

08cd14d

remove hasattr or getattr for original metadata

33cd54b

_validate_cags -> _validate_cag

1d62613

move out _convert_original_cols from _preprocess_helper

8d7994f

define _validate_transform_constraints

c5b3e6f

R-Palazzo force-pushed the issue-2470-validation branch from 61cb10e to c5b3e6f Compare May 13, 2025 12:59

R-Palazzo requested review from amontanez24 and pvk-developer May 13, 2025 13:03

amontanez24 reviewed May 13, 2025

View reviewed changes

R-Palazzo requested a review from amontanez24 May 13, 2025 18:15

amontanez24 approved these changes May 13, 2025

View reviewed changes

frances-h mentioned this pull request May 13, 2025

ValueError if conditionally sampling on a column dropped by constraints #2521

Merged

pvk-developer approved these changes May 13, 2025

View reviewed changes

frances-h approved these changes May 13, 2025

View reviewed changes

add enforce_constraint_fitting parameter

5b86f37

R-Palazzo requested a review from pvk-developer May 14, 2025 09:57

pvk-developer approved these changes May 14, 2025

View reviewed changes

R-Palazzo merged commit 0a5f98e into feature/single-table-CAG May 14, 2025
45 checks passed

R-Palazzo deleted the issue-2470-validation branch May 14, 2025 11:07

R-Palazzo added a commit that referenced this pull request May 19, 2025

Add CAG validation to synthesizer.validate (#2480)

002c39e

R-Palazzo added a commit that referenced this pull request May 26, 2025

Add CAG validation to synthesizer.validate (#2480)

b4699b2

R-Palazzo added a commit that referenced this pull request May 30, 2025

Add CAG validation to synthesizer.validate (#2480)

949a122

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add CAG validation to synthesizer.validate #2480

Add CAG validation to synthesizer.validate #2480

Uh oh!

R-Palazzo commented Apr 24, 2025

Uh oh!

sdv-team commented Apr 24, 2025

Uh oh!

codecov bot commented Apr 24, 2025 •

edited

Loading

Uh oh!

Uh oh!

gsheni May 8, 2025

Uh oh!

gsheni May 8, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

amontanez24 left a comment

Uh oh!

amontanez24 May 13, 2025

Uh oh!

R-Palazzo May 13, 2025

Uh oh!

pvk-developer May 13, 2025

Uh oh!

R-Palazzo May 14, 2025

Uh oh!

Uh oh!

Uh oh!

Add CAG validation to synthesizer.validate #2480

Add CAG validation to synthesizer.validate #2480

Uh oh!

Conversation

R-Palazzo commented Apr 24, 2025

Uh oh!

sdv-team commented Apr 24, 2025

Uh oh!

codecov bot commented Apr 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

gsheni May 8, 2025

Choose a reason for hiding this comment

Uh oh!

gsheni May 8, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

amontanez24 left a comment

Choose a reason for hiding this comment

Uh oh!

amontanez24 May 13, 2025

Choose a reason for hiding this comment

Uh oh!

R-Palazzo May 13, 2025

Choose a reason for hiding this comment

Uh oh!

pvk-developer May 13, 2025

Choose a reason for hiding this comment

Uh oh!

R-Palazzo May 14, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

codecov bot commented Apr 24, 2025 •

edited

Loading