DropColumnsProcessorConfig: not idempotent on re-run and no support for reasoning columns

[mvansegbroeck]

Priority Level

Medium (Annoying but has workaround)

Describe the bug

DropColumnsProcessorConfig has two issues in notebook workflows:

1. Not idempotent: Re-running add_processor with the same name but different column_names does not update the existing processor. The stale config persists.
2. Cannot use <column_name>__reasoning_content to drop a reasoning column or things like "*__reasoning_content" to drop all reasoning columns at once. The validator rejects it because the literal string doesn't match any column name.

Steps/Code to reproduce bug

Issue 1: Re-running does not update config

config_builder.add_processor(
      dd.DropColumnsProcessorConfig(
          name="cleanup",
          column_names=["col_a"],
      )
  )
data_designer.validate(config_builder)  # OK

Now change column_names and re-run the cell:

config_builder.add_processor(
      dd.DropColumnsProcessorConfig(
          name="cleanup",
          column_names=["col_b"],  # changed
      )
 )
data_designer.validate(config_builder)  # Now drops ["col_a", "col_b"] instead of ["col_b"] only

Issue 2:

  config_builder.add_processor(
      dd.DropColumnsProcessorConfig(
          name="cleanup",
          column_names=["col_a__reasoning_content"],
      )
  )
  data_designer.validate(config_builder)  # Error: column does not exist

Expected behavior

1. Calling add_processor with the same name should replace the existing processor config (upsert), so notebook cells are safely re-runnable.
2. column_names should support dropping reasoning columns

Additional context

No response

[andreatgretel]

Hey @mvansegbroeck, both issues should be fixed in #334.

For the re-run problem — add_processor now does an upsert when the name matches, same as add_column already does. Old drop=True flags get cleaned up too.

For reasoning columns — the validator now knows about side-effect columns like __reasoning_content and __trace, so dropping those works as long as the parent column has extract_reasoning_content=True.

Also went ahead and added glob support, so things like *__reasoning_content will work to drop all reasoning columns at once. Validation will warn if a pattern matches nothing.