Claude/fix stub generation y mpbt#445
Open
Edwardvaneechoud wants to merge 2 commits intomainfrom
Open
Conversation
Four targeted improvements to make canvas-authored flows feel native when rebuilt in Python via flowfile_frame: 1. Add "virtual" to WriteMode TypeAlias and FlowFrame.write_catalog_table Literal so IDE/type-checker accepts write_mode="virtual". 2. Add save_flow_to_catalog free function and FlowFrame.save_to_catalog method to register a Python-authored flow (default landing namespace: General > Python Flows, auto-created on first use). Improves the virtual-write error message with a copy-pastable hint. 3. Translate flowfile_formulas in with_columns to a single polars-code node via polars_expr_transformer.to_polars_code when all formulas convert; fall back per-formula otherwise (all-or-nothing). 4. Plumb output_field_config through read_from_cloud_storage and the four scan_*_from_cloud_storage helpers; add FlowFrame.with_output_validation for ergonomic per-step validation; emit .with_output_validation(...) from the flowframe code-generator so canvas validation round-trips. Also bump polars-expr-transformer to >=0.5.4. https://claude.ai/code/session_01KFY9WBmRT6jNhEbUMxgeKu
Sync .pyi stubs with current source after recent API additions (save_flow_to_catalog, save_to_catalog, with_output_validation, output_field_config kwargs, virtual write mode, etc.) so 'make check_stubs' passes in CI. https://claude.ai/code/session_01D28WP4tn1HL67GNVgJGGnD
✅ Deploy Preview for flowfile-wasm canceled.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces a new utility for programmatically saving flows to the catalog, improves namespace management for Python-authored flows, and makes several code quality and documentation updates. The most significant change is the addition of the
save_flow_to_catalogfunction, which allows users to register flows from Python code, aligning with the canvas "Save to Catalog" feature. Additionally, the new "Python Flows" namespace is introduced to better organize flows created programmatically.Key changes include:
New Python Flows Namespace and Catalog Registration:
ensure_python_flows_namespacemethods to bothCatalogServiceand its flows service, enabling creation and resolution of the "General > Python Flows" namespace for programmatically authored flows. [1] [2]save_flow_to_catalogfunction inflowfile_frame/catalog.py, which saves a flow to disk and registers it in the catalog, defaulting to the "Python Flows" namespace. This includes validation for flow names and checks for namespace and file collisions.__init__.pyand__init__.pyito exposesave_flow_to_catalogand related schema types (OutputFieldConfig,OutputFieldInfo) at the top level of theflowfile_framepackage for easier user access. [1] [2] [3]Documentation and Error Message Improvements:
write_catalog_tableto clarify the new 'virtual' write mode and its requirements, including the need for prior catalog registration viasave_flow_to_catalog.Code Quality and Minor Refactoring:
code_generator.py. [1] [2] [3] [4] [5] [6]_emit_output_validationmethod to support output schema validation during code generation.catalog.py. [1] [2]