Fix list table-parts always reporting 0 parts - #45
Merged
Conversation
pgcopydb list table-parts was never calling schema_list_partitions, so
s_table_part was always empty and the command always printed
"COPY will be split 0-ways" for every table, regardless of size.
The fix adds the missing computation step between the partKey setup and
the display loop:
1. Check whether DATA_SECTION_TABLE_DATA_PARTS is already marked as
fetched in the SQLite catalog (i.e. a previous run computed and
stored the parts). If it is, re-use the cached rows directly – the
catalog already has everything catalog_iter_s_table_parts needs.
2. If the section is not yet fetched, open a source connection and run
the same logic the full copy path uses:
- For CTID splits: optionally ANALYZE the table first (skipped when
--estimate-table-sizes is set), then call schema_list_relpages to
get a fresh relpages value.
- Call schema_list_partitions, which writes every part to s_table_part
and sets table->partition.partCount to the actual split count.
- Call catalog_register_section to mark the section as done so the
next invocation uses the cache.
After this, log_info("Table %s COPY will be split %d-ways") prints the
correct count and catalog_iter_s_table_parts iterates the real rows.
Closes dimitri#872
(cherry picked from commit 80dddd8)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
pgcopydb list table-partsalways printed "Table … COPY will be split 0-ways" with an empty table, regardless of size or--split-tables-larger-than.Root cause
cli_list_table_partssetpartKey(including the CTID case) but never calledschema_list_partitions, so thes_table_partcatalog was empty andtable->partition.partCountwas 0. The full copy path (copydb_prepare_table_specs_hook) does call it; the list command bypassed it.Fix
Compute the partitions (
schema_list_partitions) before the display loop, so the command reports the real split.list table-partsis a diagnostic/planning command — no effect on the migration path.Clean cherry-pick of upstream
dimitri/pgcopydbdimitri#982.Tests
unitsuite (which driveslist table-parts) green in the fullPGVERSION=18 make testsrun. The only red was the pre-existingblob-snapshot-releasesnapshot-race flake, which touches no code in this change.