[Feature] Table maintenance: snapshot expiration and orphan file cleanup

[zhuxiangyi]

Search before asking

• I searched in the issues and found nothing similar.

Motivation

Java Paimon expires snapshots after every commit and provides expire_snapshots and
remove_orphan_files procedures. paimon-rust has none of these, so a table written through the
Rust API, pypaimon native, or the C FFI keeps every snapshot forever. Each commit adds a snapshot,
manifest lists, and manifests, and files replaced by overwrites or row-level rewrites are never
removed. Files left behind by failed or interrupted writes are never cleaned up either.

Solution

Port Java's table maintenance in small, independently reviewable PRs, each following the Java
implementation and keeping its safety rules. (Every read failure deletes less, never more. A file
still referenced by any snapshot, tag, branch, or consumer is never deleted.)

• Snapshot expiration core + CALL sys.expire_snapshots: feat(table): expire snapshots and add sys.expire_snapshots #965
  ExpireSnapshotsImpl / SnapshotDeletion, with the snapshot.num-retained.*,
  snapshot.time-retained, and snapshot.expire.limit options, consumer and tag protection, and
  Java's procedure arguments.
• Expire snapshots after commit: feat(table): expire snapshots after commit #966
  Like Java's TableCommitImpl, run expiration after a commit that creates a snapshot. It honors
  write-only and skips when the changelog lifecycle is decoupled (changelog.* retention
  longer than snapshot retention), which needs a changelog manager first.
• Python API: Table.expire_snapshots: feat(python): expose Table.expire_snapshots #967
• Orphan file cleanup + CALL sys.remove_orphan_files: feat(table): remove orphan files and add sys.remove_orphan_files #968
  LocalOrphanFilesClean: delete files older than older_than (default one day ago) that no
  snapshot, tag, long-lived changelog, or branch references, with a dry_run mode.

Later, not in these PRs:

• a changelog manager and decoupled changelog expiration (changelog.num-retained.*,
  changelog.time-retained);
• snapshot.clean-empty-directories and empty-directory cleanup;
• partition expiration.

Anything else?

The PRs are stacked: #966, #967 and #968 each build on #965 and are independent of each other.

Willingness to contribute

• I'm willing to submit a PR!