Repository navigation
Conversation
hoshimura
commented
Oct 6, 2026
hoshimura
commented
Collaborator
- Partition walk entries in a single pass
- Keep SSH mkdir(exist_ok=True) idempotent on generic server errors
- Fix HTTPPath generic typing: phantom TypeVar -> Self
- Concurrent recursive rmdir for cloud backends
- Bounded-concurrent recursive rmdir for SSH + ENOTEMPTY mapping
- refactor cloud iterdir
- Simplify ssh error mapping and give rmdir a clean ENOTEMPTY
- Align rmdir(recursive=) across the public API
- Collapse UnsupportedProtocolPath I/O methods into getattr
- Type UnsupportedProtocolPath without return noise
- Ship asanypath_native stub in the native wheel
hoshimura
added this pull request to stack #26
October 6, 2026 14:21
hoshimura
force-pushed
the
fix_inconsistencies
branch
from
October 7, 2026 09:41
cc265fc to
969f072
Compare
- move the stub to rust/asanypath_native.pyi so maturin packages it + a generated py.typed into the asanypath-native wheel (PEP 561 for consumers) - add the missing http_scrape_links/http_request entries used by http.py - point mypy_path at rust/ so dev type-checking still resolves it
- type _raise_unsupported (and the bare-raise classmethods) as NoReturn so every placeholder method type-checks with no return statements - annotate the remaining untyped methods; fix PathLike[str | bytes] -> PathLike[str] | PathLike[bytes]; make protocol a writeable instance attr - unsupported.py: 44 mypy errors -> 0
- replace ~40 identical _raise_unsupported methods with one __getattr__ fallback; underscore/dunder names stay AttributeError so pickle + getattr(obj,name,default) capability probes keep working (__fspath__/cwd/home kept explicit) - harden has_async_transfer_api/has_write_api against UnsupportedProtocolError - fix pre-existing no-any-return in is_remote_destination
The abstract AsAnyPath.rmdir + its stub + HTTPPath.rmdir still had the no-arg signature while every writable backend (local/cloud/ftp/ssh) already implements rmdir(*, recursive=False). Advertise the param on the base + stub, and have HTTPPath accept it (still NotImplementedError — HTTP has no dir delete).
- _map_native_error takes an ignore= set (overloaded: NoReturn without it, None with it) so unlink/mkdir express missing_ok/exist_ok without re-catching the exception they just raised; restores uniform 'raise ... from exc' chaining - rmdir: non-recursive path now lazily checks emptiness via iterdir and raises OSError(ENOTEMPTY) like the cloud/pathlib backends, instead of surfacing the raw server error (which _map_native_error can't classify) - iterdir: flatten the nested try/except using ignore=
- recursive rmdir now deletes children via bounded asyncio.gather (Semaphore=16) instead of a serial await loop; measured ~2x faster on the loopback emulator and scales with RTT on real links. Deadlock-free: the semaphore is only held around a single network op, never across recursion. - deletes pipeline safely over the pooled session (concurrent writes do not, per measurement — so only deletes are fanned out) - _map_native_error: add 'not empty' -> OSError(ENOTEMPTY) so a non-empty rmdir surfaces the stdlib errno instead of the raw server failure
- cloud.py rmdir(recursive) fans out deletes concurrently; MicroBatcher coalesces them into batch API calls (no throttling semaphore, which would cap batch size) - share _rmtree() as the recursion contract; ssh keeps its bounded variant in _rmtree_bounded()
- T (TypeVar used only in return positions) couldn't be resolved, producing 7 mypy errors on iterdir/walk/glob/rename - switch to Self (matches cloud/ftp/artifactory); disable attr-defined for the http module like the sibling cloud backends to absorb the py3.10 typing.Self import - mypy 83 -> 76
- atmoz/sftp returns an unclassifiable 'Failure' for an existing dir, which _map_native_error can't map to FileExistsError; re-check is_dir() on the failure path so exist_ok stays idempotent
- fold the two zip comprehensions into one loop; keeps the single batched/concurrent is_dir resolution (per-entry await would serialize round-trips)
hoshimura
force-pushed
the
fix_inconsistencies
branch
from
October 7, 2026 10:04
969f072 to
2d7e104
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.