Skip to content

feat(async): greenlet Submitter and PluginManager.run_async#710

Open
RonnyPfannschmidt wants to merge 8 commits into
pytest-dev:mainfrom
RonnyPfannschmidt:refactor/async-submitter
Open

feat(async): greenlet Submitter and PluginManager.run_async#710
RonnyPfannschmidt wants to merge 8 commits into
pytest-dev:mainfrom
RonnyPfannschmidt:refactor/async-submitter

Conversation

@RonnyPfannschmidt

Copy link
Copy Markdown
Member

Step 07 — persistent greenlet Submitter threaded through the callers into _multicall; await pm.run_async(...) awaits awaitable hook results (await-me-maybe passthrough outside it); new optional pluggy[async] extra.

Depends on #709 (includes it — full diff vs main).
Per-step diff for review: RonnyPfannschmidt#11

Refactoring series

Ordered re-implementation of the try-claude architecture experiments as reviewable steps (design writeups live in design/ inside the series). Each upstream PR shows the full cumulative diff against main; the linked chain PR on my fork shows only this step's changes:

Step Chain PR (per-step diff)
01 module split RonnyPfannschmidt#6
02 configuration objects RonnyPfannschmidt#5
03 markers attach config RonnyPfannschmidt#7
04 impl subclasses + CompletionHook API RonnyPfannschmidt#8
05 Protocol callers + multicall RonnyPfannschmidt#9
06 ProjectSpec RonnyPfannschmidt#10
07 async Submitter RonnyPfannschmidt#11

🤖 Generated with Claude Code

@RonnyPfannschmidt
RonnyPfannschmidt force-pushed the refactor/async-submitter branch from 6394496 to d23ca05 Compare July 24, 2026 16:38
RonnyPfannschmidt and others added 8 commits July 24, 2026 18:48
Add the design/ writeup chain (decisions + per-step docs 01-07) that
guides the stacked refactoring branches. Exclude design/ from
blacken-docs since the docs use elided pseudo-code signatures.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Move hook types, markers, callers, implementations, and multicall out of
the monolithic _hooks/_callers modules so later typed-config and
CompletionHook work can land without thrashing one huge file. Keep
_hooks and _callers as re-export shims for import compatibility.

Co-authored-by: Cursor AI <ai@cursor.sh>
Co-authored-by: Cursor Grok 4.5 <grok@cursor.com>
Markers attach HookspecConfiguration/HookimplConfiguration objects.
Registration discovers those privately; parse_hookimpl_opts and
parse_hookspec_opts remain a deprecated pytest concession that returns
legacy dicts and is only called when a subclass overrides them and no
modern configuration attribute was found.

Co-authored-by: Cursor AI <ai@cursor.sh>
Co-authored-by: Cursor Grok 4.5 <grok@cursor.com>
Complete design step 03: markers already attach configuration objects
since step 02; this finishes the step by storing the spec configuration
as HookSpec.config (try-claude naming) with a deprecated .opts alias,
reading .config in HookCaller firstresult resolution, and covering
decoration-time historic+firstresult validation and configuration
attachment with tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… setup API

Complete design step 04:

- HookImpl becomes a base class storing hookimpl_config (deprecated
  .opts alias kept) with arg binding moved to _get_call_args.
- NormalImpl / WrapperImpl subclasses validate their configuration;
  HookimplConfiguration.create_hookimpl() returns the right subclass
  (fixing the try-claude footgun of bare HookImpl for normals).
- WrapperImpl.setup_and_get_completion_hook() runs wrapper setup and
  returns a CompletionHook (runtime-checkable Protocol) that owns
  teardown, adapting old-style hookwrappers uniformly.
- Registration and call_extra construct impls via create_hookimpl;
  multicall binds args via _get_call_args. Full dual-sequence multicall
  rewiring lands with design step 05.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…multicall

Complete design step 05:

- HookCaller is now a @runtime_checkable Protocol; concrete callers are
  NormalHookCaller (split list[NormalImpl] / list[WrapperImpl] storage),
  HistoricHookCaller (memorize/replay, rejects wrappers) and
  SubsetHookCaller (read-only filtered proxy). _HookCaller and
  _SubsetHookCaller remain as compat aliases.
- _multicall takes dual sequences and orchestrates phases only: wrapper
  setup collects CompletionHooks, normals run, completion hooks run LIFO
  and may replace (result, exception) - no wrapper flag branching.
- add_hookspecs hands a NormalHookCaller over to a HistoricHookCaller
  when a historic spec arrives after impl registration.
- PluginManager._hookexec and tracing use the dual-sequence signature;
  monitoring callbacks keep receiving one combined impl list.
- HookSpec.verify_all_args_are_provided replaces the caller-side helper;
  set_specification accepts a config object or legacy mapping (shim).
- New tests: protocol isinstance for all concretes, historic handover,
  historic direct-call/call_extra rejection.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Complete design step 06: ProjectSpec bundles hookspec/hookimpl markers
and plugin manager creation under a single project name, with
get_hookspec_config / get_hookimpl_config helpers. Markers and
PluginManager accept str | ProjectSpec (strings still work; markers and
manager now expose project_name as a property delegating to the spec).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Complete design step 07:

- New _async module with a persistent Submitter: maybe_submit awaits
  awaitable hook results while active and passes them through otherwise
  (await-me-maybe); require_await hard-fails outside async context;
  async_generator_to_sync helps wrappers consume async generators.
- Submitter.run uses a sentinel so legitimate None returns work (fixes
  the try-claude footgun) and forwards await failures to the submission
  site inside the worker greenlet.
- PluginManager owns the Submitter and threads it through the callers
  and _hookexec into _multicall - activation is purely Submitter.run,
  no _inner_hookexec monkeypatching. await pm.run_async(func) is the
  public entry point; nested runs raise.
- Packaging: new pluggy[async] extra depending on greenlet; greenlet
  added to the testing group and types-greenlet to the mypy hook.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@RonnyPfannschmidt
RonnyPfannschmidt force-pushed the refactor/async-submitter branch from d23ca05 to 3910791 Compare July 24, 2026 16:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant