Skip to content

Delay selector init until DFPManager.start#63556

Open
uranusjr wants to merge 1 commit intoapache:mainfrom
astronomer:dag-processor-daemon-selector
Open

Delay selector init until DFPManager.start#63556
uranusjr wants to merge 1 commit intoapache:mainfrom
astronomer:dag-processor-daemon-selector

Conversation

@uranusjr
Copy link
Member

Fix #50038. I mostly followed #50038 (comment) but added some additional infra to make things easier to understand in the future.

I have no idea how to test this in CI.

Some selectors do not work well in daemon mode after fork (exact reason
unknown; it's CPython internal). This stub allows us to delay creating a
selector until after forking and work around the issue.

A stub class is created to stand in the selector attribute until the
real selector is initialized. This makes Mypy happy, and also provides a
slightly better error message if a contributor get things wrong
accidentally in the future.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dag processor cannot run in daemon mode

1 participant