Skip to content

fix(api): disable uvloop if PYTHONASYNCIODEBUG=1 to prevent segfault …#61281

Merged
potiuk merged 10 commits intoapache:mainfrom
subhash-0000:fix/uvloop-asyncio-debug-segfault
Feb 15, 2026
Merged

fix(api): disable uvloop if PYTHONASYNCIODEBUG=1 to prevent segfault …#61281
potiuk merged 10 commits intoapache:mainfrom
subhash-0000:fix/uvloop-asyncio-debug-segfault

Conversation

@subhash-0000
Copy link
Copy Markdown
Contributor

This PR fixes a segmentation fault in the Airflow API server that occurs when PYTHONASYNCIODEBUG=1 is set on Python 3.13+.

Problem: uvloop is incompatible with asyncio debug mode on Python 3.13, causing the API server to crash on the first request.

Solution: Added a defensive check in the API server entrypoint (main.py) to detect PYTHONASYNCIODEBUG=1. If debug mode is enabled, uvloop is not installed, and a RuntimeWarning is emitted. Otherwise, uvloop is installed as usual for performance.

Documentation: Added a newsfragment 61214.significant.rst to note this change in the release notes.

This ensures the API server is stable under debug mode while retaining performance when debug mode is off.

Closes: #61214

@subhash-0000
Copy link
Copy Markdown
Contributor Author

Hi @ephraimbuddy @pierrejeambrun @rawwar
this PR fixes a segfault (#61214) when PYTHONASYNCIODEBUG=1 is set. All checks are passing. Would appreciate a review when you have time. Thanks

Copy link
Copy Markdown
Member

@jason810496 jason810496 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on the context in #61214 and #53105, I don't think we need to make any code change. Adding note or warning on the user-facing doc should be sufficient enough, as it will only occur when PYTHONASYNCIODEBUG=1.

@subhash-0000
Copy link
Copy Markdown
Contributor Author

Thanks for the review @jason810496

I hear you on the concern regarding code changes for an edge case. However, I’d like to propose a compromise of Code + Docs for a few key reasons:

Solving the mystery of #53105: That issue was closed as 'Not Planned' primarily because the root cause was unknown and it was thought to be environment-specific. This PR proves it’s a core Airflow/uvloop incompatibility on Python 3.13+. By fixing this in code, we officially resolve that long-standing mystery.

Segfaults are "silent" killers: Since PYTHONASYNCIODEBUG is a standard Python flag (not Airflow-specific), developers often enable it without checking application-specific docs. If it leads to a hard segmentation fault, there is no log output to tell them why it crashed, leading to more issues being opened here.

Minimal Overhead: The fix is a simple defensive check at the entry point. It ensures that when a user asks for 'Debug Mode,' Airflow prioritizes stability (standard loop) over performance (uvloop).

How about this: I will keep the defensive code fix to prevent the crash, but I will also add a note to the user-facing documentation as you suggested, explaining the fallback. This provides 'defense-in-depth'—the code prevents the crash, and the docs explain why.

Would you be open to this approach? I'm happy to push the doc updates to this PR immediately.

@subhash-0000
Copy link
Copy Markdown
Contributor Author

Hi @jason810496, just checking in on the proposal above.
If the 'Code + Docs' approach sounds good to you, let me know and I'll push the documentation updates today. Thanks!

Copy link
Copy Markdown
Member

@pierrejeambrun pierrejeambrun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've asked a couple questions, there are things I don't understand, maybe i'm missing some context. Thanks

Comment thread airflow-core/newsfragments/61214.significant.rst Outdated
Comment thread airflow-core/src/airflow/api_fastapi/main.py Outdated
Copy link
Copy Markdown
Member

@pierrejeambrun pierrejeambrun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've asked a couple questions, there are things I don't understand, maybe i'm missing some context. Thanks

Comment thread airflow-core/src/airflow/api_fastapi/main.py Outdated
@TomTomoki
Copy link
Copy Markdown

TomTomoki commented Feb 11, 2026

Just a FYI; I'm using Airflow 3.1.6 with python 3.12.12 and still facing the same segfault issue with the api-server. In my case, I had PYTHONDEVMODE enabled, which essentially sets PYTHONASYNCIODEBUG=1 as per https://docs.python.org/3/library/devmode.html#effects-of-the-python-development-mode.

This PR doesn't have any milestone, but will this be deployed in the next patch version 3.1.8?

@jason810496
Copy link
Copy Markdown
Member

jason810496 commented Feb 11, 2026

This PR doesn't have any milestone, but will this be deployed in the next patch version 3.1.8?

Yes, but as this #61281 (comment) comment mentioned, it will just add warning logs and add documentation for user when setting PYTHONASYNCIODEBUG on for api-server.

I think in most of the case, we only need to set PYTHONASYNCIODEBUG on for Triggerer to ensure the triggers are well written.

Copy link
Copy Markdown
Member

@pierrejeambrun pierrejeambrun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested this locally and it appears that this doesn't fix the problem. I still get a segmentation falt.

Maybe just a warning and it's up to the person to remove the "PYTHONASYNCIODEBUG" until python 3.13+ becomes compatible,

Comment thread airflow-core/src/airflow/api_fastapi/main.py Outdated
subhash-0000 and others added 3 commits February 12, 2026 12:27
@subhash-0000
Copy link
Copy Markdown
Contributor Author

Thanks @pierrejeambrun for testing locally and confirming that the asyncio.set_event_loop_policy() approach doesn't prevent the segfault. And thanks @TomTomoki for identifying that this also affects Python 3.12 with PYTHONDEVMODE.

I've updated the PR to take a documentation-only approach:

  • Added "Known Issues" section to web-stack.rst warning about the incompatibility
  • Updated scope to Python 3.12+ and both PYTHONASYNCIODEBUG and PYTHONDEVMODE
  • Updated newsfragment to reflect the limitation
  • Fixed RST formatting and spellcheck errors

The documentation now clearly warns users about this known limitation and suggests alternatives for debugging. Ready for review

Copy link
Copy Markdown
Member

@jason810496 jason810496 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update.

Comment thread airflow-core/src/airflow/api_fastapi/main.py Outdated
Comment thread airflow-core/newsfragments/61214.significant.rst Outdated
Copy link
Copy Markdown
Member

@jason810496 jason810496 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are just a few final nits regarding the user-facing documentation and the ‎PYTHONDEVMODE check, then the PR will be good to go.

Comment thread airflow-core/docs/administration-and-deployment/web-stack.rst Outdated
Comment thread airflow-core/src/airflow/api_fastapi/main.py Outdated
@potiuk potiuk merged commit 004f790 into apache:main Feb 15, 2026
129 checks passed
@github-actions
Copy link
Copy Markdown
Contributor

Backport failed to create: v3-1-test. View the failure log Run details

Status Branch Result
v3-1-test Commit Link

You can attempt to backport this manually by running:

cherry_picker 004f790 v3-1-test

This should apply the commit to the v3-1-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

@subhash-0000 subhash-0000 deleted the fix/uvloop-asyncio-debug-segfault branch February 15, 2026 02:18
jason810496 added a commit to jason810496/airflow that referenced this pull request Feb 15, 2026
…nt segfault … (apache#61281)

* fix(api): disable uvloop if PYTHONASYNCIODEBUG=1 to prevent segfault (issue apache#61214)

* docs(newsfragment): add newsfragment for uvloop/asyncio debug segfault fix (apache#61214)

* fix(api): add stacklevel for debug warning and fix rst literals

* fix: remove else block and correct newsfragment per review feedback

* docs: document PYTHONASYNCIODEBUG/PYTHONDEVMODE incompatibility (Python 3.12+)

* docs: fix RST formatting and spellcheck errors

- Add blank line after warning directive to fix RST error
- Quote asyncio with backticks to fix spellcheck

* refactor: simplify to warning-only approach per reviewer feedback

* refactor: apply reviewer suggestions

---------
(cherry picked from commit 004f790)

Co-authored-by: subhash-0000 <122723782+subhash-0000@users.noreply.github.com>
Co-authored-by: Jason(Zhe-You) Liu <68415893+jason810496@users.noreply.github.com>
bugraoz93 pushed a commit that referenced this pull request Feb 15, 2026
…nt segfault … (#61281) (#61933)

* fix(api): disable uvloop if PYTHONASYNCIODEBUG=1 to prevent segfault (issue #61214)

* docs(newsfragment): add newsfragment for uvloop/asyncio debug segfault fix (#61214)

* fix(api): add stacklevel for debug warning and fix rst literals

* fix: remove else block and correct newsfragment per review feedback

* docs: document PYTHONASYNCIODEBUG/PYTHONDEVMODE incompatibility (Python 3.12+)

* docs: fix RST formatting and spellcheck errors

- Add blank line after warning directive to fix RST error
- Quote asyncio with backticks to fix spellcheck

* refactor: simplify to warning-only approach per reviewer feedback

* refactor: apply reviewer suggestions

---------
(cherry picked from commit 004f790)

Co-authored-by: subhash-0000 <122723782+subhash-0000@users.noreply.github.com>
choo121600 pushed a commit to choo121600/airflow that referenced this pull request Feb 22, 2026
apache#61281)

* fix(api): disable uvloop if PYTHONASYNCIODEBUG=1 to prevent segfault (issue apache#61214)

* docs(newsfragment): add newsfragment for uvloop/asyncio debug segfault fix (apache#61214)

* fix(api): add stacklevel for debug warning and fix rst literals

* fix: remove else block and correct newsfragment per review feedback

* docs: document PYTHONASYNCIODEBUG/PYTHONDEVMODE incompatibility (Python 3.12+)

* docs: fix RST formatting and spellcheck errors

- Add blank line after warning directive to fix RST error
- Quote asyncio with backticks to fix spellcheck

* refactor: simplify to warning-only approach per reviewer feedback

* refactor: apply reviewer suggestions

---------

Co-authored-by: Jason(Zhe-You) Liu <68415893+jason810496@users.noreply.github.com>
Subham-KRLX pushed a commit to Subham-KRLX/airflow that referenced this pull request Mar 4, 2026
apache#61281)

* fix(api): disable uvloop if PYTHONASYNCIODEBUG=1 to prevent segfault (issue apache#61214)

* docs(newsfragment): add newsfragment for uvloop/asyncio debug segfault fix (apache#61214)

* fix(api): add stacklevel for debug warning and fix rst literals

* fix: remove else block and correct newsfragment per review feedback

* docs: document PYTHONASYNCIODEBUG/PYTHONDEVMODE incompatibility (Python 3.12+)

* docs: fix RST formatting and spellcheck errors

- Add blank line after warning directive to fix RST error
- Quote asyncio with backticks to fix spellcheck

* refactor: simplify to warning-only approach per reviewer feedback

* refactor: apply reviewer suggestions

---------

Co-authored-by: Jason(Zhe-You) Liu <68415893+jason810496@users.noreply.github.com>
vatsrahul1001 pushed a commit that referenced this pull request Mar 4, 2026
…nt segfault … (#61281) (#61933)

* fix(api): disable uvloop if PYTHONASYNCIODEBUG=1 to prevent segfault (issue #61214)

* docs(newsfragment): add newsfragment for uvloop/asyncio debug segfault fix (#61214)

* fix(api): add stacklevel for debug warning and fix rst literals

* fix: remove else block and correct newsfragment per review feedback

* docs: document PYTHONASYNCIODEBUG/PYTHONDEVMODE incompatibility (Python 3.12+)

* docs: fix RST formatting and spellcheck errors

- Add blank line after warning directive to fix RST error
- Quote asyncio with backticks to fix spellcheck

* refactor: simplify to warning-only approach per reviewer feedback

* refactor: apply reviewer suggestions

---------
(cherry picked from commit 004f790)

Co-authored-by: subhash-0000 <122723782+subhash-0000@users.noreply.github.com>
dominikhei pushed a commit to dominikhei/airflow that referenced this pull request Mar 11, 2026
apache#61281)

* fix(api): disable uvloop if PYTHONASYNCIODEBUG=1 to prevent segfault (issue apache#61214)

* docs(newsfragment): add newsfragment for uvloop/asyncio debug segfault fix (apache#61214)

* fix(api): add stacklevel for debug warning and fix rst literals

* fix: remove else block and correct newsfragment per review feedback

* docs: document PYTHONASYNCIODEBUG/PYTHONDEVMODE incompatibility (Python 3.12+)

* docs: fix RST formatting and spellcheck errors

- Add blank line after warning directive to fix RST error
- Quote asyncio with backticks to fix spellcheck

* refactor: simplify to warning-only approach per reviewer feedback

* refactor: apply reviewer suggestions

---------

Co-authored-by: Jason(Zhe-You) Liu <68415893+jason810496@users.noreply.github.com>
Ankurdeewan pushed a commit to Ankurdeewan/airflow that referenced this pull request Mar 15, 2026
apache#61281)

* fix(api): disable uvloop if PYTHONASYNCIODEBUG=1 to prevent segfault (issue apache#61214)

* docs(newsfragment): add newsfragment for uvloop/asyncio debug segfault fix (apache#61214)

* fix(api): add stacklevel for debug warning and fix rst literals

* fix: remove else block and correct newsfragment per review feedback

* docs: document PYTHONASYNCIODEBUG/PYTHONDEVMODE incompatibility (Python 3.12+)

* docs: fix RST formatting and spellcheck errors

- Add blank line after warning directive to fix RST error
- Quote asyncio with backticks to fix spellcheck

* refactor: simplify to warning-only approach per reviewer feedback

* refactor: apply reviewer suggestions

---------

Co-authored-by: Jason(Zhe-You) Liu <68415893+jason810496@users.noreply.github.com>
radhwene pushed a commit to radhwene/airflow that referenced this pull request Mar 21, 2026
apache#61281)

* fix(api): disable uvloop if PYTHONASYNCIODEBUG=1 to prevent segfault (issue apache#61214)

* docs(newsfragment): add newsfragment for uvloop/asyncio debug segfault fix (apache#61214)

* fix(api): add stacklevel for debug warning and fix rst literals

* fix: remove else block and correct newsfragment per review feedback

* docs: document PYTHONASYNCIODEBUG/PYTHONDEVMODE incompatibility (Python 3.12+)

* docs: fix RST formatting and spellcheck errors

- Add blank line after warning directive to fix RST error
- Quote asyncio with backticks to fix spellcheck

* refactor: simplify to warning-only approach per reviewer feedback

* refactor: apply reviewer suggestions

---------

Co-authored-by: Jason(Zhe-You) Liu <68415893+jason810496@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:API Airflow's REST/HTTP API

Projects

None yet

Development

Successfully merging this pull request may close these issues.

API server segfault when PYTHONASYNCIODEBUG=1

5 participants