Status of testing of Apache Airflow 3.1.8rc1 and Task SDK 1.1.8rc1

[vatsrahul1001]

Body

We are kindly requesting that contributors to Apache Airflow RC 3.1.8rc1 help test the RC.

Please let us know by commenting if the issue is addressed in the latest RC.

• [v3-1-test] Fix language selector state not updating on change (#61060) (#61263): @guan404ming
• [v3-1-test]Flatten grid structure endpoint memory consumption (#61273) (#61393): @vatsrahul1001
• [v3-1-test] fix: Fix grid view tooltip z-index issue (#61275) (#61403): @RoyLee1224
• [v3-1-test] Fix Liveness / Readiness / Startup probe path for Airflow… (#61411): @jvoravong @gichul-hong @potiuk
  Linked issues:
  • Liveness / Readiness / Startup probe path for Airflow 3.x (#58129)
• [v3-1-test] Update healthcheck command syntax for celery worker (#58861) (#61412): @bugraoz93 @potiuk
• [v3-1-test] Fix deferrable sensors not respecting soft_fail on timeout (#61132) (#61421): @nathadfield
  Linked issues:
  • Deferrable sensors fail to respect soft_fail=True on timeout (#61130)
• [v3-1-test] Deprecate BackfillDetails and use DagAcccessEntity.Run for backfill p… (#61456): @pierrejeambrun
• fix(ui): Remove explicit bg color from filter buttons to fix pale appearance (#60346 backport fix) (#61457): @vatsrahul1001 @bbovenzi
  Linked issues:
  • Improve Dags Filter UI (#60346)
• [v3-1-test] Add task_display_name to LightGridTaskInstanceSummary model (#61440) (#61505): @guan404ming
• [v3-1-test] select filepath using cursor (#61011) (#61506): @sarth-akvaish
• [v3-1-test] fix: fix minimap on DAG graph view not showing DAG nodes (#61511) (#61530): @itayweb
• [v3-1-test] Add error handling for pause/unpause toggle permission errors (#61389) (#61533): @Fury0508 @aliotta
  Linked issues:
  • Pause/unpause toggle does not handle DAG level perms 403 errors intuitively (#61363)
• [v3-1-test] Fix middleware order to prevent chunked FastAPI responses (#61043) (#61539): @abhijeets25012-tech @pierrejeambrun
• [v3-1-test] Fix GZipMiddleware with correct comment placement (#61538) (#61566): @Eason09053360 @jason810496
• [v3-1-test] Add access_key and connection_string to DEFAULT_SENSITIVE_FIELDS (#61580) (#61582): @sunank200
• [v3-1-test] Fix main content margin to align with navigation sidebar width (#61614) (#61622): @choo121600
• [v3-1-test] Fix missing translation keys for blocking dependencies in UI (#61314) (#61366) (#61638): @SakshamSinghal20
• [v3-1-test] Alias run_after for XComResponse (#61443) (#61672): @guan404ming
• [v3-1-test] Fix Triggerer crashing if Trigger uses builtin print function (#60258) (#61703): @henry3260
• [v3-1-test] Add XCom serilizer for pendulum.date.Date (#61176) (#61717): @amoghrajesh @mrlimacz
• [v3-1-test] Fix list dag versions permissions (#61675) (#61733): @pierrejeambrun
• [v3-1-test] Fix secrets masking in Rendered Templates for complex objects (#61394) (#61763): @vatsrahul1001 @amoghrajesh
• [v3-1-test] Fix slots negative infinity (#61140) (#61768): @KamranImaaz @pierrejeambrun
• [v3-1-test] grid merge node dict storage (#61656) (#61789): @steveahnahn
• [v3-1-test] Fix recursion depth error in _redact_exception_with_context (#61776) (#61795): @Crowiant @potiuk
  Linked issues:
  • Add handle for circular exceptions in SecretsMasker. (#61254)
• [v3-1-test] Make conn_type optional in task SDK Connection datamodel (#61728) (#61835): @amoghrajesh
• [v3-1-test] Fix scheduler crash when enqueuing TI with null dag_version_id (#61813) (#61846): @ephraimbuddy
• fix(ui): Fix star icon visibility in Favorite filter buttons when selected (#61862): @vatsrahul1001
• Backport on_task_instance_skipped listener hookspec to v3-1-test (#61863): @tstrilka
  Linked issues:
  • Add on_task_instance_skipped listener hookspec (#59370)
  • Add on_task_instance_skipped listener hookspec (#59467)
• [v3-1-test] Persist table columns visibility in local storage (#61858) (#61868): @guan404ming @pierrejeambrun
• [v3-1-test] Flush in-memory OTel metrics at process shutdown (#61808) (#61869): @xBis7
• [v3-1-test] fix(api): disable uvloop if PYTHONASYNCIODEBUG=1 to prevent segfault (#61281) (#61933): @jason810496 @subhash-0000 @lkindere
  Linked issues:
  • API server segfault when PYTHONASYNCIODEBUG=1 (#61214)
• [v3-1-test] Add LOG_MAX_SIZE environment variables to log groomer (#61559) (#61950): @pcorliss
• [v3-1-test] Fix scheduler heartbeat misses caused by slow reschedule dependency check (#61983) (#62068): @ephraimbuddy @guan404ming
• [v3-1-test] bulk connection var usage (#61570) (#62076): @steveahnahn @pierrejeambrun
• Fix DepContext mutation leak and restore reschedule-mode guard (#62089): @Lee-W @kaxil
  Linked issues:
  • Reschedule tasks on worker startup Dag load failures instead of exiting (#59604)
• [v3-1-test] Add missing ti.start and ti.finish metrics in Airflow 3 (#62019) (#62110): @amoghrajesh
• Do not dilute task failures with finalization errors during execution (#62070) (#62113): @amoghrajesh
• [v3-1-test] perf: optimize grid view refresh pressure on the API (#62085) (#62135): @viiccwen
• [v3-1-test] Use start_date and end_date to calculate duration when duration is null for running taskinstance (#61898) (#62136): @tirkarthi @guan404ming
• [v3-1-test] fix(ui): show active backfill in banner instead of first one (#61851) (#62137): @roykoand @Mr-Neutr0n
  Linked issues:
  • "Backfill in progress" banner shows the first backfill instead of the active backfill (#61811)
• [v3-1-test] Fix grid view crash when task converted to TaskGroup (#61208) (#61279) (#62181): @AlexSingle @y-sudharshan @pierrejeambrun
  Linked issues:
  • Broken grid view (#61208)
• [v3-1-test] fix(ui): only apply elk.portConstraints for LR orientation in graph view (#62144) (#62187): @rsilvestre
• [v3-1-test] docs: Fix typo and missing closing bracket in breeze installation docs (#62242) (#62268): @ahilashsasidharan
• [v3-1-test] fix(breeze): use prek from Breeze Python instead of system PATH (#62309) (#62317): @abhijeets25012-tech
• [v3-1-test] Revert "fix(breeze): use prek from Breeze Python instead of system PATH (#62309)" (#62332) (#62333): @jscheffl
• [v3-1-test] Switch all airflow logging to structlog (#52651) (#62339): @ricky-chaoju @ashb @Lee-W
  Linked issues:
  • Switch all airflow logging to structlog (#52651)
• [v3-1-test] SQL not rendered in Rendered Templates view (#60739) (#62348): @sarth-akvaish
• [v3-1-test] fix(ui): unique keys for pagination ellipses (#62352) (#62366): @yjaw
• [v3-1-test] Handle non-dictionary json payload during logging to avoid internal server error. (#62355) (#62367): @tirkarthi
• [v3-1-test] Fix WASB remote logging base path handling (#58946) (#61013) (#62456): @nailo2c
• [v3-1-test] Changed dag_bundle.signed_url_template from varchar(200) to text (#61041) (#62568): @justinpakzad @ashb
• [Backport 3-1-test] Add readable dags checks for the dependencies endpoint (#62046) (#62586): @shubhamraj-git
• Refactor DataTable to handle internal scrolling and fix overflow on narrow viewports (#62603): @choo121600
• [v3-1-test] Expose literal and ParamsDict at SDK top level (#59782) (#62756): @uranusjr
• [v3-1-test] Fix timer.duration unit labels in logs (#61824) (#62757): @jedcunningham @vatsrahul1001
• [v3-1-test]Fix: Restore task_instance_history sequence on downgrade (#62759): @vatsrahul1001
• Fix XCom migration failing for NaN/Infinity float values (#62686) (#62760): @jedcunningham @amoghrajesh
  Linked issues:
  • Ensure that DB migrations handles all kinds of NaN values in historical xcoms (#57866)
  • Fix XCom migration failing for NaN/Infinity float values (#62686)
• [v3-1-test] Remove mp_start_method remnants (#61150) (#62762): @vatsrahul1001 @Nataneljpwd
• [V3-1-test] Fix broken dag_processing.total_parse_time metric (#62128) (#62764): @vatsrahul1001 @nickstenning
• [V3-1-test] Fix Trigger UI form rendering for null enum values (#62060) (#62767): @Subham-KRLX @vatsrahul1001
• [V3-1-test] UI variables page: added option to view text as multi lines (#61679) (#62779): @Tomi-1997 @pierrejeambrun
• [V3-1-test] Variable table handle long words break when values are expanded (#62416) (#62781): @pierrejeambrun
• [V3-1-test] use ISO dates in Gantt chart for cross-browser consistency (… (#62784): @yuseok89 @pierrejeambrun
• [V3-1-test] Override tar dependency in Simple auth manager (#62787): @pierrejeambrun
• [V3-1-test] Fix minimatch ReDoS vulnerabilities via pnpm overrides (#62805): @pierrejeambrun
• [v3-1-test] Add logging to detect try number race (#62703) (#62821): @ephraimbuddy
• [v3-1-test] Fail prod image release when constraint build fails (#62387) (#62833): @potiuk
  Linked issues:
  • Do not fallback to "no-constraints" in release PROD image builds (#62349)
• [v3-1- test] Scope session token in cookie to base_url (#62771) (#62851): @vatsrahul1001 @wolfdn
• [v3-1-test] Backport UI of Scope session token in cookie to base_url (#62771) (#62859): @jason810496
• [v3-1-test] fix missing logs in UI for tasks in UP_FOR_RETRY and UP_FOR_RESCHEDULE states (#54547) (#62862): @ido177
• [v3-1-test] Fix backfill permissions (#62856) (#62873): @pierrejeambrun
• [v3-1-test] fix: use useAssetServiceGetDagAssetQueuedEvents to get the correct number of ADRQs (#62868) (#62902): @Lee-W
• [v3-1-test] Fix: Adds task instance validation for hitl (#62886) (#62909): @aritra24
• [v3-1-test] Make elasticsearch compatible with remote_task_log (#62121) (#62940): @Owen-CH-Leung
• [v3-1-test] fix: always include kid in JWT header for symmetric key tokens (#62883) (#62943): @YoannAbriel
• [v3-1-test] Upgrade tar (#62939) (#62947): @vincbeck
• [v3-1-test] Enable dependabot for simple auth manager in v3-1-test (#62951) (#62958): @vincbeck
• [v3-1-test] Only upgrade patch and minor NPM version in v3-1-test (#62961) (#62967): @vincbeck
• [v3-1-test] Return empty list for queued asset events instead of 404 (#62934) (#62976): @Lee-W @guan404ming

Thanks to all who contributed to the release (probably not a complete list!):
@jedcunningham @Fury0508 @SakshamSinghal20 @tirkarthi @ashb @AlexSingle @gichul-hong @roykoand @aliotta @yuseok89 @viiccwen @Lee-W @jscheffl @ahilashsasidharan @nailo2c @choo121600 @ido177 @xBis7 @KamranImaaz @nathadfield @vincbeck @ricky-chaoju @wolfdn @steveahnahn @Eason09053360 @subhash-0000 @kaxil @YoannAbriel @abhijeets25012-tech @Nataneljpwd @Mr-Neutr0n @nickstenning @Subham-KRLX @ephraimbuddy @jason810496 @bugraoz93 @potiuk @shubhamraj-git @vatsrahul1001 @yjaw @justinpakzad @itayweb @uranusjr @bbovenzi @rsilvestre @Owen-CH-Leung @mrlimacz @RoyLee1224 @sunank200 @sarth-akvaish @amoghrajesh @aritra24 @lkindere @jvoravong @Tomi-1997 @guan404ming @tstrilka @pcorliss @henry3260 @Crowiant @y-sudharshan @pierrejeambrun

Committer

• I acknowledge that I am a maintainer/committer of the Apache Airflow project.

[tstrilka]

Tested: #61863 - on_task_instance_skipped listener hookspec. Verified that Airflow 3.1.8rc1 natively emits OpenLineage terminal events for skipped tasks. The fix works as expected.

[nailo2c]

Hi, I've verified my part on Airflow 3.1.8rc1, and everything looks good! 🙌

[justinpakzad]

Confirming my changes are all good.

[mrlimacz]

I can confirm [v3-1-test] Add XCom serilizer for pendulum.date.Date (#61176) works as well.

[potiuk]

Tested all my changes. Everything looks good.