Skip to content

feat(worker): add upload_finisher sweep for pending uploads#758

Merged
thomasrockhu-codecov merged 6 commits intotomhu/single-finisher-gatefrom
tomhu/upload-finisher-sweep
Mar 11, 2026
Merged

feat(worker): add upload_finisher sweep for pending uploads#758
thomasrockhu-codecov merged 6 commits intotomhu/single-finisher-gatefrom
tomhu/upload-finisher-sweep

Conversation

@thomasrockhu-codecov
Copy link
Copy Markdown
Contributor

@thomasrockhu-codecov thomasrockhu-codecov commented Mar 11, 2026

Summary

  • schedule a delayed finisher sweep when coverage uploads remain in UPLOADED
  • keep gate key while work is pending and clear it once finishing is complete
  • add focused unit tests for sweep scheduling and gate cleanup behavior

Test plan

  • ruff check --fix apps/worker/tasks/upload_finisher.py apps/worker/tasks/tests/unit/test_upload_finisher_task.py
  • ruff format apps/worker/tasks/upload_finisher.py apps/worker/tasks/tests/unit/test_upload_finisher_task.py
  • worker CI

Stacked on #757.

Made with Cursor


Note

Medium Risk
Adds retry/sweep scheduling and Redis gate lifecycle management to the upload finisher, which changes task orchestration and could affect notification timing or leave commits stuck if misconfigured. Risk is mitigated by explicit max-attempt bounds and expanded unit coverage for terminal/exception paths.

Overview
Adds a delayed finisher “sweep” retry loop when coverage uploads are still in UPLOADED, rescheduling upload_finisher with a bounded sweep_attempt counter and returning structured sweep status instead of silently exiting.

Introduces explicit Redis finisher-gate lifecycle management: sweeps expire the gate to keep it alive while pending work remains, and terminal outcomes now delete the gate on success, idempotent early-exit, max-sweep exhaustion, and exception/timeout paths.

Refactors pending-upload detection into _count_remaining_coverage_uploads and expands unit tests to cover sweep scheduling, max-attempt behavior, and gate cleanup across success and error scenarios.

Written by Cursor Bugbot for commit 0f777f4. This will update automatically on new commits. Configure here.

Comment thread apps/worker/tasks/upload_finisher.py Outdated
Comment thread apps/worker/tasks/upload_finisher.py Outdated
Comment thread apps/worker/tasks/upload_finisher.py
@thomasrockhu-codecov thomasrockhu-codecov force-pushed the tomhu/single-finisher-gate branch from bebfe63 to c07692a Compare March 11, 2026 14:21
@thomasrockhu-codecov thomasrockhu-codecov force-pushed the tomhu/upload-finisher-sweep branch from ead0d67 to 0b6177f Compare March 11, 2026 14:21
Comment thread apps/worker/tasks/upload_finisher.py
@thomasrockhu-codecov thomasrockhu-codecov force-pushed the tomhu/single-finisher-gate branch from c07692a to fea2836 Compare March 11, 2026 14:28
@thomasrockhu-codecov thomasrockhu-codecov force-pushed the tomhu/upload-finisher-sweep branch from 0b6177f to a4761d3 Compare March 11, 2026 14:30
Comment thread apps/worker/tasks/upload_finisher.py
Comment thread apps/worker/tasks/upload_finisher.py
@thomasrockhu-codecov thomasrockhu-codecov force-pushed the tomhu/upload-finisher-sweep branch from a4761d3 to 14d4299 Compare March 11, 2026 14:41
Comment thread apps/worker/tasks/upload_finisher.py
@thomasrockhu-codecov thomasrockhu-codecov force-pushed the tomhu/single-finisher-gate branch from 73b8ff6 to 44de224 Compare March 11, 2026 14:56
@thomasrockhu-codecov thomasrockhu-codecov force-pushed the tomhu/upload-finisher-sweep branch from 14d4299 to ef23608 Compare March 11, 2026 14:56
Comment thread apps/worker/tasks/upload_finisher.py
Comment thread apps/worker/tasks/upload_finisher.py
@thomasrockhu-codecov thomasrockhu-codecov force-pushed the tomhu/single-finisher-gate branch from 44de224 to bf5da1c Compare March 11, 2026 15:06
@thomasrockhu-codecov thomasrockhu-codecov force-pushed the tomhu/upload-finisher-sweep branch from ef23608 to ee8bf7c Compare March 11, 2026 15:06
Comment thread apps/worker/tasks/upload_finisher.py
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Comment thread apps/worker/tasks/upload_finisher.py
Comment thread apps/worker/tasks/upload_finisher.py
@thomasrockhu-codecov thomasrockhu-codecov force-pushed the tomhu/upload-finisher-sweep branch 5 times, most recently from ad194c8 to 673b32c Compare March 11, 2026 15:33
@thomasrockhu-codecov thomasrockhu-codecov force-pushed the tomhu/single-finisher-gate branch from 5651fd6 to b508cb4 Compare March 11, 2026 15:35
Comment thread apps/worker/tasks/upload_finisher.py
@thomasrockhu-codecov thomasrockhu-codecov force-pushed the tomhu/upload-finisher-sweep branch 2 times, most recently from c7ce723 to 0315a23 Compare March 11, 2026 15:39
@thomasrockhu-codecov thomasrockhu-codecov force-pushed the tomhu/single-finisher-gate branch from 7e12f77 to c1c5f19 Compare March 11, 2026 15:48
@thomasrockhu-codecov thomasrockhu-codecov force-pushed the tomhu/upload-finisher-sweep branch from f0713fe to 166f59a Compare March 11, 2026 15:48
@thomasrockhu-codecov thomasrockhu-codecov force-pushed the tomhu/single-finisher-gate branch from c1c5f19 to e9dbaec Compare March 11, 2026 15:51
@thomasrockhu-codecov thomasrockhu-codecov force-pushed the tomhu/upload-finisher-sweep branch from 166f59a to 4d971bd Compare March 11, 2026 15:51
Comment thread apps/worker/tasks/upload_finisher.py Outdated
@thomasrockhu-codecov thomasrockhu-codecov force-pushed the tomhu/upload-finisher-sweep branch 2 times, most recently from 71b0bf0 to 4907ad9 Compare March 11, 2026 16:12
@thomasrockhu-codecov thomasrockhu-codecov force-pushed the tomhu/single-finisher-gate branch from 9bce158 to 2d657c4 Compare March 11, 2026 16:14
@thomasrockhu-codecov thomasrockhu-codecov force-pushed the tomhu/upload-finisher-sweep branch 2 times, most recently from 9b070fe to 927b5d3 Compare March 11, 2026 16:22
When coverage uploads are still pending, upload_finisher now schedules a delayed sweep instead of exiting, and it clears the commit gate key once finishing work is complete.

Made-with: Cursor
Reuse the canonical finisher gate key helper, cap sweep reschedules, preserve flow breadcrumbs during deferred sweeps, and clear gate keys on terminal failures.

Made-with: Cursor
Delete the gate and return an accurate sweep_scheduled flag when max sweep attempts are exhausted, so commits are not blocked behind stale gate keys.

Made-with: Cursor
Only clear the gate after successful finisher-lock completion so terminal lock failures do not silently drop pending commit post-processing.

Made-with: Cursor
Extend the commit gate expiration whenever a sweep is scheduled so the gate remains valid throughout long-running sweep recovery loops.

Made-with: Cursor
…ustion

Delete the finisher gate even when _handle_finisher_lock returns None (terminal lock exhaustion) and add coverage to prevent gate stalling regressions.

Made-with: Cursor
@thomasrockhu-codecov thomasrockhu-codecov force-pushed the tomhu/upload-finisher-sweep branch from 927b5d3 to 0f777f4 Compare March 11, 2026 16:35
@thomasrockhu-codecov thomasrockhu-codecov merged commit 0f777f4 into tomhu/single-finisher-gate Mar 11, 2026
20 checks passed
@thomasrockhu-codecov thomasrockhu-codecov deleted the tomhu/upload-finisher-sweep branch March 11, 2026 16:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant