Skip to content

[FIX]: test_start_ci_empty_token fails under CI rate limiting#1092

Merged
canihavesomecoffee merged 1 commit intoCCExtractor:masterfrom
x15sr71:fix/test-start-ci-empty-token-rate-limit
Apr 7, 2026
Merged

[FIX]: test_start_ci_empty_token fails under CI rate limiting#1092
canihavesomecoffee merged 1 commit intoCCExtractor:masterfrom
x15sr71:fix/test-start-ci-empty-token-rate-limit

Conversation

@x15sr71
Copy link
Copy Markdown
Contributor

@x15sr71 x15sr71 commented Apr 5, 2026

In raising this pull request, I confirm the following (please check boxes):

  • I have read and understood the contributors guide.
  • I have checked that another pull request for this purpose does not exist.
  • I have considered, and confirmed that this submission will be valuable to others.
  • I accept that this submission may not be used, and the pull request closed at the will of the maintainer.
  • I give this submission freely, and claim no ownership to its content.

My familiarity with the project is as follows (check one):

  • I have never used the project.
  • I have used the project briefly.
  • I have used the project extensively, but have not contributed previously.
  • I am an active contributor to the project.

Problem

Discovered while investigating the Python 3.13 build failure in PR #1091(failed run) test_start_ci_empty_token was failing intermittently with:

AssertionError: 418 != 500

The test is meant to verify that an empty GitHub token returns 500, but the request was being rejected by the request_from_github() decorator with 418 before ever reaching the token check.

Root cause: BaseTestCase.setUp() correctly resets the webhook IP cache to empty before every test for isolation. This means get_cached_web_hook_blocks() always makes a live call to api.github.com/meta. Under CI rate limiting, multiple parallel matrix jobs can collectively exhaust GitHub's unauthenticated rate limit. GitHub then returns a 403 whose JSON body has no hooks key. The KeyError handler in get_cached_web_hook_blocks() catches this silently, leaving the cache empty. is_github_web_hook_ip() then returns False and abort(418) fires — confirmed by this line in the failed log:

[CRITICAL] Failed to retrieve hook IP's from GitHub! API returned {'message': 'API rate limit exceeded for 172.183.157.181 ...', 'status': '403'}

Re-triggering after the rate limit reset passed, but this is nondeterministic and does not address the underlying issue.

Fix

Add @mock.patch('requests.get', side_effect=mock_api_request_github) to test_start_ci_empty_token, exactly matching the pattern already used by every other /start-ci test in the file (test_webhook_ping, test_webhook_release, test_webhook_push_github_api_failure).

mock_api_request_github already exists in tests/base.py for this purpose and returns valid GitHub hook IPs for the /meta URL.

Verification

Confirmed locally on Python 3.13:

Ran 424 tests in 61.576s OK

Notes

The test was introduced in commit 7c323ef with only @mock.patch('run.get_github_config') — the requests.get mock was missed. It passes locally because a single test run does not exhaust GitHub's unauthenticated rate limit, so the live call succeeds. CI exposes the missing mock because multiple parallel jobs can collectively exhaust the limit, causing the live call to fail.

@canihavesomecoffee canihavesomecoffee force-pushed the fix/test-start-ci-empty-token-rate-limit branch from fdc4ece to 5baca04 Compare April 7, 2026 19:17
@sonarqubecloud
Copy link
Copy Markdown

sonarqubecloud bot commented Apr 7, 2026

@canihavesomecoffee canihavesomecoffee merged commit 93e90ad into CCExtractor:master Apr 7, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants