-
Notifications
You must be signed in to change notification settings - Fork 136
Open
Labels
bugSomething isn't workingSomething isn't working
Description
Description
Summary
The GCP client currently walks paginated API responses recursively. That works for small result sets, but it can fail when an API returns a large number of pages, such as Security Command Center findings collections.
The fix is to convert pagination from recursion to an iterative loop while preserving the existing request and next-page behavior.
Impact
- Large paginated collections can fail with Python recursion depth errors.
- Collection reliability drops for APIs with many small pages.
- A low-level client failure can block otherwise valid resource collection.
Affected Files And Changes
plugins/gcp/fix_plugin_gcp/gcp_client.py- Replace recursive
next_responses()paging with an iterative request loop. - Keep page accumulation logic unchanged.
- Replace recursive
plugins/gcp/test/test_gcp_client.py- Add a regression test that simulates 1500 pages and verifies the full result is collected.
Expected Behavior
- Pagination should continue until the API returns no next request.
- Result aggregation should be identical to the recursive implementation.
- Large page counts should not fail because of recursion depth.
Version
v4.3.0
Environment
Docker
Steps to Reproduce
- Start a collection against a large GCP project with a high volume of gcp_scc_finding
- Observe a python RecursionError
Logs
Additional Context
No response
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working