Skip to content

fix(pipeline): check HTTP status in fetch step#384

Open
Astro-Han wants to merge 5 commits intojackwener:mainfrom
Astro-Han:fix/pr3-fetch-status
Open

fix(pipeline): check HTTP status in fetch step#384
Astro-Han wants to merge 5 commits intojackwener:mainfrom
Astro-Han:fix/pr3-fetch-status

Conversation

@Astro-Han
Copy link
Contributor

@Astro-Han Astro-Han commented Mar 24, 2026

Description

Normalize HTTP error handling in the pipeline fetch step. Single-request fetches now fail fast on non-ok responses, while per-item batch fetches return per-item { error } objects consistently in both browser and non-browser execution.

Behavior note: non-ok responses are no longer parsed as successful data. Single fetches throw immediately; batch fetches preserve per-item error objects instead of mixing browser/non-browser semantics.

Related issue: N/A

Type of Change

  • 🐛 Bug fix
  • ✨ New feature
  • 🌐 New site adapter
  • 📝 Documentation
  • ♻️ Refactor
  • 🔧 CI / build / tooling

Checklist

  • I ran the checks relevant to this PR
  • I updated tests or docs if needed
  • I included output or screenshots when useful

Test Plan

  • npx vitest run src/pipeline/steps/fetch.test.ts
  • npm run typecheck
  • npm test

Summary

  • fail fast on non-ok responses in single-request fetch execution
  • keep batch browser and non-browser fetch paths aligned with per-item { error } results
  • use CliError('FETCH_ERROR') instead of bare Error for consistent CLI error rendering
  • return error status from browser evaluate instead of throwing inside it (avoids CDP wrapper rewriting)
  • add log.warn() for batch item failures in both browser and non-browser paths
  • add regression tests covering error type, batch warnings, and non-Error stringification
  • remove unrelated .worktrees/ change from .gitignore

Screenshots / Output

Not applicable.

@Astro-Han
Copy link
Contributor Author

Astro-Han commented Mar 24, 2026

Updated this PR after follow-up review.

What changed:

  • kept single-request fetch as fail-fast on non-ok HTTP responses
  • aligned batch fetch semantics so browser and non-browser both return per-item { error } results
  • normalized non-Error rejected values consistently across both batch paths
  • expanded regression tests to cover browser/non-browser batch failures and primitive rejected values

Follow-up (review round 2):

  • replaced bare Error with CliError('FETCH_ERROR') across all throw sites
  • browser single-fetch now returns error status from evaluate and throws CliError outside
  • added log.warn() for batch item failures in both browser and non-browser paths
  • removed unrelated .worktrees/ from .gitignore

Validation:

  • npx vitest run src/pipeline/steps/fetch.test.ts
  • npm run typecheck
  • npm test

- Replace bare Error with CliError('FETCH_ERROR') for consistent CLI output
- Return error status from browser evaluate instead of throwing inside it
- Add log.warn() for batch item failures in both browser and non-browser paths
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant