test: add e2e tests for Scrapy spiders running as Apify Actors

[vdusek]

Summary

Add 5 new e2e tests covering key Scrapy spider types and integration patterns, all using a local HTTP server for reliability. Tests cover basic Spider, CrawlSpider with rules, ItemLoader, cb_kwargs serialization, and custom pipeline merging with apply_apify_settings.

Motivation

• Better test coverage for Scrapy integration, up until now, we have only 1 E2E test, and that was all.

Issue

• Relates: Improve test suite: increase coverage and reduce total execution time #785

Test plan

• CI passes

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.78%. Comparing base (a99b5ff) to head (252f010).
⚠️ Report is 2 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #788      +/-   ##
==========================================
- Coverage   81.86%   81.78%   -0.08%     
==========================================
  Files          46       46              
  Lines        2696     2696              
==========================================
- Hits         2207     2205       -2     
- Misses        489      491       +2     

Flag	Coverage Δ
e2e	43.50% <ø> (ø)
integration	55.45% <ø> (ø)
unit	67.95% <ø> (-0.08%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[vdusek]

@janbuchar, @Mantisus - not necessary requesting a review, up to you guys, mostly just to keep you informed about the SDK test suite improvements.