Skip to content

feat: LLMO-3330: add submitScrapeJob and lookupScrapeResults to DrsClient#1405

Open
rarescheseli wants to merge 2 commits intomainfrom
rcheseli/enhance-drs-client
Open

feat: LLMO-3330: add submitScrapeJob and lookupScrapeResults to DrsClient#1405
rarescheseli wants to merge 2 commits intomainfrom
rcheseli/enhance-drs-client

Conversation

@rarescheseli
Copy link

@rarescheseli rarescheseli commented Mar 5, 2026

Summary

  • Add submitScrapeJob method to submit scraping jobs to DRS via the Bright Data provider, supporting configurable priority (HIGH/LOW) and five dataset types (youtube_videos, youtube_comments, reddit_posts, reddit_comments, wikipedia).
  • Add lookupScrapeResults method to query scraping results for a batch of URLs via the /url-lookup endpoint, returning per-URL status (available, scraping, not_found) with presigned S3 URLs for completed results.
  • Add README.md

Please ensure your pull request adheres to the following guidelines:

  • make sure to link the related issues in this description
  • when merging / squashing, make sure the fixed issue references are visible in the commits, for easy compilation of release notes

Related Issues

Thanks for contributing!

@github-actions
Copy link

github-actions bot commented Mar 5, 2026

This PR will trigger no release when merged.

@rarescheseli rarescheseli marked this pull request as ready for review March 5, 2026 11:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant