-
|
Hi, I'm pretty new to coding. I have only been doing it for a couple months mainly doing scrappers. I made a bunch of scrapers to gather information of some bigger sites which have pretty advanced bot detection. I have managed to bypass the bot protection finally after learning to use seleniumbase instead of playwright and etc. I have been using UC as True and enabling cdp to bypass but its just running too slow, so my scrappers aren't working. I have tried running digiproxy and lighting proxy but neither are fast enough. Does anyone have any tips that can help me advance. would be much appreciated. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
|
Make sure you've tried the CDP Mode and Stealthy Playwright Mode examples provided: If the goal is to bypass bot-detection, then it's probably not the best idea to automate as fast as possible on certain websites. |
Beta Was this translation helpful? Give feedback.
Make sure you've tried the CDP Mode and Stealthy Playwright Mode examples provided:
https://github.com/seleniumbase/SeleniumBase/tree/master/examples/cdp_mode
https://github.com/seleniumbase/SeleniumBase/tree/master/examples/cdp_mode/playwright
(They're an excellent starting point, and cover a lot of ground.)
If the goal is to bypass bot-detection, then it's probably not the best idea to automate as fast as possible on certain websites.
Exceeding rate limits by moving at super-human speeds can cause automation to get flagged as a bot.