100% rate detection by Google using StealthyFetcher

[orchidkid]

Well, the title pretty much speaks for itself.

Code used:

`from scrapling.fetchers import StealthyFetcher
from scrapling.parser import Selector

page = StealthyFetcher.fetch('https://www.google.com/search?q=something',
headless=False,
block_images=False,
disable_resources=False,
google_search=True,
block_webrtc=False,
geoip=True,
load_dom=True,
)
print(page.body)`

Do I use proxies? - Nope
Can I access Google without captcha using incognito mode in my browser? - Yes

[orchidkid]

UPD. I've just looked through the docs and I see it uses Camoufox. Well, now it is clear why it gets detected. Currently Google somehow is able to detect Camoufox while old as hell Playwright patches work with way lower detection rate.

[D4Vinci]

Your IP is currently flagged due to the tests, which is why you're getting blocked. You need to start using proxies.

All the speed tests I do with every update to test the fetching speed are done on Google, and I rarely get blocked. Speed tests mean I do a lot of requests in a short time, and I still don't get blocked without proxies.

[orchidkid]

If that were really the case, Google would block me regardless of which library I use, and most likely would throw CAPTCHAs at me even with a regular browser. However, the other solutions I tested work, unlike Scrapling. Too bad, I actually liked Scrapling. Anyways, good luck with the project!

[D4Vinci]

If that were really the case, Google would block me regardless of which library I use, and most likely would throw CAPTCHAs at me even with a regular browser. However, the other solutions I tested work, unlike Scrapling. Too bad, I actually liked Scrapling. Anyways, good luck with the project!

That's weird, bro. Does that happen with DynamicFetcher while stealth mode is enabled? Does it occur with stealth mode enabled and real_chrome=True passed? What other libraries have worked with you? That would be useful to pinpoint the issue.

Calm down, mate. Most users report issues like that, which often involve problems with their IP reputation. That's why I responded as I did to save time. If the issue persists, you can use Scrapling's Selector with the other working library, so it's like your own fetcher.

[orchidkid]

I really didn’t mean any aggression or anything like that xD

I just described the problem as it is, and since StealthFetcher uses Camoufox, which is actually what gets detected (I also mentioned this in the Camoufox repo and some people confirmed the issue), I pointed out that it’s not surprising.

As for working solutions, I’d better DM you. Maybe it’s just paranoia, but Google is currently hiring a specialist to improve their anti-bot systems and to monitor the internet for what's having success, so I’d rather not reveal approaches that still work publicly.

Also, yes, I tried DynamicFetcher with stealth_mode. As for real_chrome, no, I haven’t tried that.

[D4Vinci]

Sure, could you just hit me up on Discord?