© Perplexity
Two fundamentally opposing views of the web are clashing between Cloudflare and Perplexity. On one side, the traditional functioning of the web as it has existed for decades, where bots must adhere to the publisher's choices. On the other side, an AI that claims to act on behalf of the user and does not want to be subject to the same constraints.
Back to the sender. This week, Cloudflare accused Perplexity of 'stealth crawling': the bots of the 'AI answer engine' that go to search for information on the internet bypass the blocks and restrictions set by the publishers. Since the beginning, websites can grant permissions to crawlers (for example, Google's indexing robot) by listing them in the robots.txt file, either allowing them to retrieve content or prohibiting data collection.
Read Cloudflare denounces Perplexity's practices, accused of bypassing website protections.
In its response, Perplexity disputes the accusations, but claims the right to access public content to answer its users' requests. Unlike traditional crawlers that systematically explore the web to build databases, Perplexity claims its AI acts only upon user demand: it retrieves necessary information in real-time to answer a specific question, without storing or using these data for training its models.
The argument may be valid, but one can counter Perplexity by pointing out that there is no formal or legal distinction between a search engine's robot and an AI assistant — indeed, OpenAI respects the directives in the robots.txt file, claims Cloudflare. All robots, regardless of their origin, use automated queries to access websites. And the rules apply to all.
Another issue: the Perplexity agent acting "on behalf of the user" operates anonymously, as demonstrated by Cloudflare's investigation. It changes its user-agent and IP addresses: the transparency argument advanced by the startup is weakened. If the intentions of its robot are legitimate, why not reveal its true face?
Moreover, Perplexity commercially uses the information collected from the web. The content is certainly not stored and is not used for model training (knowing that the response engine relies on third-party AI models), but it is read, summarized, and presented in a proprietary interface with no direct link to the source. What about copyright and compensation for creators? Ignoring the rules that govern robot access, even in the name of the user, amounts to denying the explicit intention of the publisher.
Perplexity's response has a bit of trouble holding up. And the accusations of incompetence towards Cloudflare are not very well-mannered, especially since the engine does not address the technical criticisms on substance.
👉🏻 Follow real-time tech news: add 01net to your Google sources, and subscribe to our WhatsApp channel.
Source : Perplexity
- Google Play Store
- 4K Video Downloader Plus
- VLC Media Player
- Adobe Reader
- uTorrent
- WinRAR
- Xender
- Google Chrome
- CCleaner
- Mozilla Firefox
- Avast Free
- ChatGPT
- Microsoft 365
- Opera One
- Avast One










- Google Play Store
- 4K Video Downloader Plus
- VLC Media Player
- Adobe Reader
- uTorrent
- WinRAR
- Xender
- Google Chrome
- CCleaner
- Mozilla Firefox
- Avast Free
- ChatGPT
- Microsoft 365
- Opera One
- Avast One






