Key Highlights
- Reddit filed a lawsuit against Perplexity for alleged data scraping.
- The lawsuit claims Perplexity bypassed Reddit’s guardrails to access content via Google SERPs.
- Rival AI company CEO speculated about a secret licensing deal between Reddit and Perplexity.
- Reddit created a “marked bill” test post to catch the data scraper, which led to Perplexity’s admission of wrongdoing.
The Data Scraping Scandal: Inside Reddit’s Legal Action Against Perplexity
In a legal battle that has raised eyebrows in the artificial intelligence (AI) community, Reddit Inc. has accused its competitor Perplexity AI LLC, a $20 billion company, of engaging in unauthorized data scraping. The incident has brought to light concerns about the integrity and boundaries of data collection practices within the rapidly growing AI industry.
The Legal Challenge
In an action filed on Wednesday in Manhattan federal court, Reddit detailed its allegations against Perplexity AI LLC, three data-scraping companies—Oxylabs UAB, AWM Proxy, and SerpApi. According to the lawsuit, Perplexity’s actions went beyond mere ethical concerns into outright legal violations. The company had agreed to follow Reddit’s instructions by blocking content scraping but reportedly continued to cite Reddit in its AI-generated answers more than ever, leading some to speculate about a secret licensing deal.
The CEO of another AI firm even suggested that the dramatic increase in citation frequency was due to Perplexity entering into such a deal.
However, Reddit’s lawsuit firmly denied this notion, asserting that there is no license between them and Perplexity. Instead, it accused Perplexity of engaging in “a scheme” to obtain Reddit’s data through circumvention of protective measures.
Rigging the Game
To prove its allegations, Reddit employed a clever tactic borrowed from internet infrastructure companies like Cloudflare. The social media giant created a test post that could only be crawled by Google’s search engine. Given that Perplexity does not have a content-licensing deal with Reddit but Google does, this setup was designed to catch any unauthorized data scraping.
The trap worked as planned.
Within hours of setting it up, queries to Perplexity’s ‘answer engine’ produced the contents of that test post, indicating that Perplexity had indeed scraped the content through Google’s SERPs. This admission left no doubt about Perplexity’s intentions and actions.
Industry Reactions
The revelation sent shockwaves through the AI industry. A spokesperson for Perplexity dismissed the allegations as a threat to “openness and the public interest.” Meanwhile, representatives from other companies involved in the lawsuit took different stances. Denas Grybauskas, chief governance and strategy officer of Oxylabs UAB, expressed shock at the accusations, while SerpApi committed to defending itself vigorously.
The case raises broader questions about data ethics in AI development and the potential for companies to circumvent contractual agreements through sophisticated data scraping techniques. As the legal proceedings continue, industry watchers will be closely following this case to understand its implications on future practices and regulations.
