
For AI builders, fresh data is everything, he said, and legacy search agent results page (SERP) scraper APIs are either “stale, restrictive, or shut down.” Enterprises spend thousands of dollars a month on web scrapers to get around this.
There are “compelling reasons” why developers would be interested in Perplexity’s Search API, agreed Thomas Randall, research director at Info-Tech Research Group. Eliminating crawling, deduplicating, ranking, and staying compliant with robots.txt (which instructs bots what they can and can’t access) can provide a “huge lift” for devs.
“Perplexity’s solution promises to abstract that away,” he said, describing a “plausible niche” where the Search API could become the default large language model (LLM) retrieval layer for startups and internal tools, especially if it integrates with popular models’ APIs.