Google has announced that it will discontinue free access to its full search index for developers. New Programmable Search Engines can now only search a maximum of 50 domains. The previously available "Search the entire web" option will no longer be available for new engines.

Operators of existing search engines that index more than 50 domains or use the full web index must switch to an alternative by January 1, 2027. Google justifies the move in the announcement with an "evolution towards focused, more powerful solutions" that are intended to offer a better user experience.

Vertex AI Search as a paid alternative

As a replacement for free full access, Google points to Vertex AI Search, a cloud-based enterprise service with AI-powered features such as conversational search and grounding (anchoring AI answers in verifiable data sources). Those who wish to continue using the full Google index must fill out a form and await a custom price quote. There are no public prices; previous paid API offers cost around US$5 per 1000 requests.

The Custom Search JSON API will also be discontinued. Users must port their implementations to Vertex AI or the new Enterprise-Full-Web service by the deadline. The free "Sites to search" feature for a maximum of 50 domains will remain available and, according to Google, is ideal for focused, site-specific search results.

Indie developers under pressure

The changes particularly affect developers of niche search engines, educational institutions, and non-profit organizations. Many WordPress plugins and Drupal modules based on Google's Programmable Search Engine will need to be rebuilt or discontinued.

Self-hosted software such as Meilisearch, Typesense, or Elasticsearch are available as alternatives. For web index-based applications, services like Common Crawl (an open web archive) could be used in combination with custom models. However, these alternatives do not achieve the same level of currency and completeness as the Google index.

Antitrust concerns?

The restrictions could raise antitrust issues in the EU. As a gatekeeper within the meaning of the Digital Markets Act, Alphabet, with Google Search, controls access to a significant infrastructural element of the internet. It remains to be seen whether the abolition of free access, coupled with the introduction of paid alternatives, will be interpreted as anti-competitive.

Google, on the other hand, argues that simplifying its product portfolio serves quality: "We are simplifying and modernizing our offerings so you can choose the best tool for your goals," the blog post states.

(fo)