Sponsored Study Argues Google’s “Rival Links” Don’t Drive Enough CTRs To Rivals

google-eu-200pxGoogle’s original antitrust settlement proposal, negotiated with the European Commission, was attacked by Google competitors and critics as being a token concession to competition. Anti-Google lobbying group FairSearch.org sponsored a consumer study that argued Google’s presentation of three “rival links” to competitors in search results did little to drive traffic to competitors’ websites.

In September Google presented a second settlement proposal that made new or additional concessions. The second proposal was immediately attacked as well. Now a second FairSearch-sponsored research study is out. It consists of three UK-based consumer surveys with a total of 2,500 respondents.

Conducted by the same two US academics who produced the first round of sponsored research, University of Illinois professor David Hyman and University of San Francisco professor David Franklyn, this second piece of research comes to an essentially identical conclusion:

[W]e found Google’s Second Commitments are not likely to materially increase or restore consumer choice or competition in the vertical search markets at issue. Google’s Second Commitments do not achieve the Commission’s stated objective of materially increasing the visibility of rivals and the awareness of those rivals by consumers.

It’s safe to say that going in everyone involved in designing and implementing the study is aware of the objectives of the sponsor — to use the research to discredit the Google settlement proposal. The fact that the research is not truly independent damages — some would argue fatally — both the credibility and the conclusions of the study it can’t be entirely dismissed on that basis. I’ve embedded the full study below.

The surveys sought to explore three primary questions:

  • Whether the Second Commitments [Google's second proposal] resulted in a material number of clicks on any of the proposed rival links
  • The effect of other modification to search page layout and labeling on click patterns
  • The effectiveness of the disclosure accompanying the rival links in communicating basic information

Survey respondents were asked to perform a number of searches for products and hotels (in both mobile and PC scenarios). The PC product-search related “rival links” examined were from three shopping engines that operate in the UK: Kelkoo, Pricerunner and Bizrate. The study found that “any given rival link secured 2.3%-7% of vertical search clicks, compared to 87.7% of vertical search clicks for Google Shopping.”

The hotel rival links were from Expedia, Priceline and Trivago. The findings were similar: “in a hotel search, any given rival link secured 2.5%-10.9% of vertical search clicks, compared to 79.9% of vertical search clicks on Google’s vertical search options.”

The mobile results were even more skewed toward Google:

FairSearch anti-Google research

The researchers concluded that the second Google proposal offers the promise of modestly better click-through rates (CTRs) for rivals than the first. However they argue that the CTRs in relation to Google’s are inadequate “to restore competition.” This of course begs the question, what CTR level are competitors “entitled” to?

Should competitive links be getting 20 percent, 35 percent or 50 percent of the traffic — or more? This question is never directly addressed. Nor is any kind of comparison baseline or benchmark established: what’s a reasonable CTR for paid or organic results? What are the comparable CTRs for these sites on Bing or other engines?

The research also tested a hypothetical proposition: how the Google Local Carousel, not yet present in UK or EU results, would impact click-through rates. The study found that presenting the Carousel further diminished CTRs on non-Google results:

We found that any incremental increases in consumer attention that might arise as a result of enhanced presentation of rival links in the Second Commitments is subject to dissipation if Google were to introduce new search features, specifically a search feature that it has already introduced elsewhere (e.g., in the United States). This search feature is allowable under the Second Commitments.

The study is impliedly arguing that once a settlement is agreed upon Google’s discretion over the look and feel of its SERP should be severely restricted to ensure “stability” of  rival links’ CTR rates.

The researchers also did some A/B testing in presenting Google and non-Google rival links to consumers. (This is the most interesting and constructive part of the research.) They created a new presentation of alternatives that looked like the following:

Rival Links

Researchers tested different placements of Google within the box to see how they impacted CTRs. Overall, however, the idea was to put all competitors within the Google SERP on an “even playing field”:

The box was headed “Compare prices on,” and had icons for four vertical search offerings. The new box contained only icons and names of the vertical search offerings, and some abbreviated text describing the products that were available. Thus, the four vertical search offerings were placed on an even playing field (including equal visual footing) with one another. The image below shows an example of the box from the Razer product search, but we tested this approach in all three surveys.

In the US regulators arguably couldn’t compel any sort of specific SERP like this because search results have been held to be protected under the First Amendment, giving Google arguably total discretion over the look and feel of its product. However in Europe the rules are different and Google can’t claim the protections of the US Constitution.

This part of the research is likely to get attention and might become the source of a third, revised settlement proposal. Later in the summary of findings the researchers argue:

Absent parity of presentation, an appreciable number of consumers will be diverted from their preferred destination, which will reduce consumer welfare. Conversely, consumer welfare is enhanced when artificial impediments to consumer choice are removed and rival sites are positioned in a manner that is visibly equivalent to the positioning and layout of Google’s own specialized search results. 

The strong implication of this statement is that UK consumers were intending to visit a non-Google site and the placement of Google content on the SERP diverted them. But such consumer preferences are never discussed in the report. It may well be that consumers have no preference or they trust/prefer Google results. This is a key question.

If consumers have no intended site preference (but 90 percent of UK consumers start with/use Google) there is an argument that rivals should be presented in a competitively neutral manner. However an argument can equally be made that rivals are not “entitled” to any level of traffic or any specific CTR expectation. Perhaps their brands are weak.

But what if Google content is preferred; what if UK consumers actually trust and prefer Google results? This is something that should have been explored in the research. Because if Google is preferred to Priceline or Kelkoo, for example, these competitors can make no claim that they’re unfairly being denied traffic by the design of Google’s SERP.

There’s a great deal more in the study that I haven’t discussed. Read it below and make your own determination about the legitimacy of the findings. Legitimate or not, it does seem likely that this study will negatively impact the prospect of Google’s second antitrust settlement proposal being accepted without further changes.