Degrading Transparency: Comparing Google, Yahoo and Microsoft

Google, Yahoo! and Microsoft all maintain versions of their search engines for the Chinese market that censor political content. One of the key issues that emerged concerned transparency. In 2006, all three search engines, following Google’s lead, introduced a message that informed user when the results of their searches were censored. The presence of a mechanism of notification is a critical component of transparency. This notification informs users that their search results have been censored and indicates, to a certain degree, the reason (often unspecified “local law”) why based on what the user searched for. The message appeared only when the user’s results were censored and thus it was possible to connect the censorship to specific keywords or websites.

By 2008 the level of transparency has decreased. While Google’s censorship notification has remained essentially the same as it was in 2006, Yahoo! and Microsoft have altered the way in which users are notified of censorship. Yahoo! has put its censorship message at the bottom of every page regardless of whether results are censored or not, in effect de-linking the censorship notification from the results. Microsoft has removed the text completely and buried the censorship notification with a separate “help” page. These developments represent a significant degrading of transparency and accountability.

By removing or hiding the placement of the censorship message, which is vague to begin with, users may be unaware that their results have been censored and by de-linking the censor message from what the user actually searched for the topics and websites that are censored remain hidden from the user. The de-linking of the censorship message from the search results impacts the ability to determine what precise sites and “key words” are being censored.

The presence and placement of a censorship notification, along with the specificity of its content and its connection to the results, is an integral component of transparency. The specificity of the reason why content has been removed is an important component that is lacking in the case of China. In other cases, Google has cited specific laws, such as the DMCA, and other legal documents with which they must comply and reported the information, to some degree, to Chilling Effects.org. Yahoo maintains a list of sites its censored for copyright violations. However, in the case of censored political content in the case China nothing other than a reference to “local law” has been provided.

The presence of a notification that is directly connected to the results (notification appears only when content is actually removed in relation to what the user searches for) positively impacts the ability to accurately identify censored website and restricted keywords. When such notifications are either absent or disconnected from the results (for example, a notification that appears on every page regardless of whether results are censored or not) the ability to determine censored sites with a high degree of confidence diminishes as sites may simply not be indexed by the search engine. Therefore the notification is critical not only for informing users but also for the monitoring process.

June 26, 2006
Engine	Presence	Placement	Specificity	Connection	Screenshot
Google	Yes	High Notification is placed under results	Low Results removed to comply with local law	Yes Notification only appears when results are censored	screenshot
Yahoo	Yes	High* Notification is placed under results	Low Results removed	Yes*	screenshot
Microsoft	Yes	High Notification is placed under results	Low Results removed, link to “help” page that mentions local law	Yes	screenshot

* Yahoo China’s web crawlers operate from within China, behind the GFW, therefore sites that are blocked by China are not indexed by Yahoo (and thus do not need to be censored by Yahoo) leaving only sites that are either not blocked by China or are indexed during periods when there is variation in the capacity of China’s filtering system to actually be censored by Yahoo. The behaviour documented here refers to sites indexed by Yahoo but subsequently censored, not sites that are not indexed by Yahoo at all.

January 25, 2008
Engine	Presence	Placement	Specificity	Connection	Screenshot
Google	Yes	High Notification is placed under results	Low Mentions “local law”	Yes Notification only appears when results are censored	screenshot
Yahoo	Yes	Medium Notification is placed at the bottom of every page	Low Mentions “local law”	No	screenshot
Microsoft	Yes**	Low A link to a separate “help” page which contains a link to section that contains the notification	Low A link to a separate “help” page which contains a link to section that mentions “local law”	No	screenshot

** There is no notification on the actual page that results the search results. The user must click a “help” page and then navigate to yet another section that state that results may be removed in compliance with local law. the notification mentions pornography as a possible reason, no mention is made of political content.

Presence: The presence of a form of notification that informs users that results may be censored.
Placement: The location of the censorship notification message, particularly its placement in relation to the results.
Specificity: The extent to which users are informed about specific laws, orders and/or regulations leading to censored results.
Connection: Notification appears only when content is actually removed in relation to what the user searches for making it possible to determine which specific web sites and keywords have actually been censored.

(The versions of the search engines tested are the specific version for China. Google (www.google.cn) and Microsoft (www.live.com/?mkt=zh-cn) have their servers located outside of China and are tested directly while Yahoo’s (www.yahoo.cn) servers are hosted in China and are tested from inside China. This is necessary to test the search engines without interference from China’s filtering system.)

This is the start of an effort to more systematically monitor transparency over time so I am asking for feedback. Is this information useful? In what ways can it be improved?

1. clemens bee

   Very interesting thanks! more descriptive stats are needed…
   really quick 2 things that i would also find appealing:
   - I’m still wondering about the impact on these sites given the dominance of baidu.
   - more specifically, how often does the google censorship notice pop up? only with really obvious keywords as tianmen or is it a rather common occurence? how significant/specific is the notice? so from a userperspective, how much can i get out of it, except the constant very noticable censorship warning?
2. nart

   Thanks. This is just the first part of the project. The next component will include analysis based on the testing of the search engines and will include results from Baidu as well.

   The Google censorship notice appears whenever a url is removed from the results, with the exception of a few select “key words” it is not dependent on what you search for but what is returned in the results. So you may often see the censor notice even when you use key words that have nothing to do with China or anything that they’d want to censor.

   For example, a search for darfur in Google China will show the censorship notification — because the BBC News site, which is censored, appears in the top ten results for that term. It has nothing to do with the topic of darfur.

   From the user perspective you are informed that your results are censored, but no specifics are required. However, it is possible to determine what is triggering the censorship message, but it is unlikely that the average user is able to do so.
3. Rebecca MacKinnon

   Hey there Nart! Great to see that you continue to follow this issue closely. A few things.
   1. Actually when Yahoo China started running the disclaimer in summer 2006 (on July 27, 2006 to be exact), I was in the middle of helping Human Rights Watch put together their “Race to the Bottom” report… and at the time I found that the disclaimer was running on all search result pages regardless of content. I was not subsequently aware of a point in time when the disclaimer was being displayed more selectively. I note that your screenshot is from June 2006, before the disclaimer started to be implemented. If you’ve got screenshots after late July 2006 that show varied use of the disclaimer I’d love to see them because I’d really like to have a more comprehensive picture of how practices have been evolving. At the time here is what we wrote in the section on Yahoo!:

   On July 27, 2006, Yahoo! China began running a disclaimer notice at the bottom of all search pages, which says in Chinese “According to relevant laws and regulations, some search results may not appear.” While this represents a step in the right direction, Human Rights Watch does not believe that this notice in small print at the very bottom of all search results pages (regardless of the search term) represents “maximum transparency to the user” as stated by Yahoo! to be the company’s goal in congressional testimony. This is especially the case when it is clear from test results that Yahoo! censors its results more heavily than its competitors but gives the user no explanation as to why this is necessary. “Maximum transparency to the user” would entail informing users of how many results have been censored and why, and giving clear information about how the search engine’s censorship decisions get made, so that the user knows what he or she is missing and knows who is responsible for the content’s absence. Without such steps, the search engine continues to play the role of non-transparent censor.

   2. At the time I was testing from May until early August 2006, I found Microsoft’s display of disclaimers - and other related information - to be very inconsistent. There were some pages where results had obviously been censored that did not display the disclaimer, and others that did. This lack of consistency was noted in our Microsoft section. As you describe, Microsoft in 2006 was burying the censorship explanation on a separate help page which the user had to click through to. The censorship message said: “The search results have omitted some content. [click here to] Find out why.” The hyperlinked text then took the user to an explanatory page containing explanations of a list of features and potential questions related to MSN search results, then at the very bottom of the page was the heading “when there is no results or filtered results” below which was the explanation that some content may be removed in accordance with laws and regulations.

   I agree with clemens, it would be really great to see much more finely- grained testing that shows patterns re: when the censorship notice does and doesn’t show up on Google and MSN search. Such testing is definitely labor intensive and to be useful requires a team of near-native Chinese speakers who are highly tuned-in to what the sensitive media topics and hot online buzz happen to be in China at any given moment.