Suicide and the Internet: Some flaws in the study

Webometric investigations rarely gain mainstream interest, yesterday, however, one did: A content analysis of the top 10 sites, on the four major search engines, for 12 searches relating to suicide. This highlighted the large number of hits that were to 'dedicated suicide sites' (e.g.. pro-suicide, encouraging, describing methods, or portraying suicide in fashionable terms): 90 out of 480 hits. Unsurprisingly this gained the interest of numerous news sites including the BBC. There are, however, a number of problems with the study: not all search terms are equal, and not all search engines are equal. Whilst we all make sweeping statements about web phenomena, we should really save it for our blogs rather than publication in the likes of the British Medical Journal (BMJ).

The main problem of the investigation is a focus on the information that is retrievable rather than the information that is actually being retrieved, which quickly muddies the water. Whilst the combining of search engines would initially seem to underestimate the scale of the problem, the propensity of users to use certain search terms would seem to indicate that the article has overestimated the scale of the problem.

The Google Effect
The majority of the statistics provided in the paper are based on the combined results of Google, Yahoo, MSN, and Ask:
-90/480 were dedicated suicide sites
-62/480 were sites forbidding suicide
-59/480 were sites discouraging suicide
However, almost 70% of searches use Google, which as the results show has the highest number of dedicated suicide sites in the results. This would seem to underestimate the problem: whereas just under a fifth of the hits were dedicated suicide sites overall, for the most influential search engine this has risen to just under a quarter. However, when looking at the search terms used, we soon reaslise that the problem has been over-stated.

Search Term Analysis
Whilst the BMJ lists the 12 search terms used, gathered partly from interview data and search suggestions used by search engines, a quick investigation quickly shows that they are by no means used in equal measure. Of the twelve terms only 4 were used often enough to generate search graphs in Google Trends:
-suicide
-sucide methods
-how to commit suicide
-how to kill yourself
And even amongst these four there was a wide variation in usage, with the overwhelming majority of queries being generated by the term suicide:


A content analysis of Google's 'suicide' results
Below are the top ten links I received when looking at the global results from google.co.uk for the term 'suicide', and how I would classify them. Whilst the BMJ study emphasises that is doesn't restrict the results to the UK, it does not mention whether it uses google.com or google.co.uk. I have used google.co.uk as, unless you ask it otherwise, google.com will redirect British users to google.co.uk.
-Miscellaneous - Wikipedia's suicide page
-Against suicide - Suicide...read this first
-Against suicide - Suicide.com
-Academic or policy site - Mind fact sheet
-Academic or policy site - Stanford encyclopaedia of philosophy
-Prevention or support site - Kids Health answers and advice -suicide
-Prevention or support site - Problems of life: Suicide
-Not relevant - Facebook suicide: the end of a virtual life
-Prevention or support site - Depression and suicide in men
-Prevention or support site- BBC: Health conditions: Suicide
Whilst classification is notoriously difficult to get agreement on, none of these sites could be considered the sort of 'dedicated suicide sites' that will spread panic through middle-England.

I have no doubt that there are plenty of sites on the web that encourage suicide, but before we start a panic we need to have a greater understanding of how people are searching on the topic of suicide when they are feeling suicidal. We can't just lump together the findings of different searches on different search engines and say that statistically we have a problem.

The most popular search on the most popular search engine on the topic of suicide does not find any 'dedicated suicide sites'.

The original BMJ article:
Biddle, L., Donovan, J., Hawton, K., Kapur, N., & Gunnell, D. (2008). Suicide and the internet. British Medical Journal, 336(12 April 2008), p. 800-802.
Can be found here.