Stupid Twitter: Don't researchers deserve more than a second-class service?

I have never particularly seen the point of Twitter; it's the background noise I can do without. However, as an all-thing-web-2.0 researcher I recognise that there needs to be further investigation of how people are using it. Unfortunately Twitter doesn't seem to think so; yesterday they turned down my request to be upgraded from the extremely limited 100 API requests per hour.

After visiting Birmingham Social Media Cafe last Friday, and noticing the prevalence of Twitter names, I thought it would be interesting to get an overview of the Birmingham Social Media Cafe Twitter network: Had clusters formed? Did these clusters reflect different interests?...the usual sort of academic questions. In no time I had collected the Twitter IDs of 50 members of BSMC, and written a program that would check which of the members were following one another (using Twitter's 'friendship exist' method).

Unfortunately, to test every combination of names requires the sending of 50*49=2,450 requests. So even this extremely small scale study would require the program to run over 24hrs!! Last time I had collected data using Twitter's API there seemed to be no such limits. Whilst Twitter do offer the opportunity to be placed on a 'whitelist' that allows you 20,000 requests per hour, "...we only approve developers for the whitelist", and seemingly by their negative response they mean the distinctly commercial type of developer.

As the explanation link suggests, I was turned down because, as a researcher, I should be asking for the second-class data-mining feed:

It returns 600 recent public statuses, cached for a minute at a time. You can request it up to once per minute to get a representative sample of the public statuses on Twitter

This is the service Twitter have decided is most appropriate for "researchers and hobbyists", albeit one that would fail provide the sort of network information that I am interested in. A distinctly second-class service in comparison to the one offered to commercial developers.

I can understand if online services such as Twitter don't want to go out of their way to help academics, but it is rather disappointing that we are penalised for doing public research rather than chasing money. Whilst I will eventually be able to find a 24hr slot to run this particular program, it's a shame that I won't be able to run more large scale studies.

[Update: within hours of this post the Twitter API was updated, but no specific improvements for academics]