Methodology and thoughts behind those PR Week Twitter stats

There’s a school of thought that says that what’s important in social media is to attempt to create debate, not consensus.

Cat Among The Pigeons

Peter Hay from PR Week and I appear to have been rather successful in this. This morning, PR Week published an article, Twitter has suddenly exploded. Almost immediately, Twitter (or at least our particular neighbourhood of Twitter) suddenly exploded.

One or two people were rather scathing: suggesting that the stats demonstrated that Peter and I didn’t understand the “essence of Twitter” or that they were “obviously flawed”, or that we had “redefined shallow”.

Indeed (horror of horrors) some people even went so far to suggest that Porter Novelli had ginned up the results to put us at the top. In fact, in PR week’s list, we came second. But no doubt this was a Machiavellian ploy — it’s details like those, Pooh Bah would say, that “give artistic verisimilitude to an otherwise bald and unconvincing narrative.”

I joke, but I can completely understand people’s strong feelings about this; PR Week was torn between a desire to cover our approach (and give credit where appropriate) and a need to keep the article readable and relevant to the greater proportion of their readers.

I’d like to share our methodology with you all so that you can repeat our experiments, should you so wish. After that, I’ll talk about the methodology that we were originally going to follow,

Tomorrow (once it’s had a chance to blow over), I’ll post some quick thoughts on the whole storm-in-a-Tweetcup thing.

Methodology

We used Michael Litman’s (@litmanlive) list of UK Media Tweeple. This was based on original work by Stephen Davies (@stedavies) but has been wikified so that agencies can (should they so choose) keep their information up to date.

Lots of people on the list were pretty borderline — there are in-house teams and vendors there, as well as agencies with a significantly broader remit than simply “PR”. I am a relative newcomer to the world of PR, and was more than happy to let PR Week define who is PR and who isn’t, but we erred on the generous side. We are Social, for example, made the cut to be on the research list.

Had we had the time, I’d have sent a note out over Twitter asking everyone to update their entries. Time, however was not on our side, and I didn’t even get around to hinting at what I was doing until the evening of the 23rd.
PR Week Twitter Stats Yahoo! Pipe
By then though it was already clear that I had a large job on my hands; there were almost 350 people on the list. On the whole, the UK PR community should be proud of how quickly it has reacted to the whole “Twitter thing”.

I took the list, published it as a Google Spreadsheet and — using a Yahoo! Pipe that I adapted for the purpose, queried the Twitter API for the summary data on each account on that list.

Twitter gives you all sorts of interesting information, but what we were grabbed were the following:

  • Date joined Twitter
  • Number of Friends
  • Number of Followers, and
  • Number of Updates.

That allowed us to create this spreadsheet, from which the stats mentioned in the PR Week article were taken.

Again, Porter Novelli took no part in the editorial decisions (although they seem pretty straightforward.) You will recall that Peter and Gemma were writing for a general readership, not for the Twitterverse!

Methodology we’d like to have used

Those of you who’ve read my blog before will know that my real interest in Twitter is more complex than the previous methodology would suggest. When Peter and I first discussed the exercise on Monday we had been hoping to do something more along the lines of the network analysis that we’ve been fiddling with at Porter Novelli.

Here are some points to bear in mind.

First of all, not all followers are created equal. If I have only ten followers, but they each have a thousand followers, that may mean I have more opportunity-to-influence than if I had a hundred followers with only ten followers each.

More to the point, the fewer people those ten people follow themselves, the more influence I wield within their networks (if I am one of only ten people they follow between them, I will have greater share-of-voice than if I am merely one of ten thousand.)

Secondly, the followers whom I don’t share with the rest of the network count for more than those who follow several (or many) of my peers. The more “exclusive” my follower-base, the greater my control over on the flow of information within the overall network, and the greater my value to the network.

I’ve been doing some work looking at unduplicated reach among twitter networks. For example, looking at Porter Novelli’s own global Twitter footprint, it was interesting to see how many of our contacts were duplicated.

So what Peter and I really wanted to do was to use some of these techniques on the PR Week data set. For those of you with a mathematical (or social network analytical) bent, we were going to run some eigenvector shizzle on the whole bizzle. Oh — and look at unduplicated reach for the various companies on the list.

What went wrong?

It was always an ambitious project. The 344 people who were under analysis had a fairly daunting 95K followers between them. The Twitter API lets you make 100 requests an hour, and each request returns data on up to 100 followers. Even if we were to assume that everyone had followers in nice tidy multiples of 100 (they don’t) then it would have taken 9.5 hours to download the data using one Twitter login.

The trick of course, is to use more than one login. Tim Hoang (@timhoang) and I quickly registered 50 temporary accounts to power the API requests. Twitter’s terms have historically been quite relaxed about this sort of thing, and we’ve always been very careful to try and stay within the spirit of those terms.

But.

Twitter has been hit lately by a bunch of bad things (like spam bots and pyramid schemes), and they’re tightening up their defenses. This past weekend, they’ve tightened up a lot, and things that used to be fine just aren’t.

We managed to collect information on only around 60K followers out of the 95K. This was too large a margin of error to correct (although we made several attempts to do so).

So — we had to abandon our grand plans, and revert to the simple counts approach (as detailed above.) This won’t stop us trying to improve our processes, but we’ll need to talk to Twitter about that.

Some thoughts

Kate Hartley from Carrot Communications (who sits with me on the PRCA’s Digital Working Group) joked that it’s strange how PR people create research-for-news-stories for their own clients on a daily basis, but are miffed when their own techniques are used against them. At one level, I agree with her — I think that some people are probably disappointed that they aren’t the ones with their names on the research.

But there’s more to worry about than that. Here are my thoughts.

  1. For God’s sake get over yourselves! We’re talking about Twitter here, not the economy. Worry about something important, why don’t you? I still can’t get over the fact that — when a pilot managed land an airplane on a river, the story we all tell each other is “how it broke on Twitter.” What — the story’s not about a man who magically landed a f*cking plane on a f*cking river? Are we really getting this right?
  2. How influential you are on Twitter is not a real thing. It doesn’t really matter how many Twitter friends you have (although I’ve now got heaps, thank you very much!) Context is everything. My boss, who runs Porter Novelli’s EMEA network and sits on our Executive Committee is on Twitter. She is more influential than I, and will continue to be, no matter how many Twitter followers I accrue.

    Twitter is just one channel through which exercise your influence. Don’t give up on your blogs, your Facebook pages, your Amazon reviews, or your Last.fm playlists or your IM friend lists, for God’s sake. But remember, it’s who you are, and your relationships that matter; your “context”, and not your “counts.”

  3. The really interesting question isn’t “who are the Twitterati” or twitter influencers. I’m interested in the Twitter thing mainly because I want to see how well it reflects real life. After today, I’d probably say that it doesn’t very well, wouldn’t you?

Be warned — I may just follow this research up with some research on “how many phone numbers PR people have on their mobile phones.”