Author Archive
« Older Entries Newer Entries »Methodology and thoughts behind those PR Week Twitter stats
Thursday, February 26th, 2009
There’s a school of thought that says that what’s important in social media is to attempt to create debate, not consensus.
Peter Hay from PR Week and I appear to have been rather successful in this. This morning, PR Week published an article, Twitter has suddenly exploded. Almost immediately, Twitter (or at least our particular neighbourhood of Twitter) suddenly exploded.
One or two people were rather scathing: suggesting that the stats demonstrated that Peter and I didn’t understand the “essence of Twitter” or that they were “obviously flawed”, or that we had “redefined shallow”.
Indeed (horror of horrors) some people even went so far to suggest that Porter Novelli had ginned up the results to put us at the top. In fact, in PR week’s list, we came second. But no doubt this was a Machiavellian ploy — it’s details like those, Pooh Bah would say, that “give artistic verisimilitude to an otherwise bald and unconvincing narrative.”
I joke, but I can completely understand people’s strong feelings about this; PR Week was torn between a desire to cover our approach (and give credit where appropriate) and a need to keep the article readable and relevant to the greater proportion of their readers.
I’d like to share our methodology with you all so that you can repeat our experiments, should you so wish. After that, I’ll talk about the methodology that we were originally going to follow,
Tomorrow (once it’s had a chance to blow over), I’ll post some quick thoughts on the whole storm-in-a-Tweetcup thing.
Methodology
We used Michael Litman’s (@litmanlive) list of UK Media Tweeple. This was based on original work by Stephen Davies (@stedavies) but has been wikified so that agencies can (should they so choose) keep their information up to date.
Lots of people on the list were pretty borderline — there are in-house teams and vendors there, as well as agencies with a significantly broader remit than simply “PR”. I am a relative newcomer to the world of PR, and was more than happy to let PR Week define who is PR and who isn’t, but we erred on the generous side. We are Social, for example, made the cut to be on the research list.
Had we had the time, I’d have sent a note out over Twitter asking everyone to update their entries. Time, however was not on our side, and I didn’t even get around to hinting at what I was doing until the evening of the 23rd.

By then though it was already clear that I had a large job on my hands; there were almost 350 people on the list. On the whole, the UK PR community should be proud of how quickly it has reacted to the whole “Twitter thing”.
I took the list, published it as a Google Spreadsheet and — using a Yahoo! Pipe that I adapted for the purpose, queried the Twitter API for the summary data on each account on that list.
Twitter gives you all sorts of interesting information, but what we were grabbed were the following:
- Date joined Twitter
- Number of Friends
- Number of Followers, and
- Number of Updates.
That allowed us to create this spreadsheet, from which the stats mentioned in the PR Week article were taken.
Again, Porter Novelli took no part in the editorial decisions (although they seem pretty straightforward.) You will recall that Peter and Gemma were writing for a general readership, not for the Twitterverse!
Methodology we’d like to have used
Those of you who’ve read my blog before will know that my real interest in Twitter is more complex than the previous methodology would suggest. When Peter and I first discussed the exercise on Monday we had been hoping to do something more along the lines of the network analysis that we’ve been fiddling with at Porter Novelli.
Here are some points to bear in mind.
First of all, not all followers are created equal. If I have only ten followers, but they each have a thousand followers, that may mean I have more opportunity-to-influence than if I had a hundred followers with only ten followers each.
More to the point, the fewer people those ten people follow themselves, the more influence I wield within their networks (if I am one of only ten people they follow between them, I will have greater share-of-voice than if I am merely one of ten thousand.)
Secondly, the followers whom I don’t share with the rest of the network count for more than those who follow several (or many) of my peers. The more “exclusive” my follower-base, the greater my control over on the flow of information within the overall network, and the greater my value to the network.
I’ve been doing some work looking at unduplicated reach among twitter networks. For example, looking at Porter Novelli’s own global Twitter footprint, it was interesting to see how many of our contacts were duplicated.
So what Peter and I really wanted to do was to use some of these techniques on the PR Week data set. For those of you with a mathematical (or social network analytical) bent, we were going to run some eigenvector shizzle on the whole bizzle. Oh — and look at unduplicated reach for the various companies on the list.
What went wrong?
It was always an ambitious project. The 344 people who were under analysis had a fairly daunting 95K followers between them. The Twitter API lets you make 100 requests an hour, and each request returns data on up to 100 followers. Even if we were to assume that everyone had followers in nice tidy multiples of 100 (they don’t) then it would have taken 9.5 hours to download the data using one Twitter login.
The trick of course, is to use more than one login. Tim Hoang (@timhoang) and I quickly registered 50 temporary accounts to power the API requests. Twitter’s terms have historically been quite relaxed about this sort of thing, and we’ve always been very careful to try and stay within the spirit of those terms.
But.
Twitter has been hit lately by a bunch of bad things (like spam bots and pyramid schemes), and they’re tightening up their defenses. This past weekend, they’ve tightened up a lot, and things that used to be fine just aren’t.
We managed to collect information on only around 60K followers out of the 95K. This was too large a margin of error to correct (although we made several attempts to do so).
So — we had to abandon our grand plans, and revert to the simple counts approach (as detailed above.) This won’t stop us trying to improve our processes, but we’ll need to talk to Twitter about that.
Some thoughts
Kate Hartley from Carrot Communications (who sits with me on the PRCA’s Digital Working Group) joked that it’s strange how PR people create research-for-news-stories for their own clients on a daily basis, but are miffed when their own techniques are used against them. At one level, I agree with her — I think that some people are probably disappointed that they aren’t the ones with their names on the research.
But there’s more to worry about than that. Here are my thoughts.
- For God’s sake get over yourselves! We’re talking about Twitter here, not the economy. Worry about something important, why don’t you? I still can’t get over the fact that — when a pilot managed land an airplane on a river, the story we all tell each other is “how it broke on Twitter.” What — the story’s not about a man who magically landed a f*cking plane on a f*cking river? Are we really getting this right?
- How influential you are on Twitter is not a real thing. It doesn’t really matter how many Twitter friends you have (although I’ve now got heaps, thank you very much!) Context is everything. My boss, who runs Porter Novelli’s EMEA network and sits on our Executive Committee is on Twitter. She is more influential than I, and will continue to be, no matter how many Twitter followers I accrue.
Twitter is just one channel through which exercise your influence. Don’t give up on your blogs, your Facebook pages, your Amazon reviews, or your Last.fm playlists or your IM friend lists, for God’s sake. But remember, it’s who you are, and your relationships that matter; your “context”, and not your “counts.”
- The really interesting question isn’t “who are the Twitterati” or twitter influencers. I’m interested in the Twitter thing mainly because I want to see how well it reflects real life. After today, I’d probably say that it doesn’t very well, wouldn’t you?
Be warned — I may just follow this research up with some research on “how many phone numbers PR people have on their mobile phones.”
Tags: pr week
Posted in pipes, porter novelli, twitter | 20 Comments »
Swedish Politicians on Twitter
Sunday, February 22nd, 2009
Twixdagen does for Swedish politics what Tweetminster does for British. Hampus Brynolf (@hampusbrynolf) just sent me a link to this map he’s pulled together for their blog:
You’ll need to click through to his blog post to experience and interact with the map properly.
Hampus says that he used aiSee to generate an SVG file which could then be opened in Illustrator to “search and replace” on shapes, colors and lines (which explains the good-looking graph.)
Tags: aisee, mapping, network analysis, networks, politics, sweden
Posted in networks, twitter | 2 Comments »
If you’re going to follow one Twitter person…
Saturday, February 14th, 2009
Can I please suggest that — if you’re looking for fresh new tweeple to follow — that you kindly consider @BriggySmalls?
Thank you for your attention. That is all.
Posted in Uncategorized | 1 Comment »
Can we calculate party affiliation? (the US Congress Edition)
Friday, February 13th, 2009
Using nothing more than their public twitter relationships, is it possible to predict whether a US Congressperson is a Republican or a Democrat? The answer seems to be a guarded “yes” — our tools predict correctly 40/46 times (or around 87% of the cases.)
This post follows on from a post earlier today in which I asked, “can we calculate party affiliation?” The data set in the earlier post was gathered from the 16 members of the UK parliament who are on Twitter and the relationships between them.
Tweetcongress maintains a list of US congresspeople on Twitter. Today (February 13, 2009) there are 76 congresspeople on the service, but when I collected my data set of “who follows who” on February 3, 2009 there were only 65. Of these 65, fully 19 (29%) lived a life of noble isolation with regards the network — none of their peers linked to them, and they in turn linked to none of their peers. Removing these Miss Havishams from the data set leaves me with 46 twittering congresspeople who form a network.
Now as both social network analysis and Aesop would have it, “a man is known by the company he keeps.” What I mean by this is that given the partisan nature of politics, we should expect that Democrats will link to other Democrat twitterers more often than they link to Republican twitterers and vice versa. So that’s what NetDraw[1] , the software I’m using for most of this stuff, looks for, or more accurately:
To identify factions, NetDraw software iteratively searches for a distribution of nodes among a selected number of factions to minimise the number of connections between factions and to maximize the number of connections within factions.
Whatever. So I let NetDraw loose on the data, and here’s what it did.
I coloured the nodes red for Republican and blue for Democrats[2], labeled the nodes by party (for the sake of clarity, and for the hard-of-thinking, that’s “R” for Republican and “D” for Democrat) then counted all the nodes where label said one thing but colour another. There were six of these nodes; so NetDraw got the answer right 40⁄46 of the time (just about 87%.) This is less than the astonishing 93.75% accuracy we got with the Westminster twittering members of parliament in the previous post. Nevertheless I think we can safely say that it’s not a particularly integrated (or bipartisan) network if we can predict party affiliation with quite such success.
Here’s exactly the same map with the errant sheep re-labeled with their proper names so it’ll be easier to refer to them (if it helps, you can click on the image to view or download a larger version.)
You’ll see, I hope, that NetDraw has made a pretty good fist of the job. Where it has gone wrong on the whole is where the data clearly suggests something else. So Rep. Jared Polis for instance follows (and is followed by) no Democrat peers. Rep. Nancy Pelosi (D) and Sen. Richard Durbin (D) follow each other, but since Pelosi is followed by several Republicans and none of her other Democrat peers you can see why the algorithm has made the incorrect guess that the two of them are Republicans. Long-serving member Neil Abercrombie, as discussed in a previous post on US Congress Twitter folk, forms a bit of a bridge between the two parties, so despite his membership of the Congressional Progressive Caucus and liberal voting record, from the Twitter network point of view, his affiliation is somewhat ambiguous.
Sen. McCain follows none of his peers, and appears to inherit his incorrect attribution from Sen. Susan Collins. For the life of me, I can’t work out what makes it think that Sen. Susan Collins is a Democrat. She really isn’t, you know.
Note 1: NetDraw is a free program written by Steve Borgatti from the University of Kentucky. If you’re interested in playing around with this stuff, you’ll need to get yourself a copy.
Note 2: Actually, that’s not true. Despite a friend sharing the simple mnemonic that “‘Republicans’ and ‘red’ begin with the same letter,” I just can’t get it out of my English head that the Republicans should be blue and the Democrats red. As a result I waste precious minutes re-colouring these maps in Illustrator. It is worth pointing out that I also have problems with “left” and “right” on occasion — preferring instead the binary opposition “left” and “No! no! The other left, for God’s sake!”
Tags: congress, democrat, gop, jared polis, john mccain, mapping, nancy pelosi, Neil Abercrombie, network analysis, networks, republican, research, richard durbin, susan collins, twitter
Posted in networks, research, twitter | 1 Comment »
Creating blog seed lists for research
Tuesday, February 10th, 2009
Colleagues and regular readers will know that we’ve been working on an “online influencer mapping” tool called Rufus. Those of you who’ve had a chance to use Rufus will know that it requires a seed list of URLs to get started. Creating this seed list can be automated in one or two ways, but one of the fastest, most effective, and most sensible ways to build a seed list is still to do it by hand.
We’ve got one or two other processes that also require us to build a seed list. No doubt other people do too — lots of web research is quite data hungry. So — because I’ve found myself telling a few of my Porter Novelli colleagues how we go about the process, I thought I’d share it here, in the interests of:
- having somewhere to point people in future,
- general good-heartedness: I’ve learned a lot from people in the past, and I like to give stuff back, and
- getting feedback and tips from people about how they might go about the same process.
Oh – and while these methods should work in any language, please bear in mind that I tend to think and work in English. I’d appreciate feedback on how best to localize these methods.
Building a seed list: 5 easy methods
With all these methods, there’s no substitute for checking out the blog. I don’t ask people to read the blog (that comes at a different stage of the process altogether) but you should at least click through and see what you’re dealing with. In fact, method 3 rather relies on you visiting the blogs you’re researching.
1. Look for someone who has already done your research for you
Start by being optimistic. Generally you’ll find that someone else has created a list of the “top ten” (or however many) blogs in the niche that interests you. Take a look at Brendan’s regularly updated PR Friendly Index for example. If you’re searching for English language blogs then you could do worse than start by looking at Guy Kawasaki’s Alltop. But simply Googling for lists of blogs or blog charts should get you a long way.
This is generally a source of fairly high-quality data. One thing to watch out for, though, are search engine spamming link farms, and shady “Make Money Online” (MMO) directories. You’ll learn to recognize these soon enough, but as long as you’re visiting all the blogs you’re putting on your seed list you should be alright.
2. Do a tag search on delicious
I picked up this technique from Anthony Mayfield, who showed me that by searching on the delicious social bookmarking site for the tags “xxx” and “cool” and/or “inspiration” you could find sites about “xxx” that people thought were cool. Knowing what your digital trendsetters think is cool is one hell of an insight.
For our purposes though, we’re looking for cool blogs. So (1) click the “Explore Tags” tab on the home page, and then (2) type your keyword and the word “blog” into the search box. Couldn’t be simpler?
Well — actually it could be simpler. You can query the delicious database when you type the URL into the address bar of your browser like this:
http://delicious.com/tag/blog+keyword
Where “keyword” is the word you’re looking for.
When you get the results, check the ones that (a) have the right kind of title (if you’re looking for French blogs, look for French titles for example), (b) have the right kind of tags and description and (c) have been bookmarked most often
If there’s a better local language social bookmarking site, I’d use that whenever possible. For example, Mister Wong is a good one for German language sites.
A quick note: social news sites like Digg and Reddit, and “serendipity browsers” like StumbleUpon tend not to work so well in my experience.
This method also owes a lot to Marshall Kirkpatrick. You might like to try out the Yahoo! Pipe that I built based on the process that Marshall documents.
3. Look for blog rolls
On every blog you visit during the research process, look for the blog roll — and check the likely-looking links. See if they’re useful or useless. Quite often you’ll find that someone who has an interest in widgets will also read and link to blogs that cover widgets. That, after all, is the principle on which Rufus works wrote small. So we reckon it’s a pretty good approach.
4. Ask your Twitter followers
Seriously — this works. Well — it worked for me and my team from around +100 followers onwards. I’d be interested in others’ experience.
5. Call someone
Get hold of someone who knows about the subject and phone them up or get them on IM. Category experts are an excellent source of low-volume but high-quality information. It’s time consuming, but can work well if you have the right contacts. Journalist friends might be a great source of blog lists.
I’ve purposefully left this one till last; I think it’s a good rule of thumb to do your desk research before picking up the phone. That way you can ask intelligent questions instead of damn fool ones.
Using a text editor
I try to keep two lists running all the time that I’m working; a scratchpad list of blogs I have yet to visit and the seed list itself. Because I’m on a Mac, I use the excellent BBEdit (there’s a free version called TextWrangler which will be just as good for most people.) If — as is more probable — you’re on a Windows machine, you might like to try the very powerful but slightly less pretty Notepad++. But if you just want to use Excel, though, that’s fine, too.
Tags: bbedit, bloggers, blogs, delicious, digg, notepad++, reddit, research, seed list, textwrangler
Posted in research | 4 Comments »
Republicans vs. Democrats: Pareto charts of unduplicated Twitter reach
Sunday, February 8th, 2009
A couple of days ago I did a little more analysis on Republican and Democratic Congresspeople on Twitter.
Towards the end of the post, I realized that the unduplicated reach pareto chart that I’d built would only make sense if the US were a one-party state (or to be fair, if both parties had a single issue that they were united in wanting to promote.)
So — wanting to make this a little more representative — I went back and produced two charts; one showing Republican unduplicated reach (which follows a typical 80:20 distribution)…
Tags: congress
Posted in twitter | 3 Comments »
Republicans still outperforming Democrats on TweetCongress
Wednesday, February 4th, 2009
Three weeks ago (and at the prompting of my colleague Eddie Garrett who heads up Porter Novelli DC’s digital team) I mapped out the interconnections between US Congress Tweeters. We’d been working on a Twitter crawler and it seemed like a good opportunity to test things out on a new data set.
This is a follow-up post. Once again it was prompted by a third party: Christie Findlay at Politics Magazine asked whether it would be OK to print a copy of one of the maps in their March edition. I’ve heard that three weeks are a long time in politics, so I thought I’d better run the crawl again just in case. Also I’ve got a new crawler that uses the proper Twitter API (I can see some of your eyes glazing over you know. Just skip ahead when that happens.) I’d tried it out on the Porter Novelli data set, but welcomed a chance to try it on something more meaty.
So yesterday morning before work I ran the crawl. I use the excellent Tweet Congress as my source of information about which congress people are on Twitter.
(more…)
Tags: congress, mapping, network analysis, twitter, visualization
Posted in networks, twitter | 9 Comments »
Pareto Novelli — Some Q&As
Sunday, February 1st, 2009
A recent post about some Pareto analysis of the Porter Novelli Twitter sample , “Porter Novelli Twitter folk – the 80/20 rule”, stirred up a little bit of interest on Twitter — and made me think again about what I’m doing and why. Partly because those conversations were off-blog (and I’d like to capture the answers I gave somewhere more permanent) and partly because I’ve now had time to think of better answers I thought I’d set them down here.
First, a little background. This Q&A is the sixth post in an impromptu series about the Twitter people where I work (Porter Novelli, the international public relations agency.) By now you might think that I’d be tired of this stuff, but you’d have another think coming. Here’s a quick list to bring you up to date.
- Map of Porter Novelli people on Twitter on 17th Jan 2008
- Map of Porter Novelli people on Twitter on 20th Jan 2008
- Introducing the Porter Novelli magic Twitter friend maker (beta)
- Porter Novelli Twitter folk ranked by number of followers
- Porter Novelli Twitter folk – the 80/20 rule
Looking at this, you might also think I clearly had nothing better to do than analyze Porter Novelli people and their Twittering ways. In fact, as an experimental data set, I couldn’t really ask for anything much better. It’s sufficiently large (more than 200 people), international (I’ve counted more than 10 countries — and I’m sure there are more), and I have some real-world access to all of the people in the sample, which means I can compare my findings with some hard data.
That said, the experiment is more about learning about how we can analyze Twitter networks — about discovering how representative they are as a word-of-mouth (WOM) channel for example, and what they can tell us about other kinds of social network, or about finding new ways to analyze such data sets — than it is about answering any specific questions. So I’ve not got any carefully mapped-out research plan. Instead I follow paths that strike me as interesting, or possible, or that are suggested to me by friends and readers.
Question 1
Tags: pareto, porter novelli, twitter
Posted in porter novelli, twitter | 7 Comments »












