Yahoo! Pipe for automating Marshall Kirkpatrick’s Social Media CheatSheet process
any way to ramp up your knowledge of these fields, fast, other than the “Google and wander” method?
He then outlines an almost perfect example of how to use social media to do this.
You should read his article before reading any further. It’s short and punchy and won’t take much time.
Read it? Good. Now you may have noticed in the comments section that the first commenter doubts that you can:
find one baker or candlestick maker that will go through all of that.
So I thought I’d see if I can automate the process. The short answer is that I can and I can’t. I can’t yet automate one or two really important bits and pieces, notably:
- ranking delicious bookmarks by popularity, not recency
- human editorial selection of bookmarks
Perhaps someone could help me with this.
But otherwise, I’ve published this Yahoo! Pipe, Automating Marshall Kirkpatrick’s Social Media Cheatsheet Process which automates 90% of the process, and may make it easier for the bakers and candlestickmakers.
All comments and — more importantly — suggestions and improvements gratefully received.
Monday, 12 Jan 2009 00:27: I’ve just added a bit to the pipe to list posts in descending order according to PostRank. Don’t know if this is useful
Very nice. I too began to think about Pipes as I was reading Marshall’s post. I will take a look at what you have done; thanks for the link! I blogged about the baby steps I have taken with Pipes in the post
http://www.thegeniusfiles.com/2008/11/putting-thegeniusfiles-through-yahoo.html
but it looks like you have a deeper understanding of how this intriguing service works. I don’t think very many people know about Pipes yet, but it deserves more attention. I’m adding your post to my bookmarks!
Nice pipe work – it opens my eyes about what sort of useful tools we can build with pipes. But I have tried it with various keywords and I have obtained very strange results. For example, the first four links returned using the keyword “senegal” are : “The Italian Invasion”, “An appeal to Italy”, “Classic Hungarian Rock, pre-crackdown” and “Magyar memories”… I have looked at Del.ico.us and it is not what we get there – so there is something wrong with the pipe. But automating Marshall’s process is a grand idea !
@Jean-Marc — thanks for the kind words. As I say, there are two important elements of Marshall’s workflow missing here; the human editorial eye (to a certain extent this is now pushed to the end of the process rather than the beginning) and the “popularity” ranking.
However I’m unable to reproduce your results: my first few stories are:
- The Harmattan is blowing in (postrank 6.7)
- Why I blog about Africa (postrank 5.6)
- Kédougou Assiégée (postrank 6.0)
- Malika Monkeys (postrank 8.7)
(all of which seem relevant.)
You shouldn’t expect to see results from del.icio.us on the pipe output. Marshall approach is to use del.icio.us as a means of identifying “relevant blogs” – the next stages (finding the RSS feed for each blog and then passing that through AideRSS’s PostRank to find the ‘top posts’ on those blogs) will always change the seed results beyond recognition.
Hope this makes sense. Will be pleased to take you through the pipe!
Quite strange – Your results look perfectly relevant. I tried again (different workstation, different network, different OS, different browser…) and I get different and mostly irrelevant results with much Hungarian references for the “senegal” query and other strange bias for others. Maybe it is a technical problem, or maybe you are right and editorial input should be required to refine the process. Maybe I’ll later look at the intermediate steps to understand what is going on with my tests. And that’ll be a good excuse to get familiar with pipes – although I prefer implementation in a script language I can run locally instead of relying on a third party…
@Jean-Marc: increasingly I use Pipes to handle RSS-related needs (a combination of Pipes and NetVibes helps us produce low-cost/high-value monitoring dashboards.) Another benefit is that it tends to scare people less than other scripting methods. But when I started using it, it was a way of prototyping the perl scripts that I’d eventually write to automate stuff (Pipes uses, among other things, the same regex structure.) Rather than sketching out my flow chart on paper, I’d build a flow chart that could do it all for me.
@Mat — knowing how to “use Pipes to handle RSS-related needs” would be incredibly useful to journalists. I teach journalism master’s students at Stanford who go on to get media jobs.I introduce them to RSS and to monitoring news searches via RSS. I’m showing them Marshall’s article and hoping that the most technically motivated among them will show the rest of us how it’s done. But if you have any idea how to get students to make something useful for themselves, using Pipes and RSS, specifically in regard to journalism — I’m interested in hearing about it.
Wow! Thank you so much for kindly sharing this pipe with us! You’re definitely more advanced with your Pipes skills than I.
nice!