Comments on: Automating Marshall Kirkpatrick’s “Social Media Cheatsheet” process with Yahoo! Pipes

By: Subject guides on web 2.0 startup pages - 12 widgets | Musings about librarianship

Thu, 30 Apr 2009 01:16:30 +0000

[...] Automating Marshall Kirkpatrick’s “Social Media Cheatsheet” process with Yahoo! Pi… [...]

By: Ray Scott

Ray Scott — Sat, 17 Jan 2009 15:16:37 +0000

Wow! Thank you so much for kindly sharing this pipe with us! You’re definitely more advanced with your Pipes skills than I.

nice!

By: Howard Rheingold

Howard Rheingold — Thu, 15 Jan 2009 22:14:52 +0000

@Mat — knowing how to “use Pipes to handle RSS-related needs” would be incredibly useful to journalists. I teach journalism master’s students at Stanford who go on to get media jobs.I introduce them to RSS and to monitoring news searches via RSS. I’m showing them Marshall’s article and hoping that the most technically motivated among them will show the rest of us how it’s done. But if you have any idea how to get students to make something useful for themselves, using Pipes and RSS, specifically in regard to journalism — I’m interested in hearing about it.

By: Mat Morrison

Mat Morrison — Mon, 12 Jan 2009 12:21:35 +0000

@Jean-Marc: increasingly I use Pipes to handle RSS-related needs (a combination of Pipes and NetVibes helps us produce low-cost/high-value monitoring dashboards.) Another benefit is that it tends to scare people less than other scripting methods. But when I started using it, it was a way of prototyping the perl scripts that I’d eventually write to automate stuff (Pipes uses, among other things, the same regex structure.) Rather than sketching out my flow chart on paper, I’d build a flow chart that could do it all for me.

By: Jean-Marc Liotier

Jean-Marc Liotier — Mon, 12 Jan 2009 09:58:41 +0000

Quite strange – Your results look perfectly relevant. I tried again (different workstation, different network, different OS, different browser…) and I get different and mostly irrelevant results with much Hungarian references for the “senegal” query and other strange bias for others. Maybe it is a technical problem, or maybe you are right and editorial input should be required to refine the process. Maybe I’ll later look at the intermediate steps to understand what is going on with my tests. And that’ll be a good excuse to get familiar with pipes – although I prefer implementation in a script language I can run locally instead of relying on a third party…

By: Mat Morrison

Mat Morrison — Mon, 12 Jan 2009 00:24:54 +0000

@Jean-Marc — thanks for the kind words. As I say, there are two important elements of Marshall’s workflow missing here; the human editorial eye (to a certain extent this is now pushed to the end of the process rather than the beginning) and the “popularity” ranking.

However I’m unable to reproduce your results: my first few stories are:

- The Harmattan is blowing in (postrank 6.7)
- Why I blog about Africa (postrank 5.6)
- Kédougou Assiégée (postrank 6.0)
- Malika Monkeys (postrank 8.7)

(all of which seem relevant.)

You shouldn’t expect to see results from del.icio.us on the pipe output. Marshall approach is to use del.icio.us as a means of identifying “relevant blogs” – the next stages (finding the RSS feed for each blog and then passing that through AideRSS’s PostRank to find the ‘top posts’ on those blogs) will always change the seed results beyond recognition.

Hope this makes sense. Will be pleased to take you through the pipe!

By: Jean-Marc Liotier

Jean-Marc Liotier — Sun, 11 Jan 2009 21:47:55 +0000

Nice pipe work – it opens my eyes about what sort of useful tools we can build with pipes. But I have tried it with various keywords and I have obtained very strange results. For example, the first four links returned using the keyword “senegal” are : “The Italian Invasion”, “An appeal to Italy”, “Classic Hungarian Rock, pre-crackdown” and “Magyar memories”… I have looked at Del.ico.us and it is not what we get there – so there is something wrong with the pipe. But automating Marshall’s process is a grand idea !

By: thegeniusfiles

thegeniusfiles — Sun, 11 Jan 2009 20:09:58 +0000

Very nice. I too began to think about Pipes as I was reading Marshall’s post. I will take a look at what you have done; thanks for the link! I blogged about the baby steps I have taken with Pipes in the post
http://www.thegeniusfiles.com/2008/11/putting-thegeniusfiles-through-yahoo.html
but it looks like you have a deeper understanding of how this intriguing service works. I don’t think very many people know about Pipes yet, but it deserves more attention. I’m adding your post to my bookmarks!