Recently was working on a project to combine RSS feeds from multiple sources and display the latest news. To combine the RSS feeds i used Yahoo Pipes web service as it was very easy to configure, maintain and for scalability. Unfortunately the combined feed was displaying only top 100 posts, whereas my combined feed was more than 100 ! After a head scratching search in the internet, i got to find a solution at a Blog.
To my frustration, i found Yahoo Pipes web service has added a pagination to its feeds in blocks of 100. So to navigate to the next 100 posts,the feed url should be of like below with parameter “page=2” at the end :
http://pipes.yahoo.com/pipes/pipe.run?_id=<pipe_id>&_render=rss&page=2The frustration gets added, if your feed count is actually 500, then you need to navigate 5 times to see the complete nodes of your feed !
This is really Bad ! But is there any alternative ?
Yes. Yahoo does not paginate its JSON object. Rather it gives you the entire feed in the JSON Object format. You can then parse the JSON Object using javascript or simply using jQuery built-in functions.
http://pipes.yahoo.com/pipes/pipe.run?_id=<pipe_id>&_render=json