Researching something on the web can be pretty annoying. Search engines get better every year, but there is a whole world of social sites that are not indexed. For example if I search for a nice photo of a red panda I use Google image search. If I want to use this photo later on I am better off using Flickr or Picasa and see what license the photo is.

Yahoo’s researchers had the same problem which is why they assembled all the social updates in one XML feed – the Yahoo! Firehose. This, in contrast to other Yahoo APIs also comes with commercial terms and conditions and is available through YQL. In terms of data, the Firehose aggregates a lot of different sources:

Yahoo! 360, AOL, Bebo, Blogger, Bloglines, Digg, Diigo, Goodreads, Google, Google Reader, Last.fm, Ma.gnolia, Movable Type, Netflix, Pandora, Picasa, Pownce, Seesmic, Slideshare, SmugMug, StumbleUpon, ThisNext, TravelPod, Tumblr, Twitter, TypePad, Vimeo, Vox, Webshots, Xanga, Yelp, YouTube, Zooomr, Yahoo! Avatars, Yahoo! Buzz, Yahoo! Profiles, Wisteria, Yahoo! Answers, Yahoo! Shopping, Yahoo! Autos, Bix for Yahoo!, Yahoo! Bookmarks, Yahoo! Briefcase, Yahoo! Calendar, Yahoo! Classifieds, Delicious, Yahoo! Family, Yahoo! Sports, Yahoo! Finance, Flickr, Yahoo! Food, Yahoo! Games, Yahoo! Geocities, Yahoo! Green, Yahoo! Greetings, Yahoo! Groups, Yahoo! Health, Yahoo! Hotjobs, Yahoo! Kids, Yahoo! Local, Yahoo! Movies, Yahoo! Music, MyBlogLog, Yahoo! News, OMG! from Yahoo!, Yahoo! Personals, Yahoo! Pets, Yahoo! Status Updates, Yahoo! Guestbook Comments, SearchMonkey from Yahoo!, Yahoo! Shopping, Yahoo! Sports, Yahoo! Tech, Yahoo! Travel, Yahoo! TV, Yahoo! Video.

You can do the data junkie part and use it in the YQL console:

This can be annoying though, especially as you cannot see the photos and videos. This is why I put together a research interface on top of the Yahoo Firehose:

You can see the research interface in action here but more importantly, the source code of the interface is available on GitHub which means that you can host it yourself – for example behind a firewall or make it part of your Intranet.

For a local install you need to sign up for a developer key, edit the keys.php file, put all the files up on your PHP enabled server and you are done. If you get stuck you can get help on the YDN Forums.

Notice that I am keeping the state of your last search by storing it in local storage when your browser supports it – this can be useful for larger searches.

Last week I was in Paris for a Yahoo Developer Network evening and Paris Web and one of the talks I gave was an introduction to Yahoo Open Applications. These are applications that you can embed in the Yahoo homepage or My Yahoo and thus allow you to reach millions of users – or extend the Yahoo homepage with your own personal app. Here are the slides and the audio recording of the talk delivered by Sophie Davies-Patrick (aka “my boss”) and me at La Cantine in Paris:

• Audio of Introduction to Yahoo Open Applications – 33MB MP3 (other audio options on archive.org)

Yahoo Open Applications use the Yahoo Application Platform – YAP. In essence, you write a small app using CSS, JavaScript and HTML and it will get embedded into the Yahoo Homepage.

TweetTrans – translate Twitter updates

The example I showed was a tool that adds little translation links to a Twitter stream:

The code of TweetTrans is available on GitHub and I’ve built it initially as a bog-standard Ajax web application.

The main step afterwards was to convert the app over. This is less hard than it seems upfront but the fact that YAP uses Caja to make the whole application more secure means you need to restrict yourself. Things you cannot use are:

Caja and HTML

• Custom attributes
• Custom tags
• Unclosed tags
• EMBED
• IFRAME
• javascript:void(0)
• Radio buttons in IE
• Relative URLs

Caja and CSS

• star hacks
• _ hacks
• IE conditionals
• Insert-after clear fix
• expression()
• @import
• Background images in IE

Caja and JavaScript

• eval()
• new Function()
• Strings as event handlers (node.onclick = ‘...’;)
• Names ending with double / triple underscores
• with function (with (obj) { ... })
• Implicit global variables (specify var variable)
• Calling a method as a function
• document.write
• window.event
• OpenSocial gadgets.io.makeRequest return JS

Get the SDK

The easiest way to build Yahoo Open Apps is to use YML which is a markup language that gives you access to the Yahoo social connections and creates Ajax functionality for you without having to write any JavaScript.

If you want to use the full Yahoo social stack the best place to start is to download the SDK which makes the oAuth authentication very easy for you.

Distribution

Once you have your app done you can easily make people install it by sending them a link. This link contains the application ID. In the case of TweetTrans this is:

http://yahoo.com/add?yapid=zKMBH94k

More info

To get more information, check out the YOS section on the Yahoo Developer Network and the forums on YAP.