Christian Heilmann

Posts Tagged ‘flickr’

A research interface for the social web – fork it now and find what people are talking about

Wednesday, September 22nd, 2010

Researching something on the web can be pretty annoying. Search engines get better every year, but there is a whole world of social sites that are not indexed. For example if I search for a nice photo of a red panda I use Google image search. If I want to use this photo later on I am better off using Flickr or Picasa and see what license the photo is.

Yahoo’s researchers had the same problem which is why they assembled all the social updates in one XML feed – the Yahoo! Firehose. This, in contrast to other Yahoo APIs also comes with commercial terms and conditions and is available through YQL. In terms of data, the Firehose aggregates a lot of different sources:

Yahoo! 360, AOL, Bebo, Blogger, Bloglines, Digg, Diigo, Goodreads, Google, Google Reader,, Ma.gnolia, Movable Type, Netflix, Pandora, Picasa, Pownce, Seesmic, Slideshare, SmugMug, StumbleUpon, ThisNext, TravelPod, Tumblr, Twitter, TypePad, Vimeo, Vox, Webshots, Xanga, Yelp, YouTube, Zooomr, Yahoo! Avatars, Yahoo! Buzz, Yahoo! Profiles, Wisteria, Yahoo! Answers, Yahoo! Shopping, Yahoo! Autos, Bix for Yahoo!, Yahoo! Bookmarks, Yahoo! Briefcase, Yahoo! Calendar, Yahoo! Classifieds, Delicious, Yahoo! Family, Yahoo! Sports, Yahoo! Finance, Flickr, Yahoo! Food, Yahoo! Games, Yahoo! Geocities, Yahoo! Green, Yahoo! Greetings, Yahoo! Groups, Yahoo! Health, Yahoo! Hotjobs, Yahoo! Kids, Yahoo! Local, Yahoo! Movies, Yahoo! Music, MyBlogLog, Yahoo! News, OMG! from Yahoo!, Yahoo! Personals, Yahoo! Pets, Yahoo! Status Updates, Yahoo! Guestbook Comments, SearchMonkey from Yahoo!, Yahoo! Shopping, Yahoo! Sports, Yahoo! Tech, Yahoo! Travel, Yahoo! TV, Yahoo! Video.

You can do the data junkie part and use it in the YQL console:

This can be annoying though, especially as you cannot see the photos and videos. This is why I put together a research interface on top of the Yahoo Firehose:

You can see the research interface in action here but more importantly, the source code of the interface is available on GitHub which means that you can host it yourself – for example behind a firewall or make it part of your Intranet.

For a local install you need to sign up for a developer key, edit the keys.php file, put all the files up on your PHP enabled server and you are done. If you get stuck you can get help on the YDN Forums.

Notice that I am keeping the state of your last search by storing it in local storage when your browser supports it – this can be useful for larger searches.

TTMMHTM: live, CrisisCamp, Pixel stuff and Windows 95 explained!

Friday, January 22nd, 2010

Things that made me happy this morning:

  • The release yesterday was a great event. UK House Prices got applauded, I sat next to Tim Berners-Lee and I managed to pester people from the government to have me come around and showcase YQL!
  • Gource is a very sexy visualisation of Git commits, forks and so on.
  • This Saturday Yahoo will host CrisisCamp Haiti
  • Metal Slug sprites is a collection of all the sprites in all their pixely goodness. Me having spent a lot of time pixeling stuff (notice the browser support info – this is how old this is), I am impressed.
  • Jennifer Aniston and Matthew Perry show us how Windows 95 works.”Look Matthew I’m computing!”
  • Want to buy a jumbo jet? Comes with slight water damage – yes, this is the Hudson River one.
  • Tickery shows you which other people two people on Twitter follow.

Getting a list of Flickr photos by location and/or search term with a YQL open table

Monday, November 2nd, 2009

Displaying photos from Flickr can be daunting. The API needs authentication and the RSS, JSON or LOLcode output is very limited. The way around is using YQL and its Flickr tables. That way it is pretty easy to search Flickr:

select * from where text=”panda”

Try out the Panda search in the YQL console.

The output format has a lot of information in there, but sadly enough, not all. For example the real name of the owner or the description is missing. Therefore you need to go through yet another Flickr API to get the whole data set:

select * from where photo_id in(
select id from where text=”panda”

Guess what? You can also try this more detailed query in the console.

I’ve shown before how easy it is to display Flickr Photos retrieved that way:

The issue with that though is that it uses JavaScript and JavaScript may be turned off (think Blackberries). Of course you can do the same thing in PHP but I’d wager to say more people to JavaScript than PHP these days.

The main issue is that Flickr returns the photos in a pretty weird format and that you need a script like the one above to turn it into a simple HTML list.

The good news is that YQL with the execute command allows you to embed JavaScript in your open tables. That way you can write a table that does all the necessary transformations and returns the data as a simple list for immediate use:

select * from {table} where location=”london,UK”
Christian Heilmann
Searches Flickr by location and/or search term and returns an HTML list that can be immediately used in a mashup.

You’ll notice that while the E4X support is very powerful, it can be a bit confusing to look at on first sight. Once you got your head around though it becomes much cleaner that way.

You can use this table like any other open table via the use command in YQL:

use “” as flickr;
select * from flickr where text=”me” and location=”uk” and amount=20

try it in the console.

I’ve wrapped one more API in there – the Yahoo Geo API to determine a place from a name should you want to search by location. All in all you have three parameters in this open table – all of which are optional:

  • text – the search text
  • location – the geographical location
  • amount – the amount of photos to be returned

If you look at the table source, you can also see that I hard-wired the license of the photos to 4 which is CC-BY. So if you link the photos back to Flickr you both satisfied Flickr’s terms and the original photographer’s.

Now, the easiest way to use this output is by using YQL’s JSON-P-X output format. This is XML with a callback which returns a JSON object with the HTML as a string instead of a convoluted JSON object. See the JSON-P-X output here.

That way you can easily use it in JavaScript:

And also in PHP:

$url = ‘;%20select%20*%20from%20flickr%20where%20text=%22me%22%20and%20location=%22uk%22%20and%20amount=20&format=xml&diagnostics=false’;
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$output = curl_exec($ch);
$output = preg_replace(‘/.*
      $output = preg_replace(‘/
$output = preg_replace(‘//’,’‘,$output);
$output = preg_replace(‘//’,’‘,$output);
echo $output;

You can see both in action on the demo page.

So by using YQL open tables you can not only access complex APIs with YQL, but you could also write complete mashups in JavaScript and have them executed in a safe environment on Yahoo’s servers. Your end of the mashup is simply the display which could be a form that works with Ajax when JavaScript is available and renders a static page in PHP (or whatever other server-side language) when JavaScript is turned off. You only need to do one HTTP request – the rest is executed and cached on the YQL server farm – everybody wins.

TTMMHTM: Obama and unicorns,3D reconstruction from Flickr photos,quick brown foxes,CSS colours and rotations and new apples

Wednesday, July 29th, 2009

Things that made me happy this morning:

TTMMHTM: Meeting ticker, iPhone JavaScript Geo API, Geekspeakr, Flickr uploadr replacement and when to pee during movies

Sunday, July 19th, 2009

Things that made me happy this morning: