Christian Heilmann

Posts Tagged ‘microsoft’

Worldinfo – my Event Apart 10KB submission (information and documented source code)

Tuesday, August 17th, 2010

As you might know, an Event Apart in association with Microsoft are currently running a competition asking developers what they can do in under 10KB and I thought I have a shot at that.

So here’s my submission: an interface to get information about any country on this planet in under 5K:

I got the idea last Thursday during Pub Standards in London when someone asked me if it is possible to get information about all the countries in the world using YQL. The main interest was not only to get the names but also the bounding box information in order to display maps with the right zoom level. And it is, all you need to do in YQL is the following:

select name,boundingBox from geo.places.children(0) where
parent_woeid=1 and placetype=”country” | sort(field=”name”)

This gets all the children of the entry with the WOEID of 1 (Earth, that is) in the GeoPlanet dataset that are a country and sorts them alphabetically for you.

Each of the results comes with bounding box information which you then can use to display a map with the Open Streetmap static image API (or any other provider). For example:

http://pafciu17.dev.openstreetmap.org/?
module=map&bbox=38.804001,37.378052,
48.575699,29.103001&width=500&height=250

Or as I use it:

var image = ‘http://pafciu17.dev.openstreetmap.org/?module=map&bbox=’+
bb.boundingBox.southWest.longitude+’,’+
bb.boundingBox.northEast.latitude+’,’+
bb.boundingBox.northEast.longitude+’,’+
bb.boundingBox.southWest.latitude+
‘&width=500&height=250’;

The last piece to the puzzle was where to get country information from and of course the easiest is Wikipedia. Every country web site in Wikipedia has a info table about it which turned out to be too much of a pain to clean up so all I did was to scrape the first three paragraphs following this table with YQL:

select * from html where
url=”http://en.wikipedia.org/wiki/Christmas_Islands”
and xpath=”//table/following-sibling::p” limit 3

The rest, as they say, is history. I built the system in all in all 2 hours and now I spent some time to clean it up and spice it up:

World Info - my 10kb app compo entry (spiced up source version) by photo

As the first loading of the data takes a long time I use HTML5 local storage to cache the country information. This means you only have to wait once and subsequently it’ll be much faster.

You can download and see the source of Worldinfo on GitHub and read through the massive amount of comments I left for you.

If I were to build this as a real product I would cache the results on a server rather than hammering the APIs every time a user comes along – as the information doesn’t change much this makes much more sense. I will probably release a PHP version of that soon. For now, this is what we have.

Building a (re)search interface for Yahoo, Bing and Google with YQL

Wednesday, December 9th, 2009

If you do a lot of research using web searches can be frustrating. Different search engines have different results, you need to open things in tabs and in general it can be pretty time consuming to find what you need.

To make this a bit easier I thought it’d be cool to have an interface that searches Yahoo, Google and Bing at the same time and thus I built GooHooBi:

As explained in the screen cast the thing under the hood of GooHooBi is YQL. Instead of fussing about with all the different search APIs all I did was to play with the YQL console and put together the following YQL statement:

select * from query.multi where queries=’
select Title,Description,Url,DisplayUrl
from microsoft.bing.web(20) where query=”cat”;
select title,clickurl,abstract,dispurl
from search.web(20) where query=”cat”;
select titleNoFormatting,url,content,visibleUrl
from google.search(20) where q=”cat”

The query.multi table in YQL allows you to list a few queries which will be executed one after the other on the YQL server. The queries themselves I put together by using the different tables in the console and only selecting what I really need from each of them.

You can try this query in the YQL console and you can see the JSON output.

The rest is pretty easy. Cut this up into a parameterized string and do a cURL call:

$query = filter_input(INPUT_GET, ‘search’, FILTER_SANITIZE_SPECIAL_CHARS);

$queries[] = ‘select Title,Description,Url,DisplayUrl ‘.
‘from microsoft.bing.web(20) where query=”’.$query.’”’;
$queries[] = ‘select title,clickurl,abstract,dispurl ‘.
‘from search.web(20) where query = “’.$query.’”’;
$queries[] = ‘select titleNoFormatting,url,content,visibleUrl ‘.
‘from google.search(20) where q=”’.$query.’”’;
$url = “select * from query.multi where queries=’”.join($queries,’;’).”’”;
$api = ‘http://query.yahooapis.com/v1/public/yql?q=’.
urlencode($url).’&format=json&env=store’.
‘%3A%2F%2Fdatatables.org%2Falltableswithkeys&diagnostics=false’;

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $api);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$output = curl_exec($ch);
curl_close($ch);
$data = json_decode($output);

Then loop over the results and assemble the HTML output.

You can check the source code of GooHooBi.

In addition to this, here’s a half hour live coding screencast how to build something similar:

Building a search mashup with YQL using Google, Yahoo and Bing – live :) from Christian Heilmann on Vimeo.

The source of the code built in this screencast is also on GitHub.

TTMMHTTM: Twitter coaching, twitter backlash, MP expenses crowdsourced, YQL for data.gov, colour checking and Pixar win

Friday, June 19th, 2009

Things that made me happy this morning:

Safer JSON-P? An interesting feature of the Bing API

Monday, June 15th, 2009

I just looked through the API of Microsoft’s new Bing search (which is really a re-branding of the live search – also, switch to “low bandwidth view” to be able to use the docs much more smoothly) and I found an interesting step in protecting code from throwing errors.

When you provide a JSON output for developers it does make sense to also allow for a callback parameter. That way your code can be used in script nodes without having to use any backend at all. If you for example provide an API to return the names of the Beatles you could have a data endpoint like getBeatles and a parameter for output format:

http://example.com/API/getBeatles?output=json

The return data then will be:

{
“members”:
[

‘Paul’:{ ... more info … },
‘Ringo’:{ ... more info … },
‘John’:{ ... more info … },
‘George’:{ ... more info … }
]

}

This I cannot use in JavaScript without hacks. I’d need to eval (or to be safe JSON parse) the results and with conventional Ajax I cannot reach data outside my domain. To make JSON work as easy as possible you can provide a callback parameter.

http://example.com/API/getBeatles?output=json&callback=eleanorRigby

This should wrap the code in a function call which means the output is already evaluated and the user has to define a callback method to read that information. The output would be:

eleanorRigby({
“members”:
[

‘Paul’:{ ... more info … },
‘Ringo’:{ ... more info … },
‘John’:{ ... more info … },
‘George’:{ ... more info … }
]

});

If I define a function like eleanorRigby(o){} o will be the object returned from the data and I can use it immediately:


Now the issue there is if eleanorRigby is not defined it throws an error.

The Bing API is the first instance where I have seen that they worked around that as the output is this:


if(typeof eleanorRigby 'function') eleanorRigby(
{

"SearchResponse":
{

Version",
"Query":
{

SearchTerms hard day's night"
},
"Translation":
{

"Results":
[

{TranslatedTerm harten Tag-Nacht "}
]

}
}

} /* pageview_candidate */);

I have no clue what the /* pageview_candidate */ is about and frown upon omitting the {} of the if statement, but I must say I do like this.

The issue is now that errors are silent, which might make debugging a pain. Maybe a better option would be to have an error case where the API writes out an error to the console when the callback is not defined:

if(typeof callback = ‘function’) {
callback(... data … );
} else {
if (typeof console!==’undefined’ &&
typeof console.log !== ‘undefined’){
console.log(‘Error: Callback method not defined’);
}

}

All in all an interesting approach though!

TTMMHTM: Wii, Star Trek office, Han Solo PI, Microsoft fix!

Saturday, June 6th, 2009

Things that made me happy this morning

Inspiring and fun

Tech stuff

My stuff:

Videos: