Christian Heilmann

Posts Tagged ‘yql’

Postcode from latitude and longitude or even IP – fun with Geo APIs and YQL

Tuesday, June 9th, 2009

One of the more complex things about GeoFill was to get postcode information from an IP. However with a collection of APIs and a collated YQL statement even this was possible.

The first thing I needed to get was the IP of the user. This is done with the GeoIP API based on the GeoLite API from MaxMind. This is available as an open table in YQL and can be used thus:

select * from ip.location where ip=”“

Try the lookup in the console or check the lookup result

Response”: {
“Ip”: “216.39.58.17”,
“Status”: “OK”,
“CountryCode”: “US”,
“CountryName”: “United States”,
“RegionCode”: “06”,
“RegionName”: “California”,
“City”: “Sunnyvale”,
“ZipPostalCode”: “94089”,
“Latitude”: “37.4249”,
“Longitude”: “-122.007”,
“Gmtoffset”: “-8.0”,
“Dstoffset”: “-7.0”
}

This gives us a lot of information. What’s really important here is latitude and longitude, as this can be used in the flickr.places API to get a where on earth ID which is a much more defined identifier:

select * from flickr.places where (lat,lon) in (
select Latitude,Longitude from ip.location where ip=””
)

Try the flickr places call in the console or check the flickr result

“places”: {
“accuracy”: “16”,
“latitude”: “37.4249”,
“longitude”: “-122.007”,
“total”: “1”,
“place”: {
“latitude”: “37.371”,
“longitude”: “-122.038”,
“name”: “Sunnyvale, California, United States”,
“place_id”: “P_ls_fybBJwdHP8t”,
“place_type”: “locality”,
“place_type_id”: “7”,
“place_url”: “/United+States/California/Sunnyvale”,
“timezone”: “America/Los_Angeles”,
“woeid”: “2502265”
}

}


Here the interesting part is the woeid which we can use to dig deeper into geo.places:

select * from geo.places where woeid in (
select place.woeid from flickr.places where (lat,lon) in (
select Latitude,Longitude from ip.location where ip=””
)

)

Try the geo places call in the console or check the geo places result

The result is all the information you’d ever want.

“place”: {
“lang”: “en-US”,
“xmlns”: “http://where.yahooapis.com/v1/schema.rng”,
“yahoo”: “http://www.yahooapis.com/v1/base.rng”,
“uri”: “http://where.yahooapis.com/v1/place/28751237”,
“woeid”: “28751237”,
“placeTypeName”: {
“code”: “22”,
“content”: “Suburb”
},
“name”: “Fairgrounds”,
“country”: {
“code”: “US”,
“type”: “Country”,
“content”: “United States”
},
“admin1”: {
“code”: “US-CA”,
“type”: “State”,
“content”: “California”
},
“admin2”: {
“code”: “”,
“type”: “County”,
“content”: “Santa Clara”
},
“admin3”: null,
“locality1”: {
“type”: “Town”,
“content”: “San Jose”
},
“locality2”: {
“type”: “Suburb”,
“content”: “Fairgrounds”
},
“postal”: {
“type”: “Zip Code”,
“content”: “95112”
},
“centroid”: {
“latitude”: “37.326611”,
“longitude”: “-121.878441”
},
“boundingBox”: {
“southWest”: {
“latitude”: “37.275379”,
“longitude”: “-121.89254”
},
“northEast”: {
“latitude”: “37.330879”,
“longitude”: “-121.808723”
}

}
}

Adding address information automatically to forms with GeoFill

Sunday, June 7th, 2009

One of the things I mentioned in my latest talk and the follow-up Q&A at Gumtree was that I am very interested in Geolocation and automatically detecting the user’s location to save them having to enter all the same info over and over again. Sure, location services like Fire Eagle and autofill options in browsers already do these kind of things, but they are power user toys for now.

Using Rasmus Lerdorf’s GeoIP, YQL and Yahoo Geo I’ve put together a small JavaScript wrapper that does exactly this for you:

GeoFill - automatically filling form data with geo information by  you.

GeoFill (also available on GitHub) allows you to use two different methods to automatically fill forms with geo information:

  • geofill.find(properties) does an IP lookup of the current user and tries to get the geographical data from that one.
  • geofill.lookup(properties,postcode) tries to get the geographical data from the postcode provided in your form.

Both methods take an object which lists the form field IDs to be filled as the parameter. The data returned is city, country, postcode, latitude and longitude.

Both methods also allow you to define a callback function to handle errors or re-use the data in other ways than filling form fields.

For example to get the information connected with a post code you could simply do the following:

geofill.lookup(
{

callback:function(o){
console.log(o);
}

},’wc2h8ad’);

The returned object is:

{
city:”London”,
country:”United Kingdom”,
latitude:”51.51384”,
longitude:”-0.12857”,
postcode:”WC2H 8AD”
}

Useful? In any case it made me play more with the Geo API. Don’t forget that on the 7th of July I’ll be giving a free Tech Talk on Yahoo Placemaker where this will be one of the demos (probably, knowing myself I’ll have hacked other stuff by then). You can sign up for the tech talk on upcoming.

TTMMHTM: Wii, Star Trek office, Han Solo PI, Microsoft fix!

Saturday, June 6th, 2009

Things that made me happy this morning

Inspiring and fun

Tech stuff

My stuff:

Videos:

How I built icant.co.uk – source code

Wednesday, June 3rd, 2009

After my talk at FOWA in Cambridge yesterday I showed off that http://icant.co.uk is fully driven by YUI and YQL and maintained elsewhere. I’ve recorded a screencast about this earlier which is also available for download as a M4V but hadn’t released the code yet.

So here goes – this is the PHP source of icant.co.uk (without the HTML which is more or less done with YUI grids builder


// get all the feeds to grab the data
$feeds = array(
‘http://feeds2.feedburner.com/wait-till-i/gwZf’,
‘http://feeds.delicious.com/v2/rss/codepo8/myvideos?count=15’,
‘http://feeds.delicious.com/v2/rss/codepo8/sandbox’,
‘http://feeds.delicious.com/v2/rss/codepo8/icantarticles’,
‘http://www.slideshare.net/rss/user/cheilmann’
);

// assemble the YQL statement
$yql = ‘select meta.views,content.thumbnail,content.description,title,’.
‘link,description from rss where url in ‘;
$yql .= “(‘” . join($feeds,”’,’”) . “’)”;

// assemble the request url
$root = ‘http://query.yahooapis.com/v1/public/yql?q=’;
$url = $root . urlencode($yql) . ‘&format=json’;

// get the feeds and populate the data to echo out in the HTML
$feeds = renderFeeds($url);
$blog = $feeds[‘blog’];
$videos = $feeds[‘videos’];
$articles = $feeds[‘articles’];
$presentations = $feeds[‘slides’];

// this function loads all the feeds and turns them into HTML
function renderFeeds($url){

// grab the content from YQL via cURL
$c = getStuff($url);

// as the content comes back as JSON, turn it into PHP objects
$x = json_decode($c);

// reset counter for videos and presentations
$count = 0;
$vidcount = 0;

// start new array to return
$out = array();

// loop over YQL results, if they exist.
if($x->query->results->item){
foreach($x->query->results->item as $i){

// if the link comes from the blog, add to the blog HTML
if(strstr($i->link,’wait-till-i’)){
$out[‘blog’] .= ‘

  • link . ‘”>’ . $i->title .

    ’ . html_entity_decode($i->description) .

  • ‘;
    $vidcount++;
    }

    // for interviews and articles, add to the articles section
    if(strstr($i->title,’Interview’) ||
    strstr($i->title,’Article:’)){
    $out[‘articles’].= ‘

  • link . ‘”>’ . $i->title .

    YQL doesn’t send a diagnostics part

    // grab the books from my blog
    $yql = ‘select * from html where url=’.
    ‘”http://wait-till-i.com/books/”’.
    ’ and xpath=”//div[@class=’entry’]”’;
    $books = renderHTML($root.urlencode($yql).’&format=xml&diagnostics=false’);

    // this is a quick and dirty solution for the HTML output
    function renderHTML($url){
    // pull the information from YQL
    $c = getStuff($url);
    // check that something came back
    if(strstr($c,’<’)){
    // remove all the XML parts
    $c = preg_replace(“/.*|.*/”,’‘,$c);
    $c = preg_replace(“/ ” encoding=”UTF-8”?>/”,’‘,$c);
    // remove all comments
    $c = preg_replace(“//”,’‘,$c);
    }

    // send it back
    return $c;
    }

    // a simple cURL function to get information
    function getstuff($url){
    $curl_handle = curl_init();
    curl_setopt($curl_handle, CURLOPT_URL, $url);
    curl_setopt($curl_handle, CURLOPT_CONNECTTIMEOUT, 2);
    curl_setopt($curl_handle, CURLOPT_RETURNTRANSFER, 1);
    $buffer = curl_exec($curl_handle);
    curl_close($curl_handle);
    if (empty($buffer)){
    return ‘Error retrieving data, please try later.’;
    } else {
    return $buffer;
    }

    }

  • YQL and BOSS cheatsheets for Open Hack Day

    Tuesday, May 5th, 2009

    With the final planning steps for the London Open Hack Day this weekend, YDN just released two sweet handouts that explain the workings of BOSS and YQL on two pages each:

    YQL and BOSS cheatsheets for Open Hack Day by  you.

    You can download them from the YDN test server, in case you want them in the office: