Christian Heilmann

You are currently browsing the Christian Heilmann blog archives for June, 2009.

Archive for June, 2009

Geo this! Geolocate WordPress posts with Greasemonkey and Yahoo Placemaker

Monday, June 22nd, 2009

Geolocating content on the web is a great idea. By embedding latitude and longitude and real place names in your document you allow data mining for location or easy display on a map.

The problem up to now was that it is quite a job to find out the correct geo information from a text or a document and it is quite a pain to enter the information by hand.

Yahoo Placemaker is a web service that helps you with that – you give it some text or a document URL and it returns you all the things it found in there that resemble a geographical location back. The issue with doing that on a live site is that you slow down your site immensely as you need to look up every time.

The more logical place to do the lookup with Placemaker is when you edit your document. I thought this would be cool to have for this WordPress install here and wrote a small GreaseMonkey script that injects a new “Geo this!” button in the main WP form:

Geo this - button by  you.

When I hit the button the script does an Ajax request using the Placemaker open YQL table to get the information for the currently edited text.

Once it found the information it adds it at the end of the document as a GEO microformat. Each found entry starts with a comment that tells you what Placemaker matched and considered a geographical location. As it is not infallible this makes it easy for you to delete wrong entries.

Geo this - added microformats by  you.

Try it out yourself:

This is pretty much rough and ready and I’d be happy for feedback how to improve it.

TTMMHTTM: Twitter coaching, twitter backlash, MP expenses crowdsourced, YQL for, colour checking and Pixar win

Friday, June 19th, 2009

Things that made me happy this morning:

Five things for you – an introduction to the offers of the Yahoo Developer Network

Thursday, June 18th, 2009

Yesterday I was in Barcelona, Spain and talked to a few dozen local developers about the things they can find on the Yahoo Developer Network to have an easier time building web sites and impress their bosses. Amongst other things I showed:

Barcelona by  codepo8.Yahoo office Barcelona by  codepo8.

Here is the presentation and my notes with the source of the code examples for you. You can “download the slides as a PDF (11mb) from S3”: or watch them on slideshare:


Five things for you

Today I am going to talk about five things Yahoo offers to developers. In case you wonder, Mortadello y Filemon is actually rather big in Germany – known as “Clever and Smart” hence I couldn’t resist using them.

English only

Sorry, but I don’t speak any Spanish, but there are colleagues of mine around for Q&A later who can translate if needed. Also, if I speak too fast, just shout and I will try to make it easier.

I am Chris

I’m Chris and I am a developer evangelist. As you can see I also speak with my hands a lot, but that is because I am passionate about what I do. I want to make the web an easier, more professional environment to work in and a media that people can enjoy regardless of ability or technical setup.

Get this talk and others later

This is my main web site where I publish all of the talks I give, videos and other goodies. So this presentation will end up here later on and you can re-mix it for own trainings or presentations. Creative Commons rocks.

The Yahoo Developer Network

I work for the Yahoo Developer Network which you might not have heard of before. In essence what the YDN does is offer almost everything you see on Yahoo – the data displayed and the interface elements used for you to use in your own products via APIs, SDKs and a great development framework. The YDN homepage is your first stop for all the information I am talking about today.

1) Research

Let me start by showing you some of the research we’ve done and the results we offer you to build your own solutions on. In addition to what I am showing here also make sure to chat to some of the colleagues here, as we have a research lab here in Barcelona and they are working on some really cool products.


The Design Pattern Library is a collection of solutions to problems or tasks users of our web sites and applications had. We do a lot of user testing in labs and we ask the right questions to find out exactly what people consider the easiest way to for example pick a date in a web form. All of this research is documented in these use patterns with a description of the problem, an explanation of the solution and a cross-reference to where in Yahoo we use the solution. If you build an interface and you don’t have time or budget to do your own research, this is a great resource to use. The patterns are creative commons so please tell us if your experience differs and we can amend them.


If you do visuals all of these patterns are also available as stencils for all major design tools. That way you can easily build and design an interface that your developers will be able to create in CSS, HTML and JavaScript.


The Exceptional Performance section of the Yahoo Developer Network is where we give you all the information how we made our web sites behave faster and more responsive. In our world every millisecond the page takes to load and display counts – we can relate every second wasted to money lost. While your sites might not be that dependent on great performance it does never hurt not to waste your visitors’ time. In this section you find tips and tricks how to tweak your server, how to write CSS and JavaScript and how to optimise images easily organised and explained in detail.

2) Flexible, professional development

As a large company we need to be professional in how we work with each other as developers. There are dozens of offices with hundreds of developers in Yahoo and we found that every developer and every office solves the same problems, makes the same mistakes and re-invents the wheel over and over again. Web development is a very undefined area and the biggest issue is that we spend 90% of our time fixing bugs rather than developing. We wanted to change that and make sure that we can concentrate on architecting our software solutions and develop and tweak them for the users’ benefit rather than fixing annoying and confusing browser bugs.

Graded Browser Support

The first thing we needed to make sure is that we define and describe the support we have for different browsers. This is a methodology and resource site we have called the Graded Browser Support and is something you can use for your product documentation and pitch, too. You cannot give every visitor of your web site the same experience regardless of browser and you actually should not even think that way as that kind of thinking is against the main principles the web was invented for. You have to make your products work for everybody, but you can give a different experience dependent on the sophistication of the visitors’ setups. This is what the GBS does. We define which browsers we support fully and test the high-end experience for. All other browsers don’t get this experience but something that works regardless.

The Yahoo User Interface library

The Yahoo User Interface library was the next big step to take towards professional development. This is a library ranging from CSS solutions, over JavaScript components up to plug and play widgets that are based on the support ideas we defined in the GBS and best practices we found in our pattern research. The YUI is where we work around all the annoying bugs and problems browsers and web technologies have to allow us to build solutions without wasting our time on wondering why they don’t work in setup xyz. By keeping all this in a base library we can fix and apply for any new technology or browser coming out. If we did all of the fixing in our sites we’d have to revisit each and every one of them when technologies change. This is not effective, hence we built ourselves a library.

The myth of the unstyled page

There is no such thing as an unstyled page in browsers. If you don’t apply a style sheet, then browsers have an in-build default style that varies from browser to browser. This makes it impossible for you to design a predictable interface. That’s why we created a style sheet that un-does all the styling browsers apply to a page. This means you can start with a clean canvas.

Predictable typography

Typography was he next hurdle. Every designer will give you font definitions in pixels and when you set the font in pixels, Internet Explorer 6 will not allow visitors to resize the text on your page to their needs. This is why we created a style sheet that resets the font of your browser to a readable minimum (again based on user testing) and has a percentage scale for you to use that relates directly to pixel sizes.

CSS layouts

Layout in HTML should not be done with tables or presentational markup but with CSS. The problem there is that browsers support the techniques we need to use to make them display our layouts in different ways. A lot of hacking has to be done to create a layout that works across the board. This is why we defined a grid structure for our sites and a style sheet that works. All we need to do now is add the right IDs and classes to a few DIV elements and we have a working layout. If you are really lazy, we even have a CSS grids builder tool for you. The CSS is hosted by us on a distributed network of servers and gets delivered to your visitors from a server near them which is great for performance. You can however host the CSS yourself, too.

Specialised solutions

The JavaScript part of the YUI is not a catch-all solution for all development tasks you’ll ever have to tackle but instead is cut up into smaller components that all do one thing and do it very well. That way you can mix and match the library components to the need of your solution and keep the footprint low.


On top of the YUI components we built widgets that relate to the design patterns. These are the things you see on Yahoo’s pages – sliders, tabs, menus, rich text editors and the like. We even have a showcase where we proved that you can rebuild the interface of Yahoo mail by using these free and open source building blocks.


The look and feel of all of these widgets can be changed by changing a style sheet. This is very important. You should never have to alter JavaScript simply to make a tab a different colour.

Working widgets

This is an example of one of the widgets – the autocomplete control. As you can see the widget is built to solve a pattern we found in the pattern design library and using our logger control gives you full information about what is happening to it at every step of the process of using it.

Progressive enhancement

Our widgets are built to work and to give a smoother experience only when and if the browser supports it. This means an autocomplete control will be a text box when JavaScript is turned off. A data grid or complex sortable and styled table will be a simple, accessible data table when JavaScript is turned off. That way you build working solutions for every user and you don’t promise functionality that isn’t delivered. Users trust you, if you don’t hold your promises you will lose them.

Event driven architecture

All of the widgets are built with custom events and if you want to enhance their functionality or override it you can do that at any point of the interaction journey by subscribing to the custom events that get fired. This allows you to build solutions that need to do something extra for you without compromising the underlying code. You should never change the library code as that stops you from being able to upgrade the library to a never version.

The logger control

One of the most annoying things in web development is that not every browser gives you good debugging tools. The Logger control works around that issue by providing you with a debugging console that works across all the browsers we support.

The profiler

If you wonder why your JavaScript is using up a lot of memory and slows the computer down you can use the YUI profiler to analyze what is going on inside your JavaScript and fix bottlenecks that way.

3) Documentation

Both the YUI and all of our APIs come with extensive documentation written by a professional team of tech writers.

Online and offline docs

The APIs come with a quick “get started” guide and a full documentation – in many cases even available offline as a PDF version.

Step by step introduction and examples

The full documentation comes in easy to digest chunks guiding you where you need to go to find exactly the right kind of information.To get you a quick start there are copy and paste examples of demo code.

Code generated documentation

In addition to this the YUI also has a JavaDoc style documentation which is generated from comments in the source code. This means it will always be up-to-date even if there are quick fixes in the code. The tool to create the documentation from JavaScript code is also available.


Another nice service the YUI has is a zip of PDF cheatsheets for all the different widgets. These are A4 sheets with all the necessary information to get you started and find out how to quickly change an implementation of a certain widgets.

Forums, lists, blogs and videos

If you are still stuck or are just not a person receptive to code or text there’s the YDN theater with lots of videos of presentations, screencasts and demos. There are forums, mailings lists and other forms of communication with the teams available in the communities section

4) Things to impress your Boss with

One of the things I am always trying to make sure is that you can go to work tomorrow and tell your boss about some things you can use to save the company money and time or just impress him/her. So here are some ideas.

Build an own search engine with BOSS

Yahoo BOSS is an API that gives you access to the search index of Yahoo. You can display results from Yahoo, re-order and display like you want and mix the results with other data. That way you can easily build a vertical search engine. Furthermore the data includes information not displayed in the normal Yahoo search results like microformats, RDF information, delicious tags and keywords.

Research keywords for your market

Using the keywords in the BOSS data I was able to quickly build Keywordfinder. What it does is searching Yahoo for a keyword you enter, return the 20 first results, get the keywords users entered to find these sites, rank them for you and give them back in order of relevance. You can use this to see what terms you should have in your titles, headings and copy to get more search engine love.

Create a performance report and recommendations for your site

YSlow is a Firefox extension that automatically tests web sites against the best practices Yahoo found in the exceptional performance section. You can easily find the issues and get inline tips how to fix them. One really impressive part is smushit, which optimizes all your images as a batch and allows you to download them as a zip to replace the original ones.

YQL to remix the web

YQL and especially the YQL console is a terribly easy way for you to access APIs of Yahoo and third parties without knowing much about their authentication, parameter structure or return data. By using YQL remixing the web is as easy as accessing a database. For more detail, check out this presentation on YQL

How about a YQL demo?

This web site about Barcelona does not contain any content on the server but pulls all of the data from third party locations – Wikipedia, Flickr, and Yahoo weather. The layout was done using the YUI grids builder and the following is all the code that is needed for it to be what it is:


$root = ‘’;
$city = ‘Barcelona’;
$loc = ‘Barcelona’;
$yql = ‘select * from html where url = ‘’.$city.’’ and xpath=”//div[@id=’bodyContent’]/p” limit 3’;
$url = $root . urlencode($yql) . ‘&format=xml’;
$info = getstuff($url);
$info = preg_replace(“/.*|.*/”,’‘,$info);
$info = preg_replace(“/ ” encoding=”UTF-8”?>/”,’‘,$info);
$info = preg_replace(“//”,’‘,$info);
$info = preg_replace(“/”/wiki/”,’”’,$info);

$yql = ‘select * from where woeid in ‘.
‘(select woeid from geo.places where text=”’.$loc.’”)’.
’ | unique(field=”description”)’;
$url = $root . urlencode($yql) . ‘&format=json’;
$events = getstuff($url);
$events = json_decode($events);
foreach($events->query->results->event as $e){

  • urls->url->content.’”>”’.<br $s->title.’” src=”’.$src.’”>

  • ‘;

    $yql=’select description from rss where ‘.
    ’ url=””’;
    $url = $root . urlencode($yql) . ‘&format=json’;
    $weather = getstuff($url);
    $weather = json_decode($weather);
    $weHTML = $weather->query->results->item->description;

    function getstuff($url){
    $curl_handle = curl_init();
    curl_setopt($curl_handle, CURLOPT_URL, $url);
    curl_setopt($curl_handle, CURLOPT_CONNECTTIMEOUT, 2);
    curl_setopt($curl_handle, CURLOPT_RETURNTRANSFER, 1);
    $buffer = curl_exec($curl_handle);
    if (empty($buffer)){
    return ‘Error retrieving data, please try later.’;
    } else {
    return $buffer;


    Doing multiple searches with a single HTTP request

    Another powerful option YQL gives you is to collate multiple API calls into a single request. Say for example you want to search the web for “vicky, cristina, Barcelona” and also for each of the terms on their own. This can be easily done with YQL in a single statement:

    select * from search.web where query in
    (‘vicky’,’cristina’,’barcelona’,’vicky cristina barcelona’)

    The result can be seen here and this is the PHP code that creates this demo:

    $root = ‘’;
    $yql=”select * from search.web where query in (‘vicky’,’cristina’,’barcelona’,’vicky cristina barcelona’)”;
    $url = $root . urlencode($yql) . ‘&format=json’;
    $search = getstuff($url);
    $search = json_decode($search);
    $search = $search->query->results;
    $vHTML = ‘’;
    $bHTML = ‘’;
    $vHTML = ‘’;
    $aHTML = ‘’;
    foreach($search->result as $i){
    $tmpl = ‘
  • clickurl.

  • ‘;
    if(preg_match(‘/vicky/i’,$i->abstract) &&
    !preg_match(‘/cristina/i’,$i->abstract) &&
    if(preg_match(‘/cristina/i’,$i->abstract) &&
    !preg_match(‘/vicky/i’,$i->abstract) &&
    if(!preg_match(‘/vicky/i’,$i->abstract) &&
    !preg_match(‘/cristina/i’,$i->abstract) &&
    if(preg_match(‘/vicky/i’,$i->abstract) &&
    preg_match(‘/cristina/i’,$i->abstract) &&
    $aHTML.= $tmpl;
    $vHTML = @join(’ ‘,array_slice($v,0,3));
    $bHTML = @join(’ ‘,array_slice($b,0,3));
    $cHTML = @join(’ ‘,array_slice($c,0,3));

    function getstuff($url){
    $curl_handle = curl_init();
    curl_setopt($curl_handle, CURLOPT_URL, $url);
    curl_setopt($curl_handle, CURLOPT_CONNECTTIMEOUT, 2);
    curl_setopt($curl_handle, CURLOPT_RETURNTRANSFER, 1);
    $buffer = curl_exec($curl_handle);
    if (empty($buffer)){
    return ‘Error retrieving data, please try later.’;
    } else {
    return $buffer;


    You can also do the whole thing in JavaScript:

    5) Partnering

    The really interesting thing is that all of this is open for you to add your own data to. Using a simple XML document called an open table you can make your information available in YQL for other developers.

    You can host this XML file yourself if you don’t want the whole world to know yet. For example this open table can be used in a statement like this:

    use ‘’ as nmm;
    select * from nmm where category = ‘art’ and
    searchterm = ‘”tower bridge”’

    This gives developers access to all the information about the objects in the National Maritime Museum in London!

    If you want your open table to show up in the console via our team (we do review them for security and quality) all you need to do is add the table to github.

    TTMMHTM: IE6 not allowed to play, YQL weather and multi query, clever words, and my own geeky facebook vanity url

    Tuesday, June 16th, 2009

    Reaching out to developers – a brown bag talk at Sky in London

    Monday, June 15th, 2009

    Last week I visited Sky in London for a brown bag session during lunch. Their head of innovation, Paul Kane had asked me to come around and talk about API design, the history of YQL and in general tips and tricks how to get innovation and developer outreach working in companies and media. One part of the presentation was the following talk.

    Here are the detailed notes:

    Reaching Out

    Strategies and ideas

    • Third party developers make your product better
    • Reaching them can be tough
    • Here are some ideas and strategies that worked in the past

    Third party developers are an amazing force to make your products better and more relevant to the users you want to reach. Reaching them can be a bit of a headache and today I am going to talk about some of the ideas and strategies that worked well in the past and some of them I’d love to see followed more consistently.

    The importance of being open

    • Opening your data and services multiplies your reach
    • Developers are happy to use your products and test them at the same time
    • You can not test for all environments, other people can

    The first thing to be aware of is that being open to developers multiplies your reach and makes it much easier to create a solid product. With each developer using your APIs or data feeds you reach audiences you don’t have access to and have another developer looking over your product.

    Developers are very happy to point out issues and in fact do report issues – users are not that vocal about these issues. This means that a lot of companies are happily polluting the web with terribly unusable products without being aware of it. You can’t anticipate all environments and use cases to test in – if you open your data to the world the developers using it can do that for you.

    Building for web use

    • Be Platform independent
    • Be RESTful
    • Release all documentation and code examples in a portable format

    The first thing to make sure is that to achieve ultimate reach you should concentrate on building systems that are meant to be used on the web. This means first of all that your web service should be platform independent.

    REST does this for you. If you build a system that gives back data over HTTP implementers can use a browser, cURL on the command line or any framework they want to get your information.

    Your documentation, information about the API and code examples should be available in as easy as possible formats that are portable and independent of platform. HTML, PDF and zipped code does all this – you cannot expect people to download and install an SDK like a piece of software – we are busy developers and hate polluting our machines with unnecessary installs.

    Thinking of endpoints

    • Good URL structures
    • Easy to understand method names and parameters
    • Versioning of APIs

    The success of a REST based API stands and falls with the quality of its endpoints. Make sure that your URLs are human readable and make sense. A good web service should be navigable in the browser bar and I shouldn’t have to read up the docs to understand where I want to go.

    Keep your method names and parameters short and to the point – nobody likes long URLs as there is no tab completion in the browser bar (yet).

    One thing that is terribly important is to version your API in the URL. If you find a bug or you need to rename something in a newer version as new dependencies come up you don’t want to break implementations that were based on an older version. A v2,v3,v4 in the URL is not long but prevents that from happening.

    Thinking of output

    • Provide data that people need – a lot of data is good but don’t add information that only makes sense internally.
    • Provide data formats that people can use: xml, rss, json
    • Allow for callback parameters for json to use as JSON-P
    • Maybe allow for object transportation / ID for JSON-P calls
    • Allow people to turn off diagnostics and filter down to specific needs

    These are very important points. As a general rule it is a good idea to provide as much data as possible in your APIs. However if this data is of no use to a third party developer or there is simply no documentation for the data then it shouldn’t be in there as it simply distracts from the necessary information. The Yahoo music API for example has a catzilla and hotzilla ID in the results – I have no clue what that is.

    The data format question is pretty easy, XML for a full-fat result, RSS to really make it easy for people to use the data and JSON for people like me who don’t like to wade through XML with namespaces.

    JSON-P is JSON with a callback parameter that allows the end user to use the data immediately in JavaScript without having to use a proxy to reach your content. The callback ID is an idea I’ve mentioned before. The problem with using JSON-P and generated script nodes is that you can never be sure in which order several calls are returned. Therefore it is a good idea to allow for an ID to be sent and returned with the call so I can match up data with request in the callback method. This could be as simple as repeating the request terms in the result set, but a real transaction ID would make it even more handy.

    Diagnostic parts of API calls can help me find out what went wrong or what was going on but it is good when you can turn it off. YQL for example offers that option, check this call vs. this one – the difference is the diagnostics=false parameter. In general it is a good idea to allow developers to filter down the amount of data being sent back (f.e. with an amount parameter) as there are environments where really every byte counts.

    Providing easy access

    • For data APIs that are read only, don’t bother with authentication – just use a developer key
    • Have sandboxes where developers can play without having to sign up

    Nothing is more annoying than having to authenticate before even getting to see the kind of content you can access with your API. Therefore it is a good idea to keep the pain of authentication as low as possible.

    A great idea is to cluster the access to different developers or have a no-frills access version. Vimeo are doing a great job offering different APIs and so is Yahoo Maps. Flickr is providing an API explorer and the Guardian API also has a console for running and filtering down requests. The Rolls Royce of course is the YQL console.


    • Provide proper documentation that is quick to read and easy to navigate
    • Offer cheatsheets and quick introductions
    • Have a documentation team – developers are not writers and are too close to the product to actually write readable docs

    None of these need much more explanation. Simply be aware that documentation is something you shouldn’t take lightly as developers go to docs when they get stuck and when they encounter only more frustration and hard to understand information there you are not likely to be a big success.

    Personally I don’t think you should bother localising documentation unless you need to provide different content to different markets and have to explain developers why some content is not available to them. Let bloggers do the advertising in the local markets for you.

    Demo Code

    • Provide demo code and SDKs (if you really must)
    • Have excellent demo code – not ‘look, just two lines of code’
    • Offer different language demos

    Demo code is a very powerful tool and shows the quality of your development team. Of course every developer can wade through quickly put together code and get something done, but your code should entice developers to use your systems and create excellent solutions.

    Your demo code should not have any security holes, accessibility omissions or simply bad code practices. This code is your advertisement to other developers – it should be easy to copy and paste it and hit the ground running but you should not leave a trail of security concerns and global variables behind.

    If you provide copy and paste examples make sure they work outside your environment – all links to dependencies should be absolute and not relative (starting with http at best) so that you don’t get complaints about code not working. I found it best to list a whole example and the explain it chunk by chunk what is going on.

    Consistency is key here – keep to an agreed code standard and you won’t confuse implementers. Other than that your code should be customizable as easy as possible – use config objects to allow for that.

    Be responsive

    • Have a mailing list / forum that is monitored
    • Keep your eyes out for things people do with your code (blogsearch, twitter)

    Developers will do a lot for you and nothing gives them more satisfaction than getting their problems answered by an expert. Have a mailing list or forum with your expert developers or at least people that can ask the right person. Monitor the buzz that is going on and chime in to give praise or set things straight. There is nothing cooler than seeing your tweet about an implementation you’ve built with a certain API getting re-tweeted from an official account or by experts connected with this API.

    On the other hand there is nothing more deadly than a real implementation problem remaining unanswered for days. Be visible, be there for people and you’ll have happy users.

    Invite people to play

    • Invite selected people for a private beta – this is amazingly powerful as people’s integrity will work for you – or you get great feedback what to change
    • Give people a chance to build “official mashups” before release to show on the site and in the press conference.
    • Maybe launch with a developer day

    This is pretty important. Releasing an API is one thing, having other people that the whole audience trusts in their decisions advocate for you is a much bigger impact. Open up betas to some people beforehand, have a handful of API keys upfront for people to play with and have keys that people can give out on twitter or their blogs – all of this will have quite an impact.

    The Guardian did a great job with this when they released their APIs. It also works for product launches, as spotify has shown. Having people build official mashups with official blog coverage also allows these people to write “making of” articles and blog posts which show tips and tricks you might not have thought of before.

    You can take this idea further – have some undocumented but cool new features every few weeks or so (or previews of upcoming features) and leak them out to twitter or on developer lists. A lot of developers are very happy to be the first to show the world that your API is working on something new and will try to hack around the non-support by writing own docs.

    Collaborate with other API providers.

    • Reach out to other API providers
    • Take part in groups dealing with the same topic
    • Create an open YQL table, add your data to gnip

    Last but not least remember that you are not alone. We all release APIs and can learn from each other’s mistakes and gains. This presentation is living proof for that and I am not alone in reaching out to companies who want to join the ranks of those who build and maintain the web of data. An open YQL table is written pretty fast and gives you a chance to let developers play with your data without having to worry about limitations and caching for now – the YQL access restrictions and server does that for you.