Christian Heilmann

Posts Tagged ‘xml’

The annoying thing about YQL’s JSON output and the reason for it

Wednesday, September 22nd, 2010

A few days ago, Simon Willison vented on Twitter about one of the things that annoyed me about YQL, too:

Battling YQL's utterly horrible JSON output, where a response with one result is represented completely differently from a response with two

The interesting thing about this is that it is not a YQL bug – instead it is simply a problem of generic code and data conversion.

The problem with YQL JSON results

Let’s have an example. If you use YQL to get for example the geo information about “London” you get several results in XML:

select * from geo.places where text=”london”:

XML output with multiple places

If you use JSON as the output format this becomes an array:

JSON output with multiple places

Cool, so you can loop over, right? No, cause when there is only one result, we have a different situation. Instead of being an array, place becomes an object:

select * from geo.places where text=”london,uk”:

Single results in JSON become an object instead of an array

This is what ailed Simon as it means you need to write a function that works around that issue. For example in the Geoplanet Explorer, I use a render() function to show each place result and I use the following to work around the JSON output issue:

function renderlist($p){
foreach($p as $pp){
$o .= render($pp);

} else{
$o = render($p);

return $o;

In JavaScript you’d have to check the length and either show the results directly or loop over the results.

Generic conversion vs. creating a single API

This is a problem when you convert XML to JSON - as the resulting XML from the GeoPlanet API doesn’t have a places element with place in it but instead repeats the place element for every result the JSON parser must make a decision: if it is only one element, this becomes a property of the results object. If there are more than one place element it becomes a property that is an array (as you cannot repeat the same property name).

If you build your own API and you know the format of the different results you can force this to be an array even when there is only one result. If you do not know the XML schema there is no way of assuming what should be an array and what should not be an array. So the generic solution for XML to JSON conversion of an unknown XML with 1 to n results would be to always create an array. That way you couldn’t get to place.admin1.code for example but you’d always have to do place[0].admin1[0].code[0] – not really a sensible thing to do.

As YQL is an open system and the XML results are provided by anyone in their open table definition the only way around I could think of is to tell YQL in the table that some elements should always be forced to become arrays.

Do you have a solution?

Another interesting YQL feature: XML with callback (JSON-P-X)

Thursday, July 9th, 2009

Yesterday’s announcement of YQL now supporting insert, update and delete somehow overshadowed another interesting new feature: XML with callback!

XML with callback?

On first glance having a callback parameter on an XML output doesn’t make any sense. The normal use case of callbacks is to turn JSON data into JSON-P so that you can use the data immediately in JavaScript. The reason for XML with callback is to make it easier to use the same data when getting data from the web. Say I want to use YQL to get the images and links of the people I follow from twitter. The YQL statement is:

select * from html where url=””
and xpath = “//div[@id=’following_list’]”

The XML output of this call is a pretty hefty XML document with all the HTML as an XML node structure.

The JSON (and JSON-P output) is even worse as it gives you a structure of all the elements and their attributes as properties of nested objects:

div: {
id: “following_list”
span: [

a: {
href: “/jemimakiss”
hreflang: “en”
rel: “contact”
title: “Jemima Kiss”
img: {
alt: “Jemima Kiss”
height: “24”
src: “http://s3.amazonaws.c…glasto_mini.jpg”
width: “24”


//... and so on …

Converting this back into HTML could be quite an annoying job – not to say slow. This is why YQL now offers the callback parameter for XML. The JSONP-X output as it is called in the YQL changelog makes this task a lot easier by returning a JSON object with the XML as a string:



Jemima Kiss [...]



This makes it dead easy to render the results back as HTML:

Nice? I think so!

Scrabblr – A scrabble calculation/validation API

Friday, August 22nd, 2008

As it is Friday and I am jetlagged I declare “pointless API Friday” and give you Scrabblr.

Scrabblr allows you to send a word to it and it returns the scrabble value or an error in case the word has too many instances of a character than there are tiles available. Output formats are either a simple string (no format defined), XML or JSON. For JSON you also can have a callback parameter to use the API in script nodes.