Christian Heilmann

Posts Tagged ‘yql’

TTMMHTM: Reference in FireBug, YQL even cooler, Unicorns, Second Base explained, and much more

Friday, February 6th, 2009
This is our second installment of Ask SM, featuring reader questions about Web design, focusing on HTML, CSS and JavaScript. If you have a question about one of these topics, feel free to reach me (Chris Coyier) through one of these methods:
1. Send an email to ask [at] smashingmagazine [dot] com with your question.
2. Post your question in our forum. You will need to sign up (yes, the forum is not officially launched yet, but it is running!).
3. Or, if you have a quick question, just tweet us @smashingmag or @chriscoyier with the tag [Ask SM].
Please note: I will do what I can to answer questions, but I certainly won’t be able to answer them all. However, posting questions to the forum gives you the best opportunity to get help from the community.

Displaying useful tweets on your blog (second version using YQL)

Friday, January 16th, 2009

Back in September I showed a solution how to display filtered tweets on your blog using Yahoo Pipes which is in use on this page in the sidebar on the right.

There were a few issues with it: the data returned was pretty big and slowed down the page (yeah I could delay this after page load, but size down the wire is size down the wire) and I only used the last 20 updates, which – with my twitter pace – meant that a lot of times there wasn’t any ‘blogworthy’ tweet available any longer.

So, I dug around and realized that instead of analyzing the RSS feed of your tweets it makes much more sense to use the API, in detail the user timeline which allows me to get up to 200 of the latest tweets using the count parameter.

This is awesome, however, as the API repeats all my information in every status (I guess to cover changes, like location) the result can get very large, right now for example the JSON output of my twitter updates is 120KB – not good.

By using YQL I can easily cut this down to the bare necessities using the following command:

select status.text from xml where url = 'http://twitter.com/statuses/user_timeline/codepo8.xml?count=200' and status.text like '%§%'

Using this with JSON output and wrapper I only get the updates I need as a 1.2KB large JavaScript file!

The HTML and JavaScript solution doesn’t change much:




Now I got a faster useful tweets badge with information that stays longer. Easy.

TTMMHTM: Google and public transport, Microformats in YQL, ICTP and a free book on webdesign

Wednesday, January 14th, 2009

Things that made me happy this morning:

How long is yours and how happy are you with it (twitter style)?

Tuesday, January 13th, 2009

One of the “useful” services of the web made the rounds on Twitter today: Twicksize allows you to show the world the size of your twick in gradient and round goodness (incidently, mine is 11 inch).

Whilst being stunned by the usefulness of the service and the aesthetic perfection I felt a little prick of annoyance that there was no API for it. Surely showing off your twicksize to the world on your blog would be useful. Well, this can be fixed.

Using the net tab of Firebug, it is pretty easy to detect the API driving the site and you find out that http://twicksize.com/ajax.measureme.php?twittername={user_name} gets you the HTML to show your twember in all its glory.

Using YQL and the correct XPATH you can easily turn this into a data set to work with.

The size is one thing, but as a lot of well-intentioned email keeps telling me it is all about how happy you are with your size. Luckily there is another service called Happy Tweets that uses science, magic and an obviously stunningly clever algorithm to determine the happiness of a twitter user. Again using Firebug we can easily determine that http://happytweets.com/TweetAnalysis.aspx?u={user_name} is the magic URL to get the data and using YQL gives us this information

Putting both together, it is easy to write a simple script to tell the world all about the size of your twick and how happy you are with it:



I am codepo8


I’ve whipped it out for you here. If you want to know about your size and happiness, just use your username as the hash at http://icant.co.uk/sandbox/are-you-twick-happy.html#techcrunch, for example techcrunch or icaaq.

The options are endless, we could for example use Google charts to create a nice image :)

Adding transcripts to presentations embedded from SlideShare using YQL

Sunday, January 11th, 2009

I like SlideShare a lot (yeah, repetition, I know). It is a great way of spreading your presentations as it allows others to embed them into their blogs and web sites and it also allows people to download and re-use what you’ve done.

One thing I really like about SlideShare is that it creates HTML transcripts of your presentation displayed far down the page as shown in the following screenshot:

Screenshot of slideshare showing transcripts

Now, the annoying thing is that the SlideShare API - full of goodness as it is – does not provide a way to get to these transcripts in case you want to display them alongside your presentation. You also need to authenticate for detailed information which is very needed but also bad if you just want to offer a JavaScript solution.

Good thing that there is an easy way to retrieve information from any web site out there now and get it back as JSON to use in JavaScript: YQL. Using the YQL console and some XPATH I can for example extract the transcription information and get it back as XML (the slideshare URL is http://www.slideshare.net/cheilmann/shifting-gears-presentation and the XPATH //ol/li):


http://query.yahooapis.com/v1/public/yql?q=select * from html where url%3D”http%3A%2F%2Fwww.slideshare.net%2Fcheilmann%2Fshifting-gears-presentation” and xpath%3D’%2F%2Fol%2Fli’&format=xml

If you define the format as JSON and provide a callback parameter you can easily use this to write a small script to inject transcripts into SlideShare embeds:

slideshareTranscripts = function(){
var div = document.getElementsByTagName(‘div’);
var containers = {};
for(var i=0;div[i];i++){
if(div[i].id.indexOf(‘__ss’)!==-1){
var slideurl = div[i].getElementsByTagName(‘a’)[0].href.split(‘?’)[0];
containers[slideurl]={c:div[i],id};
get(slideurl);
}

}
function get(url){
var url = ‘http://query.yahooapis.com/v1/public/yql?’ +
‘format=json&callback=slideshareTranscripts.doit&q=’ +
‘select%20strong,p%20from%20html%20where%20url%3D%22’ +
encodeURIComponent(slideurl) +
‘%22%20and%20xpath%3D%27%2F%2Fol%2Fli%27&’;
var s = document.createElement(‘script’);
s.src = url;
s.type = ‘text/javascript’;
document.getElementsByTagName(‘head’)[0].appendChild(s);
}

function doit(o){
var url = decodeURIComponent(o.query.uri).split(‘”’);
var out = document.createElement(‘ol’);
var lis = o.query.results.li;
for(var i=0,j=lis.length;i var li = document.createElement(‘li’);
var strong = document.createElement(‘strong’);
var p = document.createElement(‘p’);
strong.appendChild(document.createTextNode(lis[i].strong));
p.appendChild(document.createTextNode(lis[i].p));
li.appendChild(strong);
li.appendChild(p);
out.appendChild(li);
}

containers[url[1]].c.appendChild(out);
}

return{doit:doit}
}();

You can see the script in action and download it for your own use. All you need to do is add it to the bottom of any document with one or several SlideShare embed codes in it. Here’s what the script does:

slideshareTranscripts = function(){
var div = document.getElementsByTagName(‘div’);

We get all the DIV elements in the page (there is probably a faster way using CSSQuery these days, but let’s be bullet-proof).

In order to find out which one of the DIVs is a SlideShare embed, we check for an ID that contains __ss as the embed codes have a generated ID starting with this.

What we will do with each of these is to find out what the url of the slideshow is. This is needed because of two reasons: first of all we want to retrieve the transcript and secondly we need a way of matching the returned data from YQL with the right DIV container.

As generated script nodes are not safe to load one after the other this makes sure that the right transcript gets added to the right slides.

So, what we do is check the ID of the DIV, get the first link, retrieve the url from it, store the DIV in an object called containers and use the url as the property name. This property gets a shortcut to the correct DIV as its value. All we need to do then is to get the transcript data via get():


var containers = {};
for(var i=0;div[i];i++){
if(div[i].id.indexOf(‘__ss’)!==-1){
var slideurl = div[i].getElementsByTagName(‘a’)[0].href.split(‘?’)[0];
containers[slideurl]={c:div[i],id};
get(slideurl);
}

}

The get() method is your garden variety script node generation function that calls the YQL API and defines slideshareTranscripts.doit() as the callback parameter. This means that the script will generate a script node, point it to YQL, YQL then gets the transcript information from SlideShare, turns it into JSON and calls doit() with the transcript information as a parameter.


function get(url){
var url = ‘http://query.yahooapis.com/v1/public/yql?’ +
‘format=json&callback=slideshareTranscripts.doit&q=’ +
‘select%20strong,p%20from%20html%20where%20url%3D%22’ +
encodeURIComponent(slideurl) +
‘%22%20and%20xpath%3D%27%2F%2Fol%2Fli%27&’;
var s = document.createElement(‘script’);
s.src = url;
s.type = ‘text/javascript’;
document.getElementsByTagName(‘head’)[0].appendChild(s);
}

The job of doit() is to generate the right markup and inject it into the embed codes. We create an ordered list and loop over all the li elements. We create the fitting HTML code for each of the list items and add the content as text nodes (innerHTML would try to render markup used in the slides – not a good idea).

We then retrieve the URL from the uri property and add the list as a new child node to the container div stored previously in the container object.

nt
function doit(o){
var out = document.createElement(‘ol’);
var lis = o.query.results.li;
for(var i=0,j=lis.length;i var li = document.createElement(‘li’);
var strong = document.createElement(‘strong’);
var p = document.createElement(‘p’);
strong.appendChild(document.createTextNode(lis[i].strong));
p.appendChild(document.createTextNode(lis[i].p));
li.appendChild(strong);
li.appendChild(p);
out.appendChild(li);
}

var url = decodeURIComponent(o.query.uri).split(‘”’);
containers[url[1]].c.appendChild(out);
}

return{doit:doit}
}();

That’s it. Of course you can simplify this by matching only the OL and using innerHTML to write out the list in one go, but this solutions also allows you to alter the HTML output of the list to – for example – make every http://example.com a link.

Happy transcribing!