Christian Heilmann

You are currently browsing the Christian Heilmann blog archives for July, 2010.

Archive for July, 2010

Analyzing the FIFA2010 worldcup with Guardian Data and YQL

Wednesday, July 14th, 2010

Breaking news: The Guardian once again involved in committing a data awesome! As before, the UK newspaper graced developers with a really cool piece of information published on the web: all the World Cup 2010 statistics as an Excel Sheet.

Now, the easiest way to play with this data is to use YQL, so I simply took a copy of the information and shared it as a CSV document on Google Docs. That way I can use it in YQL:

select * from csv where url=”http://spreadsheets.google.com/pub?
key=0AhphLklK1Ve4dEdrWC1YcjVKN0ZRbTlHQUhaWXBKdGc&single=true&gid=1&x=1&
output=csv” and columns=”surname,team,position,time,shots,passes,tackles,saves”

You can Try this out in the console and see the results here.

Using YQL to filter and sort this, you can do some interesting searches on that information. For example:

What were the German mid field Players?

select * from csv where url=”http://spreadsheets.google.com/pub?
key=0AhphLklK1Ve4dEdrWC1YcjVKN0ZRbTlHQUhaWXBKdGc&single=true&gid=1&x=1&
output=csv” and columns=”surname,team,position,time,shots,passes,tackles,saves”
and team=”Germany” and position=”Midfielder”

You can Try this out in the console and see the results here.

Using sort() and reverse() you can do rankings. For example:

Who was the goalkeeper with the most saves?

(Neuner of Germany, Kingson of Ghana and Enyeama of Nigeria in case you wonder)

select * from csv where url=”http://spreadsheets.google.com/pub?
key=0AhphLklK1Ve4dEdrWC1YcjVKN0ZRbTlHQUhaWXBKdGc&single=true&gid=1&x=1&
output=csv” and
columns=”surname,team,position,time,shots,passes,tackles,saves”
and position=”Goalkeeper” | sort(field=”saves”) | reverse()

You can Try this out in the console and see the results here.

Which was the player that spent most time on the pitch?

select * from csv where url=”http://spreadsheets.google.com/pub?
key=0AhphLklK1Ve4dEdrWC1YcjVKN0ZRbTlHQUhaWXBKdGc&single=true&gid=1&
output=csv” and
columns=”surname,team,position,time,shots,passes,tackles,saves”
| sort(field=”time”) | reverse()

You can Try this out in the console and see the results here.

Who were the players who were the least on the pitch in the German and Brazilian teams?

select * from csv where url=”http://spreadsheets.google.com/pub?
key=0AhphLklK1Ve4dEdrWC1YcjVKN0ZRbTlHQUhaWXBKdGc&single=true&gid=1&x=1&
output=csv” and columns=”surname,team,position,time,shots,passes,tackles,saves”
and team in (“Germany”,”Brazil”) | sort(field=”time”)

You can Try this out in the console and see the results here.

Using the CSV output and YQL you can do all kind of cool things with that data – as YQL also releases it as JSON it makes it easy to create interactive interfaces and visualizations, too – why don’t you have a go?

Webstylemag – my trip into non-technical blogging

Friday, July 9th, 2010

When my esteemed colleague Ara Pehlivanian told me that he plans to set up a web development lifestyle magazine called http://webstylemag.com and asked me for participation I was reluctant at first as it seems a bit inbred. However, Ara had done such a nice job and I remember having had such good feedback to my Chris’ travel tips – Japan post here that I thought I have a go at sharing my wealth of experiences in traveling to conferences world-wide in the last year and set up a series of posts on webstylemag that give tips how to survive a mental travel schedule like mine. I’ve cut it up into 6 parts:

  • Part 1: “So what did I forget to bring?” tells you about how to pack, what to take with you and how to get to the airport
  • Part 2: “Dealing with discomfort and public humiliation” covers your experiences at airports
  • Part 3: “God I wished I had packed tranquilliser blow darts” covers the journey in the air
  • Part 4: “Parlez-vous Anglais?” – travel from the airport to the hotel
  • Part 5: “Checkout at 11” – going to the conference and back to the hotel
  • Part 6: “Here’s my business card” – travel back and conference follow-up

The first part dealing with packing your bags and getting to the airport is now online for your reading pleasure.

Hope you like it.

Cheers,
Chris

Paypal and Yahoo developer challenge – over $160,000 in prizes

Friday, July 9th, 2010

I am right now working with Paypal on setting up a developer evening in the next few days to talk people through Paypal and Yahoo’s APIs and systems to build something cool for the Paypal X Developer Challenge. All in all the contest has prizes of $160,000 with $100,000 being the main prize and $10,000 prizes sponsored by different Paypal partners.

If you register your product idea by the fourth of August you also enter a special sweepstake for one of 10 iPads.

So what are you supposed to build for this? Mainly a product that uses Paypal’s payment APIs in a new and innovative way.

While PayPal, the external judges, and the X Award sponsors may select winners at their sole discretion, here are some of the general guidelines that they will use:

  • Innovative and “buzzworthy”: Innovative concepts that are able to capture the public imagination will be viewed more favorably than concepts that reflect existing ideas.
  • Viable business: The judges will favor viable business concepts over those that are less likely to succeed.
  • Newer applications favored: We welcome applications that have been built previously, or that have been entered into prior contests, but the judges will tend to favor newer applications and ideas.
  • Merchant integrations are less interesting: While it is possible to enter an online store into the contest because it meets the requirement of using PayPal’s products, classic ecommerce applications are unlikely to win the judges’ hearts.  Tools and services that “change how we pay” will be much more interesting for the judges.

In addition to this we have the X Awards:

We’ve set up some special awards offered by both our partners and us which are designed to reward innovation in key areas. These X Awards are judged independently and a single team could, in theory, win multiple X Awards and/or the Grand Prize. To be considered for an X Award, you will need to meet the eligibility requirements for the overall contest as well as any additional requirements for the X Award(s) for which you are applying. The top X Award entries will be invited to exhibit their applications at the PayPal X Innovate 2010 conference October 26th and 27th in San Francisco.

The Yahoo special award is tied to the following ideas:

Yahoo! Developer Network X Award: Yahoo! Developer Network is excited to participate as a sponsor in the PayPal Developer Challenge. Our prize of $10,000 will go to the application that demonstrates the most useful and inventive integration of PayPal and Yahoo! Technologies. We’re interested in apps that combine PayPal’s open platform technologies with the Yahoo! Application Platform (YAP), our social platform APIs (YOS), and those that use YQL (the Yahoo! Query Language) to access and manipulate web data from multiple sources in combination with PayPal’s transactions platform. Show us apps that solve real world problems in creative new ways; that make it easier or simpler to get transactions done; that enable transactions in new contexts; that create “wow” experiences for consumers, or help us create new business opportunities around the world. Entrants in this category must agree to the Yahoo! Developer Network Terms of Use, located at http://info.yahoo.com/legal/us/yahoo/api/api-2140.html, and the Yahoo! Query Language Terms of use, located at http://info.yahoo.com/legal/us/yahoo/yql/yql-4307.html.   By entering an Application in this category, Entrants agree that Yahoo! may feature your Application in the Yahoo! Developer Network website.  Also note that PayPal remains the sole legal sponsor of this X Award.

I’ll be working with Paypal the next few days to set up an event in London, so watch this space.

TTMMHTM: Accessibility personas, prettier images, Bieber in Korea and lots more!

Wednesday, July 7th, 2010

Things that made me happy this morning

Decreasing the server load of Keywordfinder with YQL

Sunday, July 4th, 2010

Back in the days when Yahoo BOSS came out, I built Keywordfinder to show off the “keywords” functionality of BOSS. BOSS returns you keywords people entered for a certain web site in its results and I used that to list the most popular search terms for a term you enter.

Now, yesterday I got an alert by my host that Keywordfinder is exceeding my GPU limit and checking the logs I realized that it is heavily hit by users from South America. So I decided to add a JavaScript layer to the app that would take away some of the traffic and a lot of the computation from the server.

I could have just replicated the PHP functionality in JavaScript but I thought it is more beneficial for everybody to turn the functionality into a YQL table and use Yahoo’s servers to do the heavy lifting for me.

You can see the keywordfinder open table on GitHub and it will be soon replicated over to datatables and show up in the YQL console. For now you can apply it using the “use” command:

use “http://github.com/yql/yql-tables/raw/master/keywordfinder/keywordfinder.xml” as keywordfinder;
select * from keywordfinder where term=”oil spill”;

try it in the console or see the results as XML

The results right now are: oil, oil spill, spill, the Gulf of Mexico, BP, the Gulf, BP oil, Oil Spill, Deepwater, gallons, Gulf of Mexico, Gulf oil, Horizon oil, crude oil, Gulf, Gulf Coast, Gulf Oil, oil rig, birds, Mexico oil.

The table returns the search term, the region, the language and the terms. It sanitizes all of them for HTML display. By default, the table uses English as the language and USA as the region. If you provide a different language and region you get other results:

use “http://github.com/yql/yql-tables/raw/master/keywordfinder/keywordfinder.xml” as keywordfinder;
select * from keywordfinder where term=”fussball weltmeister”
and language=”de” and region=”de”;

try it out in the console or see the result as XML.

I think it is a nice example how you can speed up an app and help others by adding the functionality to YQL. Enjoy.