Christian Heilmann

Posts Tagged ‘ajax’

My wishlist for a great Ajax API

Tuesday, April 8th, 2008

Coming back from The Highland Fling it was interesting to see that people seem not to be quite convinced yet about the necessity of APIs and the large part they are playing in the next few years of web development. I guess this is partly based on experiences with APIs that aren’t properly explained to non-geeks and inconsistent or hard to use. There is just not much fun in trying to find information bit by bit if all you want to do is write some code (unless you have the old school hacker/cracker mind and didn’t consider spending hours looking at hexdumps trying to find a way to get endless lives in a games a waste of time).

During my interview with Paul Boag at I pointed out that designing a good API is as important as designing any other user interface – including your web page. Gareth Rushgrove agreed in his splendid talk How to be a first class web citizen. I also pointed out that there is a lack of clear and easy tutorials and articles on the matter, so I decided to have a go at it now.

Designing a great Ajax API

As an example I will use the recently released Google translation API, point out its good parts and list things I consider missing. I will not go into the part of actually writing the API but instead explain why I consider the missing parts important. This is not an attack towards Google, I just really liked working with this API and wanted to have it a bit easier to use, so no hard feelings, I really take off my hat that you offer an API like that!

Here are the points I consider important when we’re talking about Ajax APIs in JavaScript (Ajax implies that but you’d be surprised how often a REST API is advertised as Ajax):

  • Good documentation
  • Usage examples to copy + paste
  • Modularity
  • Link results to entries
  • Offer flexible input
  • Allow for custom object transportation
  • Cover usability basics

Documentation and presentation

Let’s start with a positive: the documentation of the Google Ajax Language API is great. You have all the information you need on one page including copy and paste examples. This allows you to work through the API online, read it offline and even print it out to read it on a crowded bus without having to take out your laptop.

Tip: If you are offering copy and paste examples – which by all means you should as this is what people do as a first step – make sure they work! I learnt the hard way in my book Beginning JavaScript with DOM Scripting and Ajax that there is nothing more dangerous than showcasing code snippets instead of full examples – people will copy and paste parts of a script, try to run it and either email you that your code is broken or – even worse – complain in book reviews on Amazon. If you offer copy and paste examples make sure all of them work independently.

Google offer explanations what the API is, what you can do with it, a list of all the parameters and what they mean. This is great for a first-glance user. For the hard-core audience they also offer a class reference.

Usage example

The first code example is quite good, you can copy and paste it and if your computer is connected to the Internet it will work – or it would, if the HTML got some fixes.

First of all it lacks a DOCTYPE, which is a bit annoying as it is a very important part of an HTML document. The more important bit is that the encoding is not set. The live example version has both – bit of a nuisance, as especially when we talk about different languages and using traditional Chinese as the example, the correct encoding is a must.

(Note: the irony, seems like wordpress doesn’t do this right for some reason …)







??????????



Tip: make sure you explain to people that your code examples need an internet connection and other dependencies (like requiring HTTP and thus having to run on a local server). JavaScript historically didn’t have any other dependency than a browser, this is changing lately and can be confusing, especially when you use Ajax behind the scenes like some Flash/Ajax APIs do!

Modularity is good!

The first bit that threw me off to a certain degree was the google.load("langage","1") line, but there is an immediate explanation what it means.

The first script include loads a generic Google Ajax API that has a load() method to add other, smaller APIs build on top of this one. In this case the line means you want to load the language API with the version number 1.

This appears clunky and you will get bad feedback for it (it seems there is nothing better the woo the masses to have a one script include solution) but is actually rather clever.

By modularizing the Ajax code in a base library changes to the core functionality will be easy and by asking the implementer to include the APIs he needs with a version number you can make it the choice of the implementer to upgrade instead of breaking older implementations or having to carry the weight of full backwards compatibility.

Yes, the perfect world scenario is that you’ll never have to change the functionality of your API - just add new features – but in the real world there are constant changes that will make it necessary for you to mess with the original API. There is no such thing as perfect code that is built for eternity. Using a loader function in the base API is also pretty clever, as it means that implementers don’t need to change URLs.

What goes in should come out.

This is where Google created a problem. Both the google.language.detect() and the google.language.translate() methods are quite cool insofar they offer you to send a string and define a callback method when the API returned a value. However, the returning object in both cases gives a result and a status code, but not what was entered. You get all kind of other information (described in the class documentation) but having the original entry would be very useful.

Why? Well the great thing about Ajax is that it is asynchronous, and that is also its weakness. It means that I can send lots of requests in the background in parallel and wait for the results. However, this does not mean that the requests also return in the right order!

This means that if you want to loop through an array of texts to translate, the following is an unsafe way of doing it:

var translations = [ ‘one’,’two’,’three’,’four’,’five’,’six’,’seven’,’eight’,’nine’,’ten’];
var gtl = google.language.translate;
for(var i=0,j=translations.length;i gtl(translations[i],’en’,’de’,function(result) {
if (!result.error) {
var container = document.getElementById(‘translation’);
container.innerHTML += result.translation;
}

});
}

Instead you need to wrap the incrementation of the array counter in a recursive function:

var translations = [ ‘one’,’two’,’three’,’four’,’five’,’six’,’seven’,’eight’,’nine’,’ten’];
var gtl = google.language.translate;
var i=0;
function doTranslation(){
var gtl = google.language.translate;
if(translations[i]){
gtl(translations[i], ‘en’, ‘de’, function(result) {
if (!result.error) {
var container = document.getElementById(‘translation’);
container.innerHTML += result.translation;
i++;
doTranslation();
}

});
}

}
doTranslation();

This is safer, but we lost the opportunity to have several connections running in parallel and thus getting results faster. If the result of the API call had the original text in it, things would be easier, as we could for example populate a result object and match the right request with the right result that way:

var translations = [ ‘one’,’two’,’three’,’four’,’five’,’six’,’seven’,’eight’,’nine’,’ten’];
var gtl = google.language.translate;
var results = {};
for(var i=0,j=translations.length;i gtl(translations[i],’en’,’de’,function(result) {
if (!result.error) {
results[result.input] = result.translation;
}

});
}

Even easier would be a transaction ID to pass in which could be the counter of the loop. Another option of course would be to allow more flexibility in the data that goes in.

Offering flexible input

Both the matching of the input text with the result and a transaction ID still would mean a lot of requests to the API, which is not really nice as it costs money and clobbers the server and the client alike. An easier option would be to not only allow a string as the text parameter but also an array of strings. The return then would also become an array and a lot of the overhead of calling the translation engine would be done on the server in a single call instead of lots and lots of API calls.

This is not hard to do and most JavaScript framework methods work that way, by checking the type of the first argument and branching accordingly. You can even go further and allow the implementers to send an own bespoke object as a third parameter.

Transporting a custom object allows implementers write a lot less code

The benefit of a custom object going out and in is that you can add more parameters to the API call that are only specific to the implementation. Most likely this could be a reference to a namespace to avoid having to repeat long method names or global variables. You could start by providing parameters that make sense to any Ajax call in terms of usability.

Thinking Ajax usability

The main thing any Ajax call should offer a user is a timeout. There is nothing more disappointing than getting the promise of a brave new Ajax world with much more interactive interfaces and then getting stuck looking at spinning wheels or worse hitting a link and getting nothing. Right now the language API has nothing like this, and you’d have to roll a solution by hand. You’d also have to check the error status code to see if the data could not be retrieved and call a failure case of the connection that way.

A nice API would offer me these options, most likely all rolled in one parameters object.

My dream translation API

Taking all these into consideration it would be perfect to get the API to offer these options:

google.language.translate(input,parameters);

The parameters would be:


input // string or array
parameters // object with the following properties
sourceLanguage:string,
targetLanguage:string,
transactionId:string,
customparameters:object, // to transport
timeout:integer, // (in milliseconds)
failure:function(result,params), // (method to call when there is a timeout or an error)
success:function(result,params), // (method to call when all is fine)

The returned data from the API should have both the result and the parameters provided. This would make the life of implementers dead easy.

Summary

In summary, here’s what I expect from a great Ajax API:

  • Have a good documentation with immediate copy and paste examples backed up by a full class documentation
  • Build your APIs modular and allow the implementer to choose the version they want to have
  • Provide a hook to link the result of the API methods to the initial data entered. The easiest way is to repeat this data, more sophisticated is to allow for a connection ID.
  • Allow for multiple values to be sent through, it’ll save you API calls and the implementer hacking around the problem of unreliable order of returns.
  • Allow implementers to add an own object to send and get back to allow for namespacing and other data state retention.
  • Allow for a timeout, connections are not to be trusted.

This is a work in progress

I hope you found something here to agree with and if you know things to add, just drop a comment.

AjaxWorld East – Cold weather, Arctic Snowcruisers and a shift in perception

Thursday, March 27th, 2008

I am just sitting at the airport in Montréal, waiting for the delayed flight to arrive and get me back to New York City to get back to London. I’ve spent the last few days first in New York to speak at the AjaxWorld East conference, then go to Montréal, Canada to speak at the Coder’s Saturday and give a two presentation workshop at Nurun. Suffice to say, I am pretty bombed out, but this is a good chance to jot down my experiences during this trip. Let’s start with AjaxWorld.

The audience

The AjaxWorld conference was a bit of a surprise to me. Judging by the price of a ticket and the web site explaining all the other presentations I had braced myself to stand in front of a large room full of suits that needed to go to the conference rather than wanting to go. Turns out I was wrong. The audience was a mixed crowd of company owners, project managers, designers and developers and everybody was very involved and interested. I’ve met a lot of companies selling developer tools like frameworks and IDEs, a few implementing companies (including the lovely people from the food network who took me out for dinner for a real Italian NYC style) and really surprising edge developers like someone who ran software that controls geostationary satellites! I got a lot of interesting questions and feedback and I am happy to have met everybody there.

The location

The conference took place in the Roosevelt Hotel, located a block away from New York’s Grand Central Station. The hotel is a 1920s affair full of charm and grandeur of a bigone aera. You entered the place and felt like you are in a Hercule Poirot episode. However, for a tech conference this size it was not a great choice. For starters there was no internet connection that was either affordable or stable. Even the $24/day room connection went up and down like a roller coaster. This meant that I was driven to the bagel place (Cosi) on the other side of the street a lot as they had free Wi-fi (I am sure they made a killing these few days).

As for presentation facilities there was the main presentation hall which was a ball-room with really high ceilings reserved for sponsor talks and the keynote. This one easily allowed for all attendees to fit in, but was excruciatingly cold. All the other presentations took place in smaller meeting rooms not quite big enough to hold all interested delegates. The projectors were put in between rows of chairs which made it tricky for you to reach the screen and from time to time attendees would bump against them.

The hotel itself is also grossly overpriced for the comfort the rooms offer. My hotel in Montréal was about a third of the price per room per night, ultra-modern and had free connectivity, a fax and a safe in each room and came with complimentary breakfast. OK, it is New York city and the Roosevelt was built in the 20s, but a fleecing is a fleecing, no matter how your dress it up.

The organization and coverage

Apart from that, the hotel had a cozy atmosphere in the bar and the organizers did their best to keep everything in check. They did a splendid job filming the whole event and live-streaming parts of it. When you needed help, they were immediately there to help you out, something that is quite an uncommon event in NYC. There were more than enough handouts of the current schedules, signs were put up everywhere that you needed and every delegate got a free book on Ajax with the massive badge telling others who that person is and what he or she does. I felt pretty well looked after and was easily convinced to deliver an extra presentation in place of another presenter who had missed his flight.

The only thing I didn’t like too much is the amount of parallel presentations. I missed a lot of things I wanted to see, but couldn’t because I can’t split myself in two. Clever companies made sure that they sent several people there to allow for full coverage.

My presentations

My first presentation was a gap-filler for a speaker that didn’t show up and I wrote it a day before. I talked about the opportunity we have right now to use Flash to deliver rich media content on web sites and control the experience with JavaScript provided that we get a proper API. I got really interested in that lately, working with great people like Aral Balkan, Niqui Merret and Steve Webster. As examples I showed how SWFObject allows you to progressively enhance HTML to contain Flash, how the YUI image uploader allows you to batch upload files using Flash and how the YouTube API allows you to control online video with JavasScript.

My main and planned presentation was about architecting JavaScript for large applications using Event Driven Design showing off Yahoo Maps and Eurosport as examples. The unexpected bit of the presentation was that I explained it using the Antarctic Snowcruiser as an analogy.

The Antarctic Snowcruiser was a massive vehicle built in 1937 to explore Antarctica. The task the cruiser had to fulfill was amazingly problematic – it was to host a crew of 5 and keep them safe from the terrible cold for a year to cross 5000 miles of hostile environment. The cruiser was an absolute marvel of technology and its inventors thought up amazingly clever solutions to the problems they anticipated, including the possibility to retract the wheels into the body of the cruiser to overcome crevasses and to re-use the engine heat to keep the crew from freezing to death. The only thing they forgot to plan for was traction and when the cruiser arrived in Antarctica its wheels spun uselessly and the engine overheated almost immediately. The cruiser got abandoned and was found 20 years later encased in ice. Now it is unknown where it went as the piece of ice it was last seen on broke off and got dragged out to sea.

This was my example how we build applications – we assume a lot and over-engineer on the server side without realizing that most of the application work will be done by a browser in an to us unknown environment. I wanted to inspire people to consider the usability and restrictions of web applications before making assumptions that everything will work and that we can predict the environment our apps run in.

Both presentations were packed and I got good feedback, I’d be interested in more from people that went there and I am looking forward to seeing the videos.

Other presentations

I didn’t see many other presentations, but here is what I got: Douglas Crockford’s Keynote was a stark reminder how broken the technologies we use to create web applications really are, showed security flaws and ideas as to how we can try to fix these. Caja was mentioned along with adSafe and Douglas explained his idea of vats to secure the web. There was more and more detail published on his blog the last few days, so make sure to check there.

In terms of security, there was a full-on attack vector talk by
Danny Allan of Watchfire which showed exactly how far an unrestricted scripting injection will get attackers. It is pretty easy to disregard XSS as a banal penetration of your site but this presentation showed how a fully patched new Firefox running on a windows machine with firewall and up-to-date virus software can be accessed with malicious JavaScript and get all kind of passwords and deeper access using a mix of phishing and social engineering.

The IBM people involved in the Dojo framework showed off the internationalization and accessibility options of Dojo in a joint presentation and I was very impressed as to how far the framework embraced ARIA and how much information was given on how to do both i18n and a10y with usability in mind. Great work!

I’ve spent a lot of time with the people who build qooxdoo, a widget framework exclusively written in JavaScript. Their presentation showed what the framework is and how it works cross-browser. They also showed the very impressive performance of the widgets and how they can be skinned. The really cool stuff of qooxdoo is still in the making though and I will make sure that we watch the guys more closely and reveal some of the clever tricks they use to boost performance and build application code cleverly using a Python build script.

The last presentation I’ve seen was introducing ways to build iPhone applications that have the native look and feel and it was more or less a re-hash of the developer guidelines given out by Apple mixed with an introduction to iUI. I was not at all inspired by the talk which is amazing as iPhone is a really nice platform and highly interesting at the moment.

Summary and learnings

All in all it was well worth it going to the conference. The mix of presentations and topics covered was well done and I myself realized that it is high time we stop advocating web standards as a technical solution. Web standards are there to ensure maintainability and predictability but in the enterprise world we are already one step further and technologies like Flex, Silverlight and Comet are what need our close attention to ensure that we have predictable and maintainable solutions in these as well. Douglas’ view as the browser model as being broken can be considered as too strong a message, but I realized that a lot of the things end users expect these days are just not possible with the HTML/CSS/JS trinity we keep preaching about. This does not mean standards are unimportant – they are best practice in terms of implementation. Technologically we have to brace ourself to be surprised of the changes in the next year.

Code tutorials for lazy people with Ajax Code Display

Monday, January 28th, 2008

Currently I am writing a lot of tutorials for an online self-training course about web standards and I ran into the annoyance of having to maintain example code in two places: the code itself and the HTML document with the explanations. Therefore I took jQuery and wrote a small script that automatically turns links to HTML code examples with HTML entities and line numbers. You can define which lines to display, which lines should be highlighted and you can add a live preview in an IFRAME when the link is clicked.

Let’s make 2008 the year of embracing the server side with Ajax

Sunday, December 30th, 2007

I am always fascinated by the amount of Ajax tutorials and examples out there that totally ignore the backend part of an Ajax app. A lot of times you’ll find page-long ravings about the 6-7 lines of JavaScript that allow the client to make an HTTP request but when it comes to talking about the proxy script needed to allow for cross-domain requests a lot is glossed over as “you don’t need to know this, just use this script”.

That would not really be an issue if the scripts offered weren’t that bad. Unsanitized URLs are the main attacking point for cross-server-scripting attacks. If you use a PHP_SELF as the action of your forms you shouldn’t be too confused about a lot of mail traffic from your server or text links on your site you didn’t sign off and get money for.

The other thing about Ajax information on the web that amazes me is that people keep complaining about the slowness and problems with converting data from one format to another on the client side. Let us not kid ourselves: even after all the articles, books and podcasts about Ajax we still have no clue whatsoever what a visitor uses to look at our products. We cannot tell for sure what browser is used, if there is assistive technology involved or anything about the specs of the computer the browser runs on. This to me makes the client side the least preferable place to do heavy calculation and conversion.

The server side, on the other hand, is in your control and you know what it can do. Complex regular expressions, XSLT conversion, all of this is much easier to do on the backend – and you know that the text encoding will work to boot. A lot of complexity of Ajax apps is based on bad architecture and design decisions and on relying on the client side to provide necessary functionality.

So if you ask me what the ratio of client-to-server code of a good Ajax app is I’d say 30% client and 70% server. The 70% on the server should be used to provide security, non-JavaScript fallback functionality (yay accessibility) and conversion of data to small, easy-to-digest chunks for the client (think HTML and JSON). The 30% client side code should mainly be used up to enhance the usability of the product and make it easier for your visitors to reach their goals.

So here’s my plan for 2008: whenever I talk Ajax I will try to cover as much backend as frontend. I’ll do this by partnering with other experts as I myself created some terrible PHP in the past. I hope that others will follow that example as Ajax is a wonderful opportunity to bridge the gap between frontend and backend engineering – and we have to talk to each other to create a good app.

Planning JavaScript and Ajax for larger teams, equine invigorating imagery, one voice for libraries and a lot of good speakers – this was @mediaAjax 2007

Wednesday, November 21st, 2007

I’ve just been over the @mediaAjax drinks, worked through around 670 emails that accumulated in my inbox since last Wednesday and now there is some time to talk about my experience there.

First of all, here are the slides of my talk:

I didn’t cover many technical issues but instead tried to convince some people to recognize that the way you use JavaScript in your team is the most important part of development. It is not important that you write amazing code, but instead it is much more important to work smoothly together to ensure that you can deliver fast and on budget. This is the only thing that keeps us from being able to train and grow developers – we are too busy playing catch-up with estimates and deliveries. Instead we should concentrate on bringing the fun back into developing by working together rather than competing or being the “JavaScript hero” for the rest of the team.

I got some good feedback and I am happy to get more, so if you’ve been there, tell me about it. Even more importantly, if you are considering on taking on some of the ideas, I’d be interested in hearing how that went in your environment.

Generally I have to say that I enjoyed the conference a lot. I was at first disappointed by the lack of wireless but I guess that in the end it made people listen more intently to what the speakers had to say.

My faves:

  • Dion and Ben of Ajaxian covered the State of Ajax, and compared the current happenings to a wishlist they’d drafted 2 years ago. It was a fun enough keynote, but I am not too sure about some of the CSS bashing that was going on. It is very interesting to see that while CSS people want to do everything with their technology, a lot of hardcore coders just don’t grok CSS. Hey, the future is hybrids, play technologies to their strengths and allow people who care to do them.
  • Derek Featherstone’s “Real World Accessibility for Ajax-enhanced Web Apps” talked about the problems we have in creating rich client application modules like tree menus and how we fail doing them in an accessible manner. It was very entertaining and he had some good real-world photos to drive his points home. My personal favourite was the irony that the stage had a wheelchair ramp that lead nowhere because of the backdrop
  • Stuart Langridges “How to destroy the web” was as unique as he is as it was an anti-presentation as to what to do. He ran the devil’s advocate idea of us building web products in the most obtrusive and bloated manner possible as otherwise people would use all the bandwidth for looking at horse porn (his words, not mine). He had a lot of very good points (don’t do this, as it leaves too much bandwidth) and it reminded me of the Vincent Flander’s approach to teaching web development
  • John Resig’s introduction to Prototyping with jQuery showed how you can easily take a saved copy of a web site (in this case an Apple page) and enhance it with jQuery after analyzing the HTML structure. This is pretty cool for just showing a product person how a change could feel like and I’ve done it in the past with Greasemonkey scripts.
  • Alex Russell of Dojo gave us some food for thought about how libraries should become one voice and talk about the same ideas and overlap in terms of implementation to ensure we all help developers the same way instead of competing. Something I can wholeheartedly agree with.
  • Brendan Eich showed the ideas of a “new JavaScript” while Douglas Crockford explained why the “now” JavaScript is actually a cool language as it is.
  • Dann Webb showed ways how you can use JavaScript’s trickier parts like prototype and clever uses of the arguments array to enhance the language and do meta programming with it.

All in all I was happy to be part of this and I thought it one of the best conferences if you wanted to learn about JS. I did hear people complain that there wasn’t that much about Ajax, though. To me, Ajax is a methodology, not necessarily a fixed set of technologies, and to cover the whole aspect you’d need to explain both the client and the server architecture. True, this could have been done more, and I also lacked the coverage of Flash Developers as programmers that already use a lot of things that JavaScript2 is promising us.

All in all there were several great finds for me:

  • Never sit next to Bruce Lawson, Chris Willison and Stuart Langridge. It is evil but also terribly funny.
  • It is great to see that people involved in the development of competing libraries all want the same stuff and really don’t mind the success of the other competitors. Take this down, fanboys and mailing list flamers: we all want to help you and think you should use whatever suits you best, which is not necessarily our stuff.
  • The best way to recognize in your server side component if a call came from Ajax and not from a normal page submit is to check the HTTP header. Most libraries send a bespoke identifier!
  • Make sure you check if a queen has a jubilee before you go out of the train in the Westminster area.

I hope the people who went also had fun and the others will consider coming to the next conference in the London area. It is great fun and value for money.