Last Friday I went to sunny Southwark to go to IPC media and gave a brown-bag presentation. The topic: IPC wanted to have a chat about using libraries and why YUI might be a good choice.

I covered the fact that almost every web site out there is broken, that the reason for that first and foremost is bad communication in development teams and that users, developers and clients are all unhappy about it.

I explained that we develop for an unknown environment and that the development tools we have are just not good enough (albeit getting better every year).

I lamented the lack of documentation and handover procedures and that as developers we are inclined to build small solutions for one problem over and over again rather than contributing to a larger framework of solutions.

I pointed out that we tend to go from visual display to functionality rather than the other way around which is why we produce inaccessible and overly complicated products.

As solutions to these problems I showed how YUI is built and maintained, how its development tools allow you to build in a predictable fashion and that it does very well what every library out there should do for you: making web development easier and less random.

Slides

Here are the slides of the talk:

• PDF without notes – 9.5MB.
• PDF with notes – 9.5MB.

Audio

I also recorded the talk audio and you can download the recording at archive.org. Listening to the audio it sounds a bit of a rant, however it is not, I am just very passionate about the subject of professional web development and making the internet the #1 media :).

Notes

Professional Web Development Tools

Almost every web site is broken.

If you look around the web you will find that almost every site is broken in one way or another. This starts with small display glitches and ends with the sites being inaccessible or not working for users out there.

This is bad.

This is really bad. It hurts the web as a media. We re-invent the web every year as we just cannot seem to get it to work for us.

Unhappy visitors.

Broken web sites lead to unhappy visitors. The real problem there is that unhappy visitors do not complain to the people who could fix the issues. Most visitors either think they’ve done something wrong or just try to find another site that offers the same content and works. Both of these visitors will never come back. Other visitors complain but get stuck in help desks and never get their problem fixed as it is highly unlikely to ever reach the developers who could fix them.

Unhappy developers.

It doesn’t reach the developers as they are too busy with building new functionality and other sites. If we don’t build new things all the time we are neither happy developers nor seen as efficient employers. Fixing things isn’t sexy.

Unhappy clients.

This leads to unhappy clients. If a client realizes something doesn’t work on the site they paid good money for they want it fixed, regardless of how fringe the problem is and if it only shows up on their machine with their (most of the time outdated) setup.

Reasons

There are many reasons for the broken web, and nearly all of them are our own fault or based on misconceptions.

Lack of communication

Probably the biggest problem of web development is that the different parties involved do not talk to each other or know each others tasks. Developers think they know more than designers, designers think developers are not creative enough in using the arsenal at their hands and product managers see the brand more than the media and are oblivious to the technical boundaries and freedoms the internet gives us. Furthermore we all have our deadlines, deliveries and reports to make and write which takes up too much of our time.

Development environment

Web development has the most terrible and undefined environment ever. There are thousands of browser configurations and versions, each of them failing in different ways. There is a lack of good error reporting, difference in server configurations, connection issues… you name it. Our development is hit and miss and we fix more bugs than we write code.

Piecemeal development

As web developers we always try to build small solutions that solve a problem we have right now. We don’t really consider that all things on a site and across sites should work smoothly together. We’ve been disappointed so many times that we don’t really believe in that.

Lack of handover and documentation

The piecemeal development also means we don’t really document or hand something over. As the next developer is most likely as inclined as we are to build something new (as it surely will be much better than the crud we are asked to maintain) there is no point in that.

Interface to functionality

The biggest issue is that we start with the interface and the cool effect and then work our way down to what the user needs to achieve. We tend to forget very fast that not everybody has the same experience or could benefit from the great shiny interface we want to build. There is a skeleton under every web application and if that skeleton is weak it will break no matter how pretty and shiny we make it.

Solutions

There are solutions for all these issues.

Back to Basics

The first thing to think about is going back to basics when it comes to development. How does the web work, what is the most basic way of reaching a certain goal.

http://uk.tv.yahoo.com/#yeug-search – is a search box with several options. It used JavaScript to change the form’s action when any of the links were clicked.

Task: Define type of search, enter search term,submit form.

There was no need for JavaScript – all we needed was a radio button group and doing the forking on the backend. Notice that the fieldset, the options and the search button form a logical sentence. This is very important for accessibility.

TV channel programme

http://uk.tv.yahoo.com/#ytv-listings – the hardest interface to build as a web developer. Looks like a data table but could have shows that are one minute long! This would mean the table has to have 180 columns and use colspan on every table cell.

Analyse what data you display, and find the easiest way to show it.
Then make it look the way you want it to.

The information the data displays is much easier shown as headlines and ordered lists. CSS does the rest.

Build things people want and know how to use.

Here is where Yahoo offers their findings of user testing with real end users – http://developer.yahoo.com/ypatterns. There is nothing that can replace this knowledge and it is normally very expensive to come by. Before you even think about building an own interface to solve a problem users have to solve, give this a whirl.

Using technology for good

Flash video players are to date the best way to show video – http://uk.video.yahoo.com/. However, they have no reliable keyboard control. By providing buttons that work in HTML and control the video via an API you can make it accessible to all.

Aiming for excellence.

This is the new Yahoo currency converter. It is an amazing piece of web development. It works for all users (including screen reader users) and makes it easy to convert currencies.

Here we explained in detail how it works and the approach we took in developing it.

Removing browsers

The biggest step to professional development and keeping our sanity is to get the random element of browsers out of the equation. You cannot support all the browsers in the world and neither should you.

The graded browser support is a framework to define which browsers you test for and get the full experience. Unknown browsers only get what works in them – no JavaScript and even more obscure browsers get no CSS either.

Making browsers behave.

Libraries have one job: make browsers work. Support is the most random thing in our world as web developers therefore it makes a lot of sense to put all the dirty hacking and fixing of wrong browser behaviour into libraries. YUI is what Yahoo built and uses exactly for that purpose.

First issue is that every browser has an internal style sheet that renders HTML. All of them are different which makes it impossible to develop a reliable look and feel across browsers. YUI Reset works around that.

The same applies to typography. By using the YUI fonts CSS you reset the browser typography to allow you to define pixel sizes as percentages, thus having control and allowing users to resize the fonts.

The CSS grids allow you to create multi column layouts that work across all the A-level browsers easily and reliably. Source order independence comes free, too.

If you are lazy, you can also use the grids builder, define your layout, hit the show code button and get a copy + paste HTML document. The CSS will come from our CDN, which means it gets delivered to your customers from a computer near them geographically.

Doing one job at a time.

YUI does what we as developers would love to be able to do: concentrating on one task at a time. Other than “catch-all” libraries, YUI is cut up into several components, each doing one thing. You can mix and match them to your needs.

DOM access.

One of these components is YAHOO.util.Dom which gives you access to everything that happens in the DOM and convenience methods around the more annoying things the W3C DOM API has.

Using this I can write a script that shows the perfect YUI grid for every size of browser.

Predicting issues and fixing them.

One thing you should do as a developer is being paranoid about things breaking. You should be able to see what can go wrong and set traps for it not to happen.

position:fixed is sexy!

Positioning elements fixed can be very cool. Say for example you have a long document but you want to show the navigation next to regardless of how far down the page you scrolled. Another cool use would be a comments field that allows you to copy and paste quotes from the document.

Positioning the navigation as fixed makes it always visible on the page.

However if the browser window is too small there is no way to reach the elements below.

This small script fixes this problem. Using getRegion I can get the size of any element on the page and getViewportHeight() gives me the available space. If there is more space than needed, fixed can be applied.

Once fixed, let’s re-use.

Using the YUI components we build all kind of widgets based on the design patterns.

Using these free widgets you can re-build yahoo mail yourself.

Re-use means the ability to style differently.

All the widgets are style-able using CSS. You don’t need to know JavaScript or change their code to make them look completely different.

Document your work.

The YUI comes with extensive documentation, both created from comments in the code (JavaDoc style) and step-by-step tutorials. The system that generates the docs from the source code is also available as open source.

Learn by example.

YUI comes with over 300 copy and paste examples of how to use the different components and widgets. As this is how most developers work, we realized that this is a very important part of our success.

Allow for extension.

YUI uses custom events for all of this. This allows you to completely separate your own code from the library. Instead of having to call library methods or call your functions from the library all you need to do is to fire or subscribe to events.

Know what is happening.

Not every browser comes with a great debugging suite like FireBug or Opera’s Dragonfly. This is why Yahoo comes with a logging control.

The logger allows you to debug in any browser that the YUI works in. In addition to this all the YUI widgets and components are shipped as debug versions which report everything they do to the logger. This gives you full control over what is happening and when.

Monitor performance.

The YUI profiler allows you to monitor JavaScript performance – even of non-YUI scripts.

Test before you write.

The same applies to the YUI Test suite. Using this you can apply test-driven development methodologies to JavaScript development.

YUI 3

YUI3 is the new version of YUI, there are many speed and size improvements and we changed the way YUI works significantly to make it more secure, performant and allows you to write much less code to achieve your goal.

Performance

JavaScript performance is one thing, but in order to deliver really successful web sites there are many tricks to apply to create happy end users. The exceptional performance section of the Yahoo Developer Network has them all listed.

YSlow – a Firefox extension allows you to test any web site against these tips and rules and you get immediate, relevant information how to improve the performance of your site.

July and August is a very slow time for me this year as there aren’t any conferences I am speaking at and no hack days planned. September will be packed again, though.

This is why I am right now planning to do some brown bag presentations in the London area. I’ve already been at IPC media, will go next Friday to ebay/gumtree/skype/paypal/shopping.com in Richmond and have had Sky and Thomson Group/TUI show some interest on twitter.

Brown bag presentations are by definition talks during lunch break where people can bring their food along. That way your employees can get some information or discuss their issues with an invited expert without disruption of your normal office hours and times.

I’d be happy to come around and give a talk on all the things I am passionate about. Currently these things are:

• Opening your own and reaching data on the web easily with YQL
• Mining your content and enhancing it with geographical information (Yahoo Placemaker)
• Building specialist search engines (BOSS)
• Accessibility and that it is not magic
• Professional Web Development
• Performance of web products and how to improve it

I am open to other topics, though.

If you are interested in some of that and you are in the Oyster card area, comment here or contact me on Twitter and we can talk date, location, topic and time. I don’t expect any pay, but normally a coffee, some water and a sandwich :). The whole idea of the exercise is to be able to reach developers that normally do not get the chance to go to conferences.

Things that made me happy this morning:

• Naughty photos with the people removed is just not the same
• Panic has a 3 day half-price sale on Transmit, Coda, Candybar and Unison and even a corny commercial for it (you gotta love the Kleenex)
• The German mail released a pretty cool collaborative magazine creator in Flash – in German of course.
• The sales of this 3 Wolves howling at the moon T-shirt allegedly went up 2300% after the fake overly excited comments started.
• Simon Willison creates geocoders – a meta geocoder using three different APIs under the hood
• Keyboard cat the true Internet story gives us all the info about keyboard cat. Now let’s bury it.
• More videos: Transforminators mashup, Damien Walters show reel (parcours, free running – also read the hilarious b3ta comments and check his terrible myspace profile and Bach Bach revolution – Two women play Bach’s Toccata and Fugue in D minor on a giant piano.
• Good quote “Direct HTML input is the autobahn to invalid XHTML.” -Carl Camera

Things that made me happy this morning:

• Browser testing overlord Peter-Paul Koch has a support of CSS in mobile devices table that is full of win
• I did not know that there are hard-core knitters out there. Reminds me of Monty Python’s Hell’s Grannies
• Moby has free music for film makers
• Natalie Downe has a nifty CSS3 style sheet to turn documents into foldable books
• Did you know that Mona Lisa was a firefighter?
• I like things that are what they say on the tin: Explosions and boobs
• Check out the original Star Wars scripts to see where it all went wrong

I am right now very excited about the new Placemaker beta – a location extraction web service released at Where2.0. Using Placemaker you can find all the geographical locations in a feed or a text or a web url and you get them back as an array of places.

As a demo I took the Yahoo News feed and ran it through Placemaker. The resulting places are plotted on a map and the map moves from location to location when you hover over the news items.

The result is online at http://isithackday.com/hacks/placemaker/map.php

Getting the data from the data feed and running it through placemaker is very straight forward. I explained the basic principle in this blog post on the Yahoo Developer Network blog. The only thing to think about is to define the input and output types correctly:

$key = ‘PASTE YOUR API KEY HERE’;

$apiendpoint = ‘http://wherein.yahooapis.com/v1/document’;

$url = ‘http://rss.news.yahoo.com/rss/topstories’;

$inputType = ‘text/rss’;

$outputType = ‘rss’;

$post = ‘appid=’.$key.’&documentURL=’.$url.

‘&documentType=’.$inputType.’&outputType=’.$outputType;

$ch = curl_init($apiendpoint);

curl_setopt($ch, CURLOPT_POST, 1);

curl_setopt($ch, CURLOPT_POSTFIELDS, $post);

curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);

$results = curl_exec($ch);

?>

If you look at the source of this example you will find that Placemaker injected contentlocation elements in the feed itself:

xmlns:georss=”http://www.georss.org/georss” xmlns:cl=”http://wherein.yahooapis.com/v1/cle”

xmlns:xml=”http://www.w3.org/XML/1998/namespace” xml:lang=”en”>



2514815



38.8913

-77.0337





23424793



21.511

-77.8068





23424977



48.8907

-116.982





55843872


Guantanamo, CU]]>

19.9445

-75.1541

You’ll also notice that the elements are namespaced and the names of the locations in CDATA blocks, both things I hate with a passion. Not because they don’t make sense, but because simplexml can be drag to make understand them.

What I wanted to do with this data was twofold: create a JSON array of geo locations to plot on a map and a display of the news content. This is the PHP that does that:

$places = simplexml_load_string($results, ‘SimpleXMLElement’,

LIBXML_NOCDATA);

// if there are elements found

if($places->channel->item){

// start a JSON array

$output .= ‘[‘;

// start the HTML output

$html = ‘
‘;

// set the counter – this will be needed to link news

// items and map markers

$count = 0;

// loop over RSS items

foreach($places->channel->item as $p){

// set inner counter (as there are more locations per news item)

$innercount = 0;

// start the HTML list item and give it an ID with the counter

// value

$html .=  ‘

// all child elements with the defined namespace

$locs = $p->children(‘http://wherein.yahooapis.com/v1/cle’);

// check that there is a location sub-element in this item

if($locs->contentlocation){

// if there is one, add a class to the LI

$html .= ’ class=”haslocation”’;

// start an array for displaying of the locations under the

// news items

$dlocs = array();

// loop over all the places found for this item

foreach($locs->contentlocation->place as $pl){

// append a new JS object with the location data

// and a unique ID to the locations array

$locations[] = ‘{name.’”,”title” title=”“>name“:”’.

preg_replace(‘/n+/’,’‘,addslashes($p->title)).

‘”,latitude” title=”“>lat.

‘”,longitude.’”,”id”:”m” title=”“>lon‘.

$count.’x’.$innercount.’”}’;

// add the location name to the display locations array

$dlocs[] = $pl->name;

// increase the inner count to ensure that every marker has

// a unique ID

$innercount++;

}

	
}

// append the HTML for the news item

$html.=’>
Locations: ‘.join(‘,’,$dlocs).’
‘;

}

	
// end the list item

$html.=’
‘;

// increase the counter

$count++;

}

	
// join the json object data with a comma and close the JSON array

$output .= join(‘,’,$locations);

$output .= ‘]’;

// if there are no items simply return nothing

} else {

$output = ‘’;

}
	
// and this ends the HTML

$html.= ‘
‘;

?>

The result of this can be seen here http://isithackday.com/hacks/placemaker/map-2.php.

The JavaScript to show the map is pretty straight forward and more or less the demo example of the maps API:

// will be called with the array assembled in PHP

function placeonmap(o){

// if there are locations

if(o.length > 0){

// create a new geopoints array to hold all locations

// this is needed to determine the original zoom

// level of the map

var geopoints = [];

// add map with controls

var map = new YMap(document.getElementById(‘map’));

map.addZoomLong();

map.addPanControl();

// loop over locations

for(var i=0;i
// define a new geopoint and store it in the array

var point = new YGeoPoint(o[i].lat,o[i].lon);

geopoints.push(point);

// create a new marker and give it the unique

// id defined in the PHP. Pop up the title of

// the news item and the name of the location when the

// user hovers over the marker

var newMarker = new YMarker(point,o[i].id);

newMarker.addAutoExpand(o[i].title + ‘(‘+o[i].name+’)’);

map.addOverlay(newMarker);

}

	
// define best zoom level and show map

var zac = map.getBestZoomAndCenter(geopoints);

map.drawZoomAndCenter(zac.YGeoPoint,zac.zoomLevel);

}
	
// add a mouseover handler to the list of results

YAHOO.util.Event.on(‘news’,’mouseover’,function(e){

// remove the “news” text of the ID of the current target

// as we named the list items news0 to news19

var id = YAHOO.util.Event.getTarget(e).id.replace(‘news’,’‘);

// if there is still something left we have one of the news

// items

if(id!==’‘){

// get the first marker with the ID we defined in the loop.

var marker = map.getMarkerObject(‘m’+id+’x0’);

// if there is one, pan the map there and show the message

// attached to it.

if(marker){

map.panToLatLon(marker.YGeoPoint);

marker.openAutoExpand();

}
	
}

});

}
	
// call placeonmap with the JSON array

placeonmap();

That’s pretty much it. I am sure it can be refined, but it is amazing how easy it is to get geo information into any text with Placemaker.