Christian Heilmann

You are currently browsing the archives for the api category.

Archive for the ‘api’ Category

Scripting Enabled at @mediaAjax 2008

Monday, September 15th, 2008

I am right now at @media Ajax 2008 getting ready to go on stage to deliver my “Scripting Enabled” talk, explaining how the main issue about accessibility is that we just don’t talk enough to each other. Technology is never really the boundary we have with accessibility, it is that we don’t understand how people work and what technology is capable of.

[slideshare id=598740&doc=atmediaajaxscriptingenabled-1221470402418982-9&w=425]

Links in the presentation

So you want to create accessible online video, huh?

Monday, July 21st, 2008

This is a short explanation as to what I consider a good way to make online video accessible. I’ve based it on research done to create the Easy YouTube Player. There are no code examples in this article – only links to solutions that are already in place. We will tackle the issue of a sample framework for accessible online video at Scripting Enabled

Online video is a great thing, no doubt about that. Not a day goes past where millions of people on the web spend time watching tutorials, screencasts, instructions, TV shows, movies and yes – monkeys falling out of trees – online.

The problem is that whilst video can be highly inclusive – as shown in this example of a deaf chat room – it is also alien to the technologies the web was built on.

Video is a stranger to the web (for now)

Browsers do not have any native video display yet – HTML 5 is going in that direction but it’ll take some time until all browsers in use support that. In order to play videos on web sites we need to rely on other technology embedded into web documents – so called plug-ins.

Plug-in technology came and went over the years, there was RealPlayer with its infamous “buffering” messages, Quicktime with its amazing difference in quality between Macs and lesser fortunate computers, Windows Media player with its inability to play on other platforms than Windows (until lately, that is) and a plethora of Linux geek formats – OGG, Matroska and the like – all of them free, open source, full of great ideas and lacking any significant adoption and market share.

You can use any of these to create online video and they all come with good authoring tools – most of them on the expensive side – but if you really want to make your life easier, use the plug-in that left all of the others in its wake in terms of market adoption and cross-platform portability: Flash.

Flash – aaaaaaaaaaaahhhhhh, saviour of the internet?

YouTube and other online video sites showed that the main trick to reach millions of users and consumers out there is to make embedding video information as easy as possible. Most of YouTube’s views and traffic comes from outside their network from embeds in blogs, news sites and other web sites. With Flash being very much on almost every computer it’d be almost criminal not to support it. Monopolies are never good, but lately Adobe has gone leaps and bounds in making Flash more available to third party developers and in terms of internal structures and APIs many another technology can take a leaf out of their book. And of course we have Silverlight to battle this monopoly in the nearer future – let’s hope Microsoft take the Flash learnings into consideration.

The Flash video format (FLV) is tailored for web use – small, streaming, allowing for captions and sound and video separation – it simply makes a lot of sense.

So we have the technology, we have a good format, what keeps us from creating inclusive video?

Failure to focus – browser and plugin and assistive technology not sitting in a tree

The problem with Flash and accessibility is not necessarily the system itself – it is the communication of browsers with plug-ins. These things being alien to the native environment of HTML, we have once again to live with vast differences in implementation across browsers. If you open the flash movies directly in the browser, this is less of an issue, the problem is embedding them into HTML documents. This is what we constantly do though to ensure that search engines can find our sites as until recently Flash content wasn’t indexed by them.

Microsoft, with Active-X as the poster child of amazing productivity and interaction between standalone apps and the web (and allowing many a virus and Trojan to run free) did actually a very good job in allowing communication of Flash, MSIE and assistive technology.

In Firefox, on the other hand, you cannot use your keyboard to tab into a Flash movie. You can set and release focus in Flash and tab around movies, but somehow it is terribly flaky and a lot is down to luck – or sticking to Internet Explorer and Windows.

The latter does not sit well with me. Firefox has amazing accessibility opportunities and all extensions I have encountered so far are free. Why should someone have to pay lots of money just to access the web when I can buy a 100 pound (or 200 US Dollar) laptop, get Linux and Firefox or Opera and go for it? Apple computers come out-of-the-box with voiceover as a screen reader, why not Windows?

Anyways, back to the focus issue. Whilst there is a good visual clue that something is focused in Flash this does not necessarily mean assistive technology will know about it. This is a problem, and a lot of people also don’t like this to happen – do you know how many times you get asked as a web developer to “remove the ugly border around links when you click them”?

Avoiding the focus issue by creating the browser-plug-in communication

The way around most of these issues is to allow your Flash to be controlled by JavaScript from the outside by providing an API into your players. This may sound totally ludicrous – after all accessibility experts have told you for years that both Flash and JavaScript are evil, right? Bear with me though, cause here are the reasons for advocating both JavaScript and Flash interacting.

Progressive Enhancement

Using JavaScript I can test what a browser can do before I ask it to “make it so”. I can add a link to a video file in the browser and only add a player for it when and if the browser supports it (or even the user specifically asks for it).

Embedding Flash can be tricky, as – you might have guessed – not all browsers do it the same way. Therefore it is a good idea to use an abstraction tool like SWFObject to include your Flash. This means we are already depending on scripting to embed Flash, which has the practical upshot that we automatically know that both are available.

Solid interaction patterns

Using the DOM and JavaScript I can create HTML elements that work with all kind of assistive technology. Instead of hoping that keyboard users can access my Flash content I use what browsers already have – links, buttons and form fields – to interact with the it.

Possibly fixing the keyboard access problem

Technically you should be able to fix the tabbing-into-Flash issue by having a method in your Flash movie that focuses a certain element. Then you can create a text link that calls this method to “skip into Flash”.

Easing maintainbility

Developers that know HTML, CSS and JavaScript are not hard to find (well, really good ones are). You also don’t need Flash installed to edit HTML, CSS or JavaScript. Therefore it does make sense in terms of maintenance to keep as much of the constantly changing elements of an application or player outside of Flash. Having for example all your text labels in an external XML file makes it much easier to localize your player. If you mix Flash and HTML you will most likely have a system to maintain the HTML with. You shouldn’t have to use yet another one to maintain your Flash apps.

Piggy-backing on the browser

When the first Flash applications came out people talked about a revolution in web-based Rich Interface Applications. Instead of having to show hard-to-style form elements and non-scalable graphics and links we had a vector-based interface at our disposal and could create any application and interface element we wanted to create. Somehow Ajax put a dent into this some years later (and it is pretty amazing how many Ajax problems can be solved when you look at solutions Flash developers found for them long ago) and one of the reasons was that the HTML interface – whilst ugly and hard to style consistently – simply works.

HTML is much more than just a visual representation. It is an interactive layer describing what certain elements are and what browsers should do with them supporting a multitude of input devices.

Furthermore the browser notifies all kind of other software when users interact with HTML elements like links and buttons. Just try to create something as seemingly easy as a scroll-bar yourself and analyze all the interactivity both in terms of content to scroll and keyboard controls to support – it is not easy.

Therefore it does make sense to use HTML elements to drive Flash players – the base user interactivity has already been created and tested for you by browser manufacturers.

What to use then? How do you write a player?

The good news is that you don’t have to write your own player, as there are several options already out there.

Even if you don’t want to use the players themselves you can in the JW case look at the source and get your inspiration from that.


  • Use what works – offer a description of the movie, maybe a thumbnail and link to the movie file explaining the type and the size inside the link
  • Use JavaScript and SWFObject to turn this into a playable movie when and if the browser supports it
  • Instead of relying on the player controls to be accessible, create HTML controls that drive the player using the DOM and a player API.
  • We will create proper code examples for all of this at Scripting Enabled.

I have to thank Antonia Hyde, Mike Davies, Artur Ortega, Benjamin Hawkes-Lewis, Jacob Seidelin, Ian McBurnie, Steven Webster, Dirk Ginader and Jeroen Wijering for great input and their research on the matter. I’ve been asked to put this together by Kath Moonan/AbilityNet and Julie Schiller/BBC and hope this will be beneficial for all.

Making YouTube easier and more accessible (updated 12/06)

Thursday, June 12th, 2008

Warning: The YouTube API is flaky at the moment, so there might be some outages!

At this year’s Accessibility2.0 conference in London Antonia Hyde from United Response asked the audience for technological solutions to make the social web easier accessible for people with learning disabilities.

Her presentation Rich Media and web apps for people with learning disabilities is available on slideshare.

Whilst not being able to tackle all the issues mentioned (probably the biggest one being captioning) I took some time to play with the YouTube API to create a much easier interface to watch videos. The following screenshot shows the Easy YouTube Player in action:

Easy YouTube player showing a video

Using the player

You can use the player in several ways, the easiest is to just copy and paste a youtube url in the url field. However, there is also a sort of REST interface that allows you to do more:

Shows the player without any movie loaded, empty search fields and playlists.

Pre-loads the video of this YouTube address and shows the preview image in the player.

Performs a search on YouTube for the term panda and shows links to the videos in the playlist on the right. You can use more than one search word by adding them with a “+”. For example:

One last option you have is to bookmark certain videos on and tag them for a user. In order to show these videos as a playlist you need to provide your user name and the tag separated by a dash. For example my user name on is “codepo8” and I bookmarked some videos with the tag “easyyoutubeplayer”. The following link will show them all in the playlist:

You can mix and match the different options. If you for example want to show a video and perform a search for other videos you can use:


The full player documentation with instructions on how to host the player yourself is available in the docs folder Easy YouTube Player documentation.


You can download the player with all the demo files here:



  • complete re-write of code
  • new buttons – glass were too complex
  • added video size control
  • added search and playlist support
  • added address field to send to friends
  • player now template driven – no more changes in main code needed
  • added documentation
  • added RESTful interface


  • moved buttons to the bottom of the player
  • text is now below the buttons and has an invisible extra “use this button to ” text for screen readers
  • pause button now toggles pause/play
  • mute button now toggles mute/sound
  • the URL of the buttons is not an anchor but a url now (that goes nowhere, but this is just to read out the right command)
  • removed the “current highlight” state
  • added a volume level indicator (as a visual bar and a hidden form field)

Problems with the YouTube Chromeless player being unavailable? – Change the URL

Thursday, June 12th, 2008

I am almost ready to release v2 of the easy youtube player but out of a sudden everything stopped working. I thought at first my own developer key got banned for incessant re-loading locally, but even Google’s own example is broken.

The solution to it seems to be to change the URL of the embed code:

// broken
swfobject.embedSWF(‘ ... ‘)
// working
swfobject.embedSWF(‘ ... ‘)

Thanks do d.kunchev who pointed this out on the mailing list

YouTube now with annotations – can we get those as an API please (captioning)?

Wednesday, June 4th, 2008

YouTube just released a new feature for video content generators: annotations. As you can see in this example video of someone jumping out of an aircraft the annotations show up whereever you want to put them on the screen and are time-based. You can even add links and hotspots to other videos and search results which means you can do interactive games using several videos.

Now this is all cute and nice, but what I’d want is API access to these annotations. This would allow us to provide not only captioning of the video for the hard of hearing but also information for blind visitors. I’ve written about this before, you can easily create an interface to have timed captioning on youtube but playing the captions back is trickier as you have no means of syncing the video (if the video buffers in between the captions and the video get out of sync).

Now, if YouTube came up with an API access to these captions that fires an event every time a new caption starts with the type of caption and its text value, it would be dead easy to update a hidden form field with that text (or an ARIA live region) to provide a poor man’s captioning and information for the hard of hearing.

YouTube could become both a larger consumer faced product by enabling more disabled visitors to gain access and a means of captioning video that is intuitive and easy to use.

I’d be happy to help out!