Showing multimedia fallback content when no supported source is found

There is nothing more frustrating than things going wrong without you knowing what happened. Things breaking with a very obvious reason are not as bad. Say you drop your phone and you see the display smashed – there is no question why the touch interface doesn’t work any more. But when nothing happens and everything should be OK, we get very cross very fast.

This is why code that isn’t defensive in testing for what it needs before accessing it annoys me. Worse even, when code just assumes everything is OK and doesn’t have an error clause that gives a fallback or at least tells me what is going on.

This is where HTML5 multimedia on a plain markup level is terrible. It is not easy to play sound or video on computers, not a trivial task indeed. However, I would still expect to get a fallback when things go wrong other than a grey box that does exactly nothing.

When I add an image to a document and specify an alternative text this text gets displayed when the image can’t be loaded (at least in Firefox). I can also add an “error” event handler to the image and it will get fired when the image fails to load. You can try it in this Demo on JSBin

  <img src="meh.jpg" alt="cute kitten photo">

var img = document.querySelector('img');
img.addEventListener('error', function(ev) {
  if (this.naturalWidth === 0 && 
      this.naturalHeight === 0) {
    console.log('Image ' + this.src + ' not loaded');
  }
}, false);

Not so with multimedia.

Say you add a video into the page and the browser can not play it because it can not be loaded or the browser does not support its type. All you get is a grey box that does not do anything. You don’t even get a “save as” link or anything like it. And to add to the annoyance, no error handler gets fired on the video element – although the video or audio element very much has one to use. What’s going on?

What if you offer a fallback?

Say we do the right thing and offer a sensible fallback for the video should it fail to load or the browser being incapable of playing video:

<video controls>
  <source src="dynamicsearch.mp4" type="video/mp4"></source>
  <a href="dynamicsearch.mp4">
    <img src="dynamicsearch.jpg" 
         alt="Dynamic app search in Firefox OS">
  </a>
  <p>Click image to play a video demo of 
     dynamic app search</p>
</video>

If your browser doesn’t support the video element at all you get a screenshot of the video and an explanation what to do – you can still download and watch the movie in any mediaplayer on your computer. This is great. It is also a beneficial thing to do as for example links on Facebook would get a thumbnail of the image next to them.

When the browser supports the video element but doesn’t support MP4 you are out of luck though. You get the broken grey box and not the fallback content. Wouldn’t it be great if the browser showed the fallback when the video can not be played? Not according to the standard. It is up to the implementer to develop a fallback for this. And here’s how that is done: you need to assign an error handler to the last source element in the video element. If the browser doesn’t find any playable source, this allows you to replace the video with its fallback content.

<video controls>
  <source src="dynamicsearch.mp4" type="video/mp4"></source>
  <a href="dynamicsearch.mp4">
    <img src="dynamicsearch.jpg" 
         alt="Dynamic app search in Firefox OS">
  </a>
  <p>Click image to play a video demo of 
     dynamic app search</p>
</video>

var v = document.querySelector('video'),
    sources = v.querySelectorAll('source'),
    lastsource = sources[sources.length-1];
lastsource.addEventListener('error', function(ev) {
  var d = document.createElement('div');
  d.innerHTML = v.innerHTML;
  v.parentNode.replaceChild(d, v);
}, false);

Not the most intuitive, but it works. Wouldn’t it be easier if the video element showed the fallback content when it can’t play a video or at least fire an error handler saying “no valid codec found”?

• Video element standard document