Christian Heilmann

Author Archive

Songsearch – using ServiceWorker to make a 4 MB CSV easily searchable in a browser

Friday, August 26th, 2016

A few days ago I got an email about an old project of mine that uses YQL to turn a CSV into a “web services” interface. The person asking me for help was a Karaoke DJ who wanted to turn his available songs into a search interface. Maintenance of the dataset happens in Excel, so all that was needed was a way to display it on a laptop.

The person had a need and wanted a simple solution. He was OK with doing some HTML and CSS, but felt pretty lost at anything database related or server side. Yes, the best solution for that would be a relational DB, as it would also speed up searches a lot. As I don’t know for how much longer YQL is reliably usable, I thought I use this opportunity to turn this into an offline app using ServiceWorker and do the whole work in client side JavaScript. Enter SongSearch – with source available on GitHub.

the songsearch interface in action

Here’s how I built it:

  • The first step was to parse a CSV into an array in JavaScript. This has been discussed many a time and I found this solution on StackOverflow to be adequate to my uses.
  • I started by loading the file in and covering the interface with a loading message. This is not pretty, and had I had my own way and more time I’d probably look into streams instead.
  • I load the CSV with AJAX and parse it. From there on it is memory for me to play with.
  • One thing that was fun to find out was how to get the currently selected radio button in a group. This used to be a loop. Nowadays it is totally fine to use document.querySelector(‘input[type=”radio”][name=”what”]:checked’).value where the name is the name of your radio group :)
  • One thing I found is that searching can be slow – especially when you enter just one character and the whole dataset gets returned. Therefore I wanted to show a searching message. This is not possible when you do it in the main thread as it is blocked during computation. The solution was to use a WebWorker. You show the searching message before starting the worker, and hide it once it messaged back. One interesting bit I did was to include the WebWorker code also as a resource in the page, to avoid me having to repeat the songsearch method.
  • The final part was to add a ServiceWorker to make it work offline and cache all the resources. The latter could even be done using AppCache, but this may soon be deprecated.

As you can see the first load gets the whole batch, but subsequent reloads of the app are almost immediate as all the data is cached by the ServiceWorker.

Loading before and after the serviceworker cached the content

It’s not rocket science, it is not pretty, but it does the job and I made someone very happy. Take a look, maybe there is something in there that inspires you, too. I’m especially “proud” of the logo, which wasn’t at all done in Keynote :)

New blog design – I let the browser do most of the work…

Tuesday, August 23rd, 2016

Unless you are reading the RSS feed or the AMP version of this blog, you’ll see that some things changed here. Last week I spent an hour redesigning this blog from scrarch. No, I didn’t move to another platform (WordPress does the job for me so far), but I fixed a few issues that annoyed me.

So now this blog is fully responsive, has no dependencies on any CSS frameworks or scripts and should render much nicer on mobile devices where a lot of my readers are.

It all started with my finding Dan Klammers Bytesize Icons – the icons which are now visible in the navigation on top if your screen is wide enough to allow for them. I loved their simplicity and that I could embed them, thus having a visual menu that doesn’t need any extra HTTP overhead. So I copied and pasted, coloured in the lines and that was that.

The next thing that inspired me incredibly was the trick to use a font-size of 1em + 1vw on the body of the document to ensure a readable text regardless of the resolution. It was one of the goodies in Heydon Pickering’s On writing less damn code post. He attributed this trick to Vasilis who of course is too nice and told the whole attribution story himself.

Next was creating the menu. For this, I used the power of flexbox and a single media query to ensure that my logo stays but the text links wrap into a few lines next to it. You can play with the code on JSBin.

The full CSS of the blog is now about 340 lines of code and has no dependency on any libraries or frameworks. There is no JavaScript except for ads.

The rest was tweaking some font sizes and colours and adding some enhancements like skip links to jump over the navigation. These are visible when you tab into the document, which seems a good enough solution seeing that we do not have a huge navigation as it is.

Other small fixes:

  • The code display on older posts is now fixed. I used an older plugin not compatible with the current one in the past. The fix was to write yet another plugin to un-do what the old one needed and giving it the proper HTML structure.
  • I switched the ad to a responsive one, so there should be no problems with this breaking the layout. Go on, test it out, click it a few hundred times to give it a thorough test.
  • I’ve stopped fixed image sizes for quite a while now and used 100% as width. With this new layout I also gave them a max width to avoid wasted space and massive blurring.
  • For videos I will now start using Embed responsively not to break the layout either.

All in all this was the work of an hour, live in my browser and without any staging. This is a blog, it is here for words, not to do amazing feats of code.

Here are few views of the blog on different devices (courtesy of the Chrome Devtools):

iPad:
Blog on iPad

iPhone 5:
Blog on iPhone 5

iPhone 6:
Blog on iPhone 6

Nexus 5:
Blog on Nexus 5

Hope you like it.

All in all I love working on the web these days. Our CSS toys are incredibly powerful, browsers are much more reliable and the insights you get and tweaks you can do in developer tools are amazing. When I think back when I did the first layout here in 2006, I probably wouldn’t go through these pains nowadays. Create some good stuff, just do as much as is needed.

Better keyboard navigation with progressive enhancement

Monday, August 15th, 2016
Keyboard

When building interfaces, it is important to also consider those who can only use a keyboard to use your products. This is a basic accessibility need, and in most cases it isn’t hard to allow for a basic keyboard access. It means first and foremost using keyboard accessible elements for interaction:

  • anchors with a valid href attribute if you want the user to go somewhere
  • buttons when you want to execute your own code and stay in the document

You can make almost everything keyboard accessible using the roving tab index technique, but why bother when there are HTML elements that can do the same?

Making it visual

Using the right elements isn’t quite enough though; you also need to make it obvious where a keyboard user is in a collection of elements. Browsers do this by putting an outline around active elements. Whilst dead useful this has always been a thorn in the side of people who want to control the whole visual display of any interaction. You can remove this visual aid by setting the CSS outline property to none, which is a big accessibility issue unless you also provide an alternative.

By using the most obvious HTML elements for the job and some CSS to ensure that not only hover but also focus states are defined we can make it easy for our users to navigate a list of items by tabbing through them. Shift-Tab allows you to go backwards. You can try it here and the HTML is pretty straight forward.

<ul>
  <li><button>1</button></li>
  <li><button>2</button></li>
  <li><button>3</button></li>
  …
  <li><button>20</button></li>
</ul>

example how to tab through a list of buttons

Using a list gives our elements a hierarchy and a way to navigate with accessible technology that a normal browser doesn’t have. It also gives us a lot of HTML elements to apply styling to. With a few styles, we can turn this into a grid, using less vertical space and allowing for more content in a small space.

ul, li {
  margin: 0;
  padding: 0;
  list-style: none;
}
button {
  border: none;
  display: block;
  background: goldenrod;
  color: white;
  width: 90%;
  height: 30px;  
  margin: 5%;
  transform: scale(0.8);
  transition: 300ms;
}
button:hover, button:focus {
  transform: scale(1);
  outline: none;
  background: powderblue;
  color: #333;
}
 
li {
  float: left;
}
 
/* 
  grid magic by @heydonworks 
  https://codepen.io/heydon/pen/bcdrl
*/
 
li {
  width: calc(100% / 4);
}
li:nth-child(4n+1):nth-last-child(1) {
  width: 100%;
}
li:nth-child(4n+1):nth-last-child(1) ~ li {
  width: 100%;
}
li:nth-child(4n+1):nth-last-child(2) {
  width: 50%;
}
li:nth-child(4n+1):nth-last-child(2) ~ li {
  width: 50%;
}
li:nth-child(4n+1):nth-last-child(3) {
  width: calc(100% / 4);
}
li:nth-child(4n+1):nth-last-child(3) ~ li {
  width: calc(100% / 4);
}

The result looks pretty fancy and it is very obvious where we are in our journey through the list.

tabbing through a grid item by item

Enhancing the keyboard access – providing shortcuts

However, if I am in a grid, wouldn’t it be better if I could move in two directions with my keyboard?

Using a bit of JavaScript for progressive enhancement, we get this effect and can navigate the grid either with the cursor keys or by using WASD:

navigating inside a grid of elements using the cursor keys going up, down, left and right

It is important to remember here that this is an enhancement. Our list is still fully accessible by tabbing and should JavaScript fail for any of the dozens of reasons it can, we lost a bit of convenience instead of having no interface at all.

I’ve packaged this up in a small open source, vanilla, dependency free JavaScript called gridnav and you can get it on GitHub. All you need to do is to call the script and give it a selector to reach your list of elements.

<ul id="links" data-amount="5" data-element="a">
  <li><a href="#">1</a></li>
  <li><a href="#">2</a></li>
…
  <li><a href="#">25</a></li>
</ul>
 
<script src="gridnav.js"></script>
<script>
  var linklist = new Gridnav('#links');
</script>

You define the amount of elements in each row and the keyboard accessible element as data attributes on the list element. These are optional, but make the script faster and less error prone. There’s an extensive README explaining how to use the script.

How does it work?

When I started to ponder how to do this, I started like any developer does: trying to tackle the most complex way. I thought I needed to navigate the DOM a lot using parent nodes and siblings with lots of comparing of positioning and using getBoundingClientRect.

Then I took a step back and realised that it doesn’t matter how we display the list. In the end, it is just a list and we need to navigate this one. And we don’t even need to navigate the DOM, as all we do is go from one element in a collection of buttons or anchors to another. All we need to do is to:

  1. Find the element we are on (event.target gives us that).
  2. Get the key that was pressed
  3. Depending on the key move to the next, previous, or skip a few elements to get to the next row

Like this (you can try it out here):

moving in the grid is the same as moving along an axis

The amount of elements we need to skip is defined by the amount of elements in a row. Going up is going n elements backwards and going down is n elements forwards in the collection.

diagram of navigation in the grid

The full code is pretty short if you use some tricks:

(function(){
  var list = document.querySelector('ul');
  var items = list.querySelectorAll('button');
  var amount = Math.floor(
        list.offsetWidth / 
        list.firstElementChild.offsetWidth
      );
  var codes = {
    38: -amount,
    40: amount, 
    39: 1,
    37: -1
  };
  for (var i = 0; i < items.length; i++) {
    items[i].index = i;
  }
  function handlekeys(ev) {
    var keycode = ev.keyCode;
    if (codes[keycode]) {
      var t = ev.target;
      if (t.index !== undefined) {
        if (items[t.index + codes[keycode]]) {
          items[t.index + codes[keycode]].focus();
        }
      }
    }
  }
  list.addEventListener('keyup', handlekeys);
})();

What’s going on here?

We get a handle to the list and cache all the keyboard accessible elements to navigate through

  var list = document.querySelector('ul');
  var items = list.querySelectorAll('button');

We calculate the amount of elements to skip when going up and down by dividing the width of the list element by the width of the first child element that is an HTML element (in this case this will be the LI)

  var amount = Math.floor(
        list.offsetWidth / 
        list.firstElementChild.offsetWidth
      );

Instead of creating a switch statement or lots of if statements for keyboard handling, I prefer to define a lookup table. In this case, it is called codes. They key code for up is 38, 40 is down, 39 is right and 37 is left. If we now get codes[37] for example, we get -1, which is the amount of elements to move in the list

  var codes = {
    38: -amount,
    40: amount, 
    39: 1,
    37: -1
  };

We can use event.target to get which button was pressed in the list, but we don’t know where in the list it is. To avoid having to loop through the list on each keystroke, it makes more sense to loop through all the buttons once and store their index in the list in an index property on the button itself.

  for (var i = 0; i < items.length; i++) {
    items[i].index = i;
  }

The handlekeys() function does the rest. We read the code of the key pressed and compare it with the codes lookup table. This also means we only react to arrow keys in our function. We then get the current element the key was pressed on and check if it has an index property. If it has one, we check if an element exist in the collection that is in the direction we want to move. We do this by adding the index of the current element to the value returned from the lookup table. If the element exists, we focus on it.

  function handlekeys(ev) {
    var keycode = ev.keyCode;
    if (codes[keycode]) {
      var t = ev.target;
      if (t.index !== undefined) {
        if (items[t.index + codes[keycode]]) {
          items[t.index + codes[keycode]].focus();
        }
      }
    }
  }

We apply a keyup event listener to the list and we’re done :)

  list.addEventListener('keyup', handlekeys);

If you feel like following this along live, here’s a quick video tutorial of me explaining all the bits and bobs.

The video has a small bug in the final code as I am not comparing the count property to undefined, which means the keyboard functionality doesn’t work on the first item (as 0 is falsy).

AMPed up

Wednesday, August 3rd, 2016

With yesterday’s coverage of more rumours that Google search on mobile will rank AMP versions higher than normal HTML ones (The Verge and Nieman Lab reported) I thought it high time to take a closer look at AMP.

I will follow up with more in-detailed information about this and I’m chatting to the team in Google about the project, but here is the simple news: this blog is now also AMP enabled and you can get the AMP version of any post by adding a “/amp” to the URL.

The great thing about this? I had nothing else to do but to use the WordPress plugin (the source of which is also available on GitHub).

With AMP being an alternative representation of HTML content and a subset of it, this was easy enough. And frankly, I like the way it looks even on Desktop :).

Why ChakraCore matters

Wednesday, July 27th, 2016

People who have been in the web development world for a long time might remember A List Apart’s posts from 16 years ago talking about “Why $browser matters”. In these posts, Zeldman explained how the compliance with web standards and support for new features made some browsers stand out, even if they don’t have massive market share yet.

fruit stand
Variety is the spice of life

These browsers became the test bed for future-proof solutions. Solutions that later on already were ready for a larger group of new browsers. Functionality that made the web much more exciting than the one of old. The web we built for one browser and relied on tried-and-true, yet hacky solutions like table layouts. These articles didn’t praise a new browser with flashy new functionality. Browsers featured in this series were the ones compliant with upcoming and agreed standards. That’s what made them important.

The web thrives on diversity. Not only in people, but also in engines. We would not be where we are today if we had stuck with one browser engine. We would not enjoy the openness and free availability of our technologies if Mozilla hadn’t showed that you can be open and a success. The web thrives on user choice and on tool choice for developers.

Competition makes us better and our solutions more creative. Standardisation makes it possible for users of our solutions to maintain them. To upgrade them without having to re-write them from scratch. Monoculture brings quick success but in the long run always ends in dead code on the web that has nowhere to execute. As the controlled, solution-to-end-all-solutions changed from underneath the developers without backwards compatibility.

Today my colleague Arunesh Chandra announced at the NodeSummit in San Francisco that ChakraCore, the open source JavaScript engine of Microsoft, in part powering the Microsoft Edge browser is available now for Linux and OSX.

Screen captures showing ChakraCore running inside terminal windows on Ubuntu 16.04 and OS X
ChakraCore on Linux and OS X

This is a huge step for Microsoft, who – like any other company – is a strong believer in its own products. It also does well keeping their developers happy by sticking to what they are used to. It is nothing they needed to do to stay relevant. But it is something that is the right thing to do to ensure that the world of Node also has more choice and is not dependent on one predominant VM. Many players in the market see the benefits of Node and want to support it, but are not sold on a dependency on one JavaScript VM. A few are ready to roll out their own VMs, which cater to special needs, for example in the IoT space.

This angers a few people in the Node world. They worry that with several VMs, the “browser hell” of “supporting all kind of environment” will come to Node. Yes, it will mean having to support more engines. But it is also an opportunity to understand that by using standardised code, ratified by the TC39, your solutions will be much sturdier. Relying on specialist functionality of one engine always means that you are dependent on it not changing. And we are already seeing far too many Node based solutions that can’t upgrade to the latest version as breaking changes would mean a complete re-write.

ChakraCore matters the same way browsers that dared to support web standards mattered. It is a choice showing that to be future proof, Node developers need to be ready to allow their solutions to run on various VMs. I’m looking forward to seeing how this plays out. It took the web a few years to understand the value of standards and choice. Much rhetoric was thrown around on either side. I hope that with the great opportunity that Node is to innovate and use ECMAScript for everything we will get there faster and with less dogmatic messaging.

Photo by Ian D. Keating