Christian Heilmann

Playing with Leapmotion for accessibility

Wednesday, July 31st, 2013 at 2:01 am

Today I spent a few hours in the Leap motion offices talking about the web possibilities of this cool new technology. On the way back we had an internet connectivity outage, so it was a great opportunity to start playing with some of the ideas. Here is a video of what I am working on.

In essence, this means:

  • Swiping your finger right will scroll the page down, left will scroll up. The speed of the swipe defines the amount of pixels scrolled
  • Making a clockwise circular gesture will jump to the next link, making a counter-clockwise one to the previous link
  • A tap gesture in the air will activate the link

The code is actually amazingly simple, once you use the Leap.js library:

(function(){
var linkindex = -1;
var links = document.querySelectorAll('a');
myController = new Leap.Controller("ws://localhost:6437/");
myListener = new Leap.Listener();
myListener.onFrame = function(controller) {
  var frame = controller.frame();
  var gestures = frame.gestures();
  if (gestures[0] && gestures[0]._state === 'stop') {
 
    if (gestures[0]._type === 'swipe') {
      if(gestures[0]._direction.x > 0) {
        window.scrollBy(0,gestures[0]._speed*2);
      } else {
        window.scrollBy(0,-gestures[0]._speed*2);
      }
    }
    if (gestures[0]._type === 'keyTap') {
      if (linkindex) { 
        links[linkindex].click();
      }   
    }
    if (gestures[0]._type === 'circle') {
      if (gestures[0]._clockwise === true) {
          updateIndex(1);
          if(links[linkindex]) {
            links[linkindex].focus();   
          };
        } 
      if (gestures[0]._clockwise === false) {
          updateIndex(-1);
          if(links[linkindex]) {
            links[linkindex].focus();   
          };
      } 
    }
  }
}
function updateIndex(add) {
  linkindex += add;
  if (linkindex < 0) {
    linkindex = 0;
  }
  if (linkindex >= links.length) {
    linkindex = links.length - 1;
  }
}
myController.addListener(myListener);
myController.enableGesture("keyTap", true);
myController.enableGesture("swipe", true);
myController.enableGesture("circle", true);
})();

This is now very rough and ready and I will try to wrap this is a bookmarklet to apply it on any web site.

Share on Mastodon (needs instance)

Share on BlueSky

Newsletter

Check out the Dev Digest Newsletter I write every week for WeAreDevelopers. Latest issues:

160: Graphs and RAGs explained and VS Code extension hacks Graphs and RAG explained, how AI is reshaping UI and work, how to efficiently use Cursor, VS Code extensions security issues.
159: AI pipelines, 10x faster TypeScript, How to interview How to use LLMs to help you write code and how much electricity does that use? Is your API secure? 10x faster TypeScript thanks to Go!
158: 🕹️ Super Mario AI 🔑 API keys in LLMs 🤙🏾 Vibe Coding Why is AI playing Super Mario? How is hallucinating the least of our worries and what are rules for developing Safety Critical Code?
157: CUDA in Python, Gemini Code Assist and back-dooring LLMs We met with a CUDA expert from NVIDIA about the future of hardware, we look at how AI fails and how to play pong on 140 browser tabs.
156: Enterprise dead, all about Bluesky and React moves on! Learn about Bluesky as a platform, how to build a React App and how to speed up SQL. And play an impossible game in the browser.

My other work: