I am now proud owner of a HoloLens. I am not officially trained up on it yet as a Microsoft Employee. But I wanted to share the first impressions of setting it up and using it.
These are my personal impressions and not an official stance by my company. I’m sharing my first excitement here. I hope can make some people understand what is happening here.
This is also just a user POV as I haven’t started developing for it yet, but this will happen soon – promised.
HoloLens is unique
First of all, it is important to understand that HoloLens is something pretty unique. Every time I mention it people start making comparisons to the Occulus or Vive, but those don’t work.
A high-powered, multi camera mobile on your head
HoloLens is a self-contained computer you wear on your head. You don’t need anything else. It is not a peripheral, there is no other computer or server necessary. This is important when considering the price. Many VR headsets are much cheaper, but they aren’t Mixed Reality and they need a hefty computer to run. It doesn’t even need an internet connection all the time. Just because you wear them on your head doesn’t mean you can compare these products on even ground.
You should plan coding for it like a mobile phone on your head in terms of CPU/GPU power. The specs are high, but the demand of the way it works are, too. If you build for HoloLens be conservative with the resources you need – you’ll make me happy. Waiting isn’t fun, even when it is a floating animation in your room.
Your natural movement is an event
When you’re calibrating your HoloLens and all the dog can think of is you holding a treat instead of using the “bloom”
HoloLens is a system that uses natural motion of your head and body to explore an augmented space. This means you don’t lose connection to the real world – you still see it through the device. What you get is a constant analysis of your surroundings and Holograms overlayed on it. You open apps and either use them floating before you or dock them to a wall to use later when you look at said wall. You distribute your work space in your living space – without needing to go to IKEA to buy furniture.
This means the way you move and where you look become events software can interact with. The “gaze” gesture, which is “looking at something” is akin to a hover with a mouse. The “air tap” gesture is a click or submit.
That way the relative small size of you viewport compared to Occulus or Vive is less relevant. You’re not stuck in it as your viewport follows your head movement. You’re not supposed to have a whole takeover. HoloLens is there to augment the world – not replace it.
Your whole body is now an event trigger. Instead of learning keyboard shortcuts, you learn gestures. Or you can use your voice.
Gestures vs. Voice Recognition
You can use your hands to select and interact with things. Or you can say “select” to interact and “next” to move on in menus. Voice recognition is always on and Cortana is just one “Hey Cortana” away. You can use it to open apps, search the web, research, all kind of things. It still feels odd to me to talk to my phone. I am on Android, maybe Siri is a better experience. It feels more natural to talk to a voice in a space of apps I distributed around my house.
Spatial sound
HoloLens has a lot of speakers built in which allows you to hear sounds from all directions. This is pretty amazing when it comes to games like RoboRaid:
And even more so in Fragments:
https://www.youtube.com/watch?v=9ULJV4G5XkA
When using the speakers, there is not much privacy though. It is pretty easy to hear what HoloLens says when you are close to someone wearing it.
If you want a keyboard, you can have one
If you enter a lot of text into web sites or something similar, you can also pair a bluetooth keyboard. Or clicker, or whatever. At first it annoyed me to enter my pretty secure passwords in a floating keyboard. But the more I got used to HoloLens interaction, the easier it became.
A whole new way of interaction
I’m not a big fan of VR because I am prone to get nauseous if the frame rate isn’t perfect. I am also getting car sick a lot, so it isn’t something to look forward to. I also feel confined by it – it fills me with dread missing out on things around me whilst being in a virtual space. I don’t like blindfolds and earplugs either.
The only discomfort I felt from HoloLens is having something weighing close to a kilogram on your head. But you get used to it. At the beginning you will also feel your fingers cramp up during air tapping and your shoulders hurt. This means you are doing it wrong. The more natural you move, the easier it is for HoloLens to understand you. An air tap doesn’t need full movement. Consider lifting your finger and pointing at stuff. Just like interrupting a meeting.
An outstanding onboarding experience
What made me go “wow” was the way you set up and start working with the HoloLens. The team did an incredible job there. The same way Apple did a great job getting people used to using a touch device back when the iPhone came out. Setting up a HoloLens is an experience of discovery.
You put on the device and a friendly “hello” appears with Cortana’s voice telling you what to do. You get to set up the device to your needs by calibrating it to your eyes. Cortana tells you step by step how to use the gestures you need to find your way around. Each step is full of friendly “well done” messages. When you get stuck, the system tells you flat out not to worry and come back to it later. It is an enjoyable learning experience.
How I use it
Right now, I have my kitchen cupboards as my work benches. Edge is on one of them and next to it is my task list of the day. I have a few games on the other side of the room. When it comes to Holograms, there is a cat on our dog’s bowl and a Unicorn above the bed to give us nice dreams. Because we can.
Skype is pretty amazing on HoloLens:
Some niggles I have
It is important to remember a few things about the HoloLens:
It isn’t a consumer device but for now a B2B tool. On the one side there is the high price. And there is a focus on working with it rather than playing games.
It is not an outside device. HoloLens scans your environment and turns it into meshes. After it created the meshes it stores them in “spaces” avoiding the need to keep scanning. Outside this means a constant re-evaluating of the space. This is expensive and not worth-while. So there is no danger of a re-emergence of the annoying Google Glass people in the street. It stilll is disconcerning to look at someone not knowing if they film you or not.
I agree with a lot of other people that there should be a way to have several user accounts with stored calibration info on a single HoloLens. Whilst you can share experiences with other HoloLens users, it would be great to hand it over without recalibration and giving someone else access to my Windows account.
There should be a way to wipe all the Holograms in a space with a single command. When you let other people play with your device you end up with lots of tigers, spacemen and all kind of other things in your space that you need to delete by hand.
Whilst it is easy to shoot video and take pictures of your experience, the sharing experience is very basic. You can store it to OneDrive or Facebook. No option to mail it or to add Twitter. That said, Skype helps with that.
This is truly some next level experience
I am sure that there are great things to come in the VR/AR/MR space. Many experiences might be much more detailed and Hi-Fi. Yet, I am blown away by the usefulness of this device. I see partners and companies already use it to plan architectural projects. I see how people repair devices in the field with Skype instructions from the office. I get flashbacks to Star Trek’s Holodeck – something I loved as a teen.
It is pretty damn compelling to be able to use your physical space as a digital canvas. You don’t have to leave your flat. And you don’t run the danger of bumping into things while you are off into cyberspace. It is augmentation as it should be. In a few years I will probably chuckle at this post when my cyber contacts and ear piece do the same thing for me. But for now, I am happy I had the chance to try this out.
Posted in General | Comments Off on First impressions of my HoloLens
It is amazing to work on the web. We have a huge gift in the form of a decentralized network of devices. A network that helps people around the world to communicate and learn. And a huge resource of entertainment on demand. A gift we can tweak to our needs and interests rather than unwrap and consume. Some of us even know the technologies involved in creating interfaces of the web. Interfaces that empower others to consume content and become creators.
And yet there is a common feeling of failure. The web seems to be never good enough. Despite all our efforts the web doesn’t feel like a professional software platform. Our efforts to standardise the web to allow becoming a “web developer” easier seem not to work out. We keep re-defining what that even is.
Lately a lot of conferences have talks about mental health. Talks about us working too much and not going anywhere. Talks about how fragmented, insecure and unprofessional our tech stack is.
There’s been a meme going around for a while that summarises this feeling quite well. It is about the game Tetris:
“If Tetris has taught me anything, it’s that errors pile up and accomplishments disappear”
There is some truth to that. We worked for almost two decades on the web and the current state of it isn’t too encouraging. The average web site is too big to consume on mobile devices and too annoying to do so on laptops. We limit things too much on mobile. We cram a proverbial kitchen sink of features, code and intrusive advertising into our desktop solutions.
Let’s see what’s going on there and how we can shift ourselves away from this Tetris mindset. I’ll start by zooming out and talk a bit about the way we communicate about technologies.
Back when this was all fields…
One of my older tweets – This was my old computer setup. Notice the photos of contacts on the wall.
When I started using computers my setup was basic by current standards. And yet I didn’t use the computer to play. I wanted to use it to communicate. That’s why I reached out to other creators and makers and we shared the things we wrote. We shared tools that made our life easier. We got to know each other writing long notes and explanations. We communicated by mail, sending floppy disks to each other. Once we got to know each other better, we shared photos and visited each other in person. It was a human way to create. Communication came at a high price and you had to put a lot of effort into it. Which is why we ensured that it was high quality, cordial communication. You had to wait weeks for an answer. You made sure that you said everything you wanted to say before you created the package and mailed it.
Speed cheapens communication
Fast forward to now. Communication is a dime a dozen. We’re connected all the time. Instead of spending time and effort to get to know others and explain and lead, we shout. We aim to be the most prolific communicator out there. Or to prove someone wrong who is loud and prolific. Or make fun of everything others spent a lot of effort on.
The immediacy and the ubiquity of communication is cheapening the way we act. We don’t feel like putting up much effort upfront. A stream of information gives you the impression that it is easy to undo a mistake right after you made it. But the constant barrage of information makes others in the conversation tune out. They only prick up their ears when something seems to be out of the ordinary. And this is where polarisation comes in. A drive to impress the audience in a positive way leads to unhealthy competitiveness. Giving out strong negative messages leads to witch hunts and Twitter drama. The systems we use have competition built in. They drive us having to win and release the most amazing things all the time. Gamification is what social networks are about. You get likes and views and retweets and favorites. Writing more and interacting more gets you to a higher level of access and features. You’re encouraged to communicate using in-built features like emoji, GIFs and memes. It is not about accumulating sensible content and finding solutions. It is about keeping the conversations and arguments going. All to get more clicks, views and interactions to show us more ads – the simple way to monetise on the web.
What this model brings is drama and dogma. The more outrageous your views, the more interaction you cause. The more dogmatic your approach, the more ammunition you have proving others wrong. It is not about finding consensus, it is about winning. Winning meaningless internet points and burning quite a few bridges whilst you do it. And it is addictive. The more you do it, the higher the rush to win. And the more devastating the feeling of non-accomplishment when you don’t win.
The scattered landscape of today
Let’s zoom in again into what we do as web developers. Web development is a spectrum of approaches. I’ll try to explain this using Tetronimos (that’s the official name of Tetris blocks).
All the Tetronimos, depicting the spectrum of web development approaches, from trusting and conservative to innovative and controlling
On the one end we have a conservative group that doesn’t trust any browser or device and keeps it all on the server. On the other we have people who want to run everything on the client and replace web standards with their own abstractions. Conservatives allow the user to customise their device and environment to their needs. The other extreme is OK to block users who don’t fulfil certain criteria.
The following are not quotes by me. I encountered them over the years, anonymised and edited them for brevity. Let’s start.
The right, conservative block…
This is a group of people that most likely have been around the web for a long time. They’ve had a lot of promises and seen new technology proving too flaky to use over and over again. People that remember a web where we had to support terrible environments. A web where browsers were dumb renderers without any developer tools.
Here’s some of the quotes I keep hearing:
You can’t trust the client, but you can trust your server.
Web standards are good, but let’s not get overboard with fancy things.
bq/It is great that there are evergreen browsers, but that doesn’t apply to our customers…
This approach results in bullet-proof, working interfaces. Interfaces that are likely to bore end users. It seems clumsy these days to have to re-load a page on every interaction. And it is downright frustrating on a mobile connection. Mobile devices offer a lof of web browsing features using client-side technologies. What’s even worse is that this approach doesn’t reward end users who keep their environment up to date.
The right, conservative piece…
These people are more excited about the opportunities client-side functionality give you. But they tread carefully. They don’t want to break the web or lock out end users based on their abilities or technical setup.
Some quotes may be:
Progressive enhancement is the way to build working solutions. You build on static HTML generated from a backend and you will never break the user experience.
You have no right to block any user. The web is independent of device and ability. It is our job to make sure people can use our products. We do that by relying on standards.
Web products do not have to look and work the same in every environment.
Working, great looking interfaces that vary with the environment.
This is the best of both worlds. But it also means that you need to test and understand limits of older environments. Browser makers learned a lot from this group. It wanted to make the web work and get more control over what browsers do. That helps standard bodies a lot.
The Square
This is the group most concerned about the HTML we create. They are the people advocating for semantic HTML and sensible structures.
Some common phrases you’d hear from them are:
No matter what you do, HTML is always the thing the browser gets. HTML is fault tolerant and will work wherever and forever.
Using semantic HTML gives you a lot of things for free. Accessibility, caching, fast rendering. You can’t lose.
HTML is the final truth, that’s 100% correct. But we dropped the ball when HTML5 leap-frogged innovation. Advanced form interactions aren’t as reliable as we want them to be. Adding ARIA attributes to HTML elements isn’t a silver bullet to make them accessible. Browser makers should have spent more time fixing this. But instead, we competed with each other trying to be the most innovative browser. When it comes to basic HTML support, there was hardly any demand from developers to improve. HTML is boring to most developers and browsers are forgiving. This is why we drown in DIVs and SPANs. We keep having to remind people of the benefits of using real buttons, anchors and headings.
The straight and narrow…
This is a group advocating for a web that allows developers to achieve more by writing less code. They don’t want to have to worry about browser differences. They use libraries, polyfills and frameworks. These give us functionality not yet supported in browsers today.
Some famous ways of saying this are:
Browser differences are annoying and shouldn’t be in the way of the developer. That’s why we need abstraction libraries to fix these issues.
Understanding standards while they are in the making is nothing we have time for.
$library is so much easier – why don’t we add it to browsers?
There is no doubt that jQuery, Modernizr and polyfills are to thank for the web we have today. The issue is that far too many things depend on stop-gap solutions that never went away. Developers became dependent on them and never looked at the problems they solve. Problems that don’t exist in current browsers and yet we keep their fixes alive. That said, browser makers and standard creators learned a lot from these abstractions. We have quite a few convenience methods in JavaScript now because jQuery paved the way.
The T-Block
In Tetris, this is the most versatile block. It helps you to remove lines or build a solid foundation for fixing a situation that seems hopeless. On the web, this is JavaScript. It is the only language that spans the whole experience. You can use it on the server and on the client. In an environment that supports JavaScript, you can create CSS and HTML with it.
This leads to a lot of enthusiastic quotes about it:
I can do everything in JavaScript. Every developer on the web should know it.
Using JavaScript, I can test if something I wanted to happen happened. There is no hoping that the browser did it right – we know.
JavaScript was the necessary part to make the web we have now happen. It can make things more accessible and much more responsive. You can even find out what’s possible in a certain environment and cater a fitting solution to it. It is a fickle friend though, many things can go wrong until the script loads and executes. And it is unforgiving. One error and nothing happens.
The innovation piece…
This group considers JavaScript support a given. They want to have development tool chains. Tooling as rich as that of other programming environments.
This leads to quotes like these:
It is OK to rely on JavaScript to be available. The benefits of computational values without reloads are too many to miss out on.
I don’t want to have to think about older browsers and broken environments. Frameworks and build processes can take care of that.
The concept of starting with a text editor and static files doesn’t exist any more. We have so much more benefits from using a proper toolchain. If that’s too hard for you, then you’re not a web developer.
Web standards like CSS, JavaScript and HTML are conversion targets. Sass, CoffeeScript, Elm, Markdown and Jade gives us more reliable control right now. We should not wait until browsers catch up.
It is a waste of time to know about browser differences and broken implementations. It is much more efficient to start with an abstraction.
Developer convenience trumps end users experience here. This can result in bloated solutions. We know about that bloat and we create a lot of technologies to fix that issue. Browser makers can’t help much here. Except creating developer tools that connect the abstraction with the final output (sourcemaps).
The innovative blocker…
These are the bleeding edge developers. They want to re-invent what browsers and standards do as they’re not fast enough.
Common quotes coming from this end of the spectrum are:
Browsers and web standards are too slow and don’t give us enough control. We want to know what’s going on and control every part of the interface.
CSS is broken. The DOM is broken. We have the technologies in our evergreen browsers to fix all that reliably as we have insight into what’s happening and can optimise it.
This approach yields high fidelity, beautiful and responsive interfaces. Interfaces that lock out a large group of users as they demand a lot from the device they run on. We assume that everybody has access to a high end environment and we don’t cater to others. Any environment with a high level of control also comes with high responsibility. If you replace web technologies with your own, you are responsible for everything – including maintenance. Browser makers can take these new ideas and standardise them. The danger is that they will never get used once we have them.
The more innovative you are, the more you have to be responsible for the interface working for everybody. The more conservative you are, the more you have to trust the browser to do the right thing.
Every single group in this spectrum have their place on the web. In their frame of reference, each result in better experiences for our users. The difference is responsibility and support.
If we create interfaces dependent on JavaScript we’re responsible to make them accessible. If we rely on preprocessors it is up to us to explain these to the maintainers of our products. We also demand more of the devices and connectivity of our end users. This can block out quite a large chunk of people.
The less we rely on browsers and devices, the more we allow end users to customise the interface to their needs. Our products run everywhere. But are they also delivering an enjoyable experience? If I have a high-end device with an up-to-date, evergreen browser I don’t want a interface that was hot in 1998.
Who defines what and who is in charge?
Who defines who is allowed to use our products?
Who has the final say how an interface looks and what it is used for?
The W3C has covered this problem quite a long time ago. In its design principles you can find this gem:
If our users are the end goal, findings and best practices should go both ways. Advocates of a sturdy web can learn from innovators and vice versa. We’re not quite doing that. Instead we build silos. Each of the approaches has dedicated conferences, blogs, slack channels and communities. Often our argumentation why one or the other is better means discrediting the other one. This is not helpful in the long run.
This is in part based on the developer mindset. We seem to be hard-wired to fix problems, as fast as possible, and with a technological approach. Often this means that we fix some smaller issue and cause a much larger problem.
How great is the web for developers these days?
It is time to understand that we work in a creative, fast moving space that is in constant flux. It is a mess, but mess can be fun and working in a creative space needs a certain attitude.
As a storyboard artist in Disney you know that most of your work will be thrown away. The idea is to fail fast and fail often and create and try out a lot of ideas. Using this process you find two or three great ideas that make the movie a great story. (paraphrased)
Instead of trying to fix all the small niggles, let’s celebrate how far we’ve come with the standards and technologies we use. We have solid foundations.
JavaScript: JavaScript is eating the world. We use it from creating everything on the web up to creating APIs and Web Servers. We write lots of tools in it to build our solutions. JavaScript engines are Open Source and embeddable. NodeJS and NPM allow us to build packages and re-use them on demand. In ES6 we got much more solid DOM access and traversal methods. Inspired by jQuery, we have querySelector() and classList() and many more convenience methods. We even replaced the unwieldy XMLHttpRequest with fetch(). And the concept or Promises and Async/Await allow us to build much more responsive systems.
CSS: CSS evolved beyond choosing fonts and setting colours. We have transitions to get from one unknown state to another in a smooth fashion. We have animations to aid our users along an information flow. Both of these fire events to interact with JavaScript. We have Flexbox and Grids to lay out elements and pages. And we have Custom Properties, which are variables in CSS but so much more. Custom Properties are a great way to have CSS and JavaScript interact. You change the property value instead of having to add and remove classes on parent elements. Or – even worse – having to manipulate inline styles.
Progressive Web Apps: The concept of Progressive Web Apps is an amazing opportunity. By creating a manifest file we can define that what we built is an app and not a site. That way User Agents and Search Engines can offer install interfaces. And browsers can allow for access to device and Operating System functionality. Service Workers offer offline functionality and work around the problem of unreliable connectivity. They also allow us to convert content on the fly before loading and showing the document. Notifications can make our content stickier without even having to show the apps.
Tooling: Our developers tools have also gone leaps and bounds. Every browser is also a debugging environment. We haveh ackable editors written in web technologies. Toolchains allow us to produce what we need when it makes sense. No more sending code to environments that don’t even know how to execute it.
Information availability: Staying up up-to-date is also simple these days. Browser makers are available for feedback and information. We have collaboration tools by the truckload. We have more events than we can count and almost all publish videos of their talks.
It is time for us to fill the gaps. It is time we realise that not everything we do has to last forever and needs to add up to a perfect solution. It is OK for our accomplishments to vanish. It is not OK for them to become landfill of the web.
We have come a long way. We made the world much smaller and more connected. Let’s stop fussing over minor details, and show more love to the web, love to the craft and much more respect for another. Things are looking up, and I can’t wait to see what we – together – will come up with next.
You don’t owe the world perfection, but you have a voice that should be heard and your input matters. Get creative – no creativity is a waste.
Posted in General | Comments Off on Breaking out of the Tetris mindset
Yesterday morning I was lucky enough to give the opening keynote at the excellent Beyond Tellerrand conference in Dusseldorf, Germany. I wrote a talk for the occasion that covered a strange disconnect that we’re experiencing at the moment.
Whilst web technology advanced leaps and bounds we still seem to be discontent all the time. I called this the Tetris mind set: all our mistakes are perceived as piling up whilst our accomplishments vanish.
I will follow this up with a more in-depth article on the subject in due course, but for today I am very happy how well received the keynote was and I want to remind people that it is OK to build things that don’t last and that you don’t owe the world perfection. Creativity is a messy process and we should feel at ease about learning from mistakes.
Posted in General | Comments Off on You don’t owe the world perfection! – keynote at Beyond Tellerrand
Yesterday I was proud to be an invited speaker at the HackingUI masterclass where I presented about what Machine Learning and Artificial Intelligence means for us as developers and designers. I will be giving a similar talk tomorrow in Poland in my Code Europe talk.
The Masterclass is using Crowdcast to allow for discussions between the moderators and the presenter, for the presenter to show his slides/demos and for people to chat and submit questions. You can see the whole one hour 45 minutes session by signing up to Hacking UI.
It was exciting to give this presentation and the questions of the audience were interesting which meant that in addition to the topics covered in the talk I also managed to discuss the ethics of learning machines, how having more diverse teams can battle the issue of job loss because of automation and how AI can help combat bullying and antisocial behaviour online.
All in all there is a lot for us to be excited about and I hope I managed to make some people understand that the machine revolution is already happening and our job is it to make it benefit humankind, not work against it.
Posted in General | Comments Off on Talking about building the next interfaces with Machine Learning and AI at hackingui
Sometimes it is great to work for a large company that gives you opportunities to do some good. I am currently in New York to run a workshop with All Star Code in our offices. Originally Aaron Gustafson was supposed to also be part of this but he got sick. Instead I am happy to work with Rachel White, Claudius Mbemba and Adina Shanholtz to help All Star Code.
Originally All Star Code approached me to get a bulk order for Surfaces for their students to work with. When I heard that their curriculum was involving Git, Node, Web Development and Debugging in Browsers and the stack was Sublime Text and Chrome Devtools I offered a small change. So now we’ll be teaching the teachers of All Star Code’s next course how to use Visual Studio Code and do all the development and debugging inside that one. My main driver there was that Code is open source and thus the students don’t need to get another license.
If you wonder what All Star Code does you can head over to the Decoded Chats blog, where I interviewed Mahdi Shadkamfarrokhi, their head of curriculum.
You work for All Star Code. Can you give us a quick introduction what that is and what you do? (00:13)
How low are the numbers of developers that came from a minority background? What are the main reasons? (01:40)
Do you think that by teaching communication skills together with technological skills you become more interesting for someone with a less privileged background? Is selling technology skills as a part of a whole package more successful? (02:49)
The program has been running for quite a while. Is there a success story you are really proud of? (04:20)
You learn a lot by teaching as you can’t fake it – you have to know. Do you find that it is easier to keep your skills up-to-date by running this program? (04:46)
What are the biggest barriers for your students to get into development? Is it hardware access? Connectivity? The style and language of documentation out there? (06:14)
I learned a lot because when I started computers didn’t do much and you had to program. Do you think that nowadays kids are less inclined to learn as computers are more seen as a consumption device? (07:47)
There is a vast amount of online courses to choose from when it comes to learning how to program. Many of them decayed a bit after the first round of funding dried out. How do you find great and trustworthy resources? (10:10)
A lot of creativity happens on the web but these makers don’t know or don’t get into professional development. Where do you go to find people for your course? (12:04)
Do you see Open Source and services like GitHub to host, document and discuss your projects as an opportunity for newcomers? (14:49)
How can people help you? Are there ways to volunteer? (18:07)
I’m very excited to be working on this.
Posted in General | Comments Off on Working with All Star Code in NYC to empower minorities to get into development
