Into the web multiverse
Wednesday, September 25th, 2019This is the article version of a talk I gave at Web Unleashed in Toronto and Halfstack in Vienna. You can also check the slides here .
I love the web, and it has been only good to me from a career point of view. I am also worried about where it is going. I am worried about how we keep advocating for it in the same manner we did when I discovered it. And maybe it is time to re-evaluate our approach to “saving the web”.
Over the last 20 years I’ve worked on the web on various products and in different companies. I’ve built web sites, templates for eCommerce platforms and enterprise CMS. I thought this was best for the web at the time as the latter had a worrying tendency of creating horrible markup. I then moved on to working on big internet portals like government sites and subsequently Yahoo. There I then pivoted into building things for developers to create faster and cleaner products. Examples of that were internal build tools, templates and subsequently library components for things like YUI. Still, I felt something was missing so I went to work on browsers and the documentation around them. And now I am in a space where I work on developer tools in browsers and the ones we use to build web products.
I like this, as it shows me that there are lots of different ways to support the web and that we are flexible in our careers.
The web in our eyes
When it comes to promoting the web, we keep talking about its benefits:
- It is decentralised – not one company owns it and it isn’t beholden to its values and needs
- It is independent of platform and device
- It is accessible to all
- It has a low barrier to contribution
- It gives you the ability to own your communication channel
- It is standardised based on open platform technologies
- It can be the sum of human knowledge as it is easy to contribute to and to archive on it.
We cherish these ideas and want them to prevail. Thus, we keep communicating and defending our values when it comes to the web. Everything people do on the web should strive to be:
- Open and free – we try to battle the closed app infrastructure
- Read and write – in a perfect scenario, all services should also offer APIs to access content
- Editable – if you find a mistake, it would be great to give feedback and fix it
- Inclusive and accessible – we shouldn’t lock out people because of their physical or educational state
- Publicly available – we hate paywalls and interstitials
- Based on open standards – we’re tired of having to cater to the needs of companies instead of the plaform
We are also all about giving users and ourselves choices. Our world consists of:
- Different browsers customisable with lots of add-ons.
- Different editors – there is no “one IDE” for the web
- Different form factors and platforms (desktops, laptops, tablets, mobile, voice assistants…)
- Different abilities, which means both device capabilities and user needs driving our interface choices
The web is a huge success
It is hard to imagine a world without the World Wide Web. Web technologies are standardised and browser differences are negligible. Open source has matured from a niche to the technology that runs the web – most servers are Linux based. Documentation is plentiful, free and interactive. The MDN Web Docs are a shared platform where all the big players of the web contribute documentation. We have Can I use to tell us what features are safe to implement and lately even can I email to document the (sorry) state of HTML email.
So, what are we actually fighting for? What is the problem we are trying the solve?
The web is a huge mess
If we take a look at the web of today with our quality and value goggles, things don’t look rosy at all.
- There is a blatant lack of basic accessibility. Webaim did an analysis of the biggest one million web sites and the results are terrible. Almost all sites fail the most simple and easy to check tests.
- Security problems are another big issue. In the past it was annoying enough to have some spammers trying to inject a backlink. Nowadays it gets a lot more costly when your web site starts mining bitcoins in the background. Almost every month we have reports of leaks of user data.
- Performance issues are also rampant. It is not uncommon that my beefy laptop starts spinning up the fan when surfing a news portal. There is no lack of great presentations and articles explaining how to optimise our code. Developer tools give us more insights than ever where the memory and CPU hogs are. And yet a huge amount of people on the web are blocked out because of performance problems.
- This also leads to a perceived need for blockers to keep the web safe and a lot less annoying. Tracking prevention is a topic all browser vendors work with right now. A huge proportion of our end users couldn’t even think of a web without ad-blockers any longer.
Why is the web in such a state?
When it comes to pointing fingers and finding someone to blame about the current state of affairs there is no lack of opinions. The following aren’t mine, but I’ve seen oodles of presentations and posts with these or similar topics:
- Users are terrible and have horrible setups
- The web stack is just not good enough
- Greedy corporations plaster the web with ads and see users as sheep to milk (?)
- We don’t have the right tools
- Tech bros build horrible things without consideration
- Aliens
Maybe it is time to find the fault in ourselves, or at least re-evaluate our attempts to make things better.
Counting misses instead of hits
Abraham Wald was a statistician in the second world war who was responsible for saving quite a few lives. He was called in to help with an analysis of planes that returned from the field in an effort to make them safer for pilots.
The team looked at the bullet holes on the planes and wondered where to add more plating to ensure that hits don’t cause as much damage.
The first, understandable course of action would be to fortify the planes where most bullet holes are. Wald, however, rightfully pointed out that this is a fallacy. He explained that despite the damage in these parts, the planes made it back. The planes that were hit in the spots that aren’t marked are the ones that didn’t make it back. Therefore it makes much more sense to add armour to the plane parts that showed no bullet holes.
(If you want to know more about this excellent example of out-of-the-box thinking, Daniel Siegel gave a talk about it and the outcome of this analysis became known as Survivorship Bias)
Why do I bring that up? Well, I found that when it comes to advocating for the web, we’ve always had a bit of a romantic or absolute notion. We fondly think back of the times where the web was easier. We keep wondering why people don’t follow the easy and sensible advice we talk about.
Having been a part of this for a long time, I wonder if we are not falling into the trap of remembering the good things and repeating them as a goal. If we are honest, then there never were any “good old days” of the web. The work of the W3C and the WASP was to counterbalance proprietary solutions for the web from the very beginning. People saw the web as a rendering target even in the old days. Back then it wasn’t React or complex frameworks but solutions like enterprise CMS, Frontpage or even Dreamweaver. Maybe it is time to stop telling people about what could be and listen to what they need instead. If we fill the gaps in the needs and wants of our audience rather than concentrating on where the web is attacked we may just be able to make it sturdier.
We have two ears and one mouth – maybe we should spend more time listening and less talking.
I am lucky to be working for an interesting project at the moment. My company is moving our browser over to the Chromium project for very pragmatic reasons. It turned out that offering an alternative browser that isn’t compatible with the market leader isn’t as exciting as we wanted it to be. Instead we listened to what people want and we looked at what they use. And that’s an ongoing thing. We spend a lot of time and effort doing user research. New product features should always be backed up by users explicitly demanding them. That way we don’t build things for the sake of building them or having one more than the competition. Instead, we put the needs of the customer over our drive to keep telling them what to use.
The interesting thing to me is that by taking a step back and listening you get all kind of insights. You learn about the diversity of your end users. Here are some of them we encountered with quotes on how they see the web.
The “keeping it current” user
This is a dream come true user for people who love the web. Our cutting edge code finds a welcome customer in this group. It is, however, not as big as we’d like it to be.
The “I am OK with what I have” user
Backwards compatibility is the most important thing we have to care about with that user. And whilst it seems that they are holding back the progress of the web, these are loyal users that watch ads or even go for subscriptions.
The “I just go online” user
The web isn’t exciting for these users. It is something you use at work, and at home you probably watch television or play a few games on a tablet. It feels like a shame that they got introduced to the web as a work tool, but if there is no desire, there is not much we can do.
The cautious user
This is an interesting one. We want people to question web content and not fall for phishing and malware download attacks. But we also want to make sure that they get the best experience and some of them get overboard in their blocking agenda.
The “Stockholm syndrome” user
We never want users to feel stupid or incapable of using our products. An overly complex, new feature can make people become defensive and long for simpler times. It isn’t our place to call them luddites, it is ours to make new technology and interactions not look scary.
Bitter pills to swallow – part one
If I look at the results of these user tests and interviews, there are a few things that come to light:
To different users, a browser is
- something they can’t control
- something they cherish and highly customize
- irrelevant, as all work OK enough
- nothing worth changing their habits for
Security and privacy aren’t things that people control, but trust in the platform to do for them.
This kind of goes against our belief in an open web driven by the people. And, if we are honest with ourselves, this shouldn’t be a surprise. Times have changed and the web isn’t a cool new thing. Instead it is akin to plumbing. We don’t care where the water is coming from and how the whole delivery cycle works. As long as there is fresh water when we open the tap. We only care about the workings when things go wrong. And that is how people feel about the web. It is a commodity. Instead of thinking about the inner workings it is more exciting to customise the consumption devices. Right now these are mobile devices and “smart” products but the market is trying hard to sell AR and VR as the next big thing.
Own the platform?
This also impacts our idea of the web as a read and write medium. When it comes to “owning the platform” it should be a simple step. It is easy to get a domain and hosting these days. There are a lot of CMS and static page generators that enable building a web presence.
But whenever we tell people about that, we forget about mentioning the annoyances. You don’t only need to set up the presence, you also need to maintain it. Any outdated setup is a potential attack vector. In the past this meant a few injected spam links, but these days, with a more capable web, things are more dangerous. It is not uncommon to have a bitcoin miner or malware download injections. Again, this means constant vigilance or to be part of a botnet.
Other things to consider are legal requirements like GDPR compliance and cookie consent. There’s a lot of misinformation and confusion around these matters.
A bigger concern to me is that the web isn’t quite the simple and instantly gratifying platform it used to be. I was mesmerised how easy it was to publish and how fun it was to get my first comments. I soon started dreading comments. I felt like I was part of an arms race of whack-a-mole deleting unwanted content.
Closed publication platforms these days are enticing. They come with spam protection, a great editor and gorgeous out-of-the box design. The first time I used Medium, it was a fun experience and the highlighting/commenting interface is excellent. Of course you can achieve the same with WordPress and self-hosting, but it is a lot of work and it doesn’t come with amazing stats visualisation either.
Finding a fix for the open web somewhere else…
I think it is fair to say that the storm towards the open web from the side of users isn’t quite a thing any longer – if it ever existed. Which means, that we, as people who care about the web need to change our ways. It might be up to us to fix the web. And that means taking ownership and caring for the needs of different constituencies. If we can’t expect users to care enough to fix the web – maybe we need to empower developers?
User tests conducted with developers was also interesting. It is tempting to see the avalanche of demands coming from Twitter and Slack as an indicator what developers want. As it turns out, even in the developer space the web has long ago outgrown our cozy little family of web do-gooders. During research, we discovered quite a bunch of different approaches to web development.
“This is a job” developer
It is fair to see web development as a job and the tools of your trade as something your company provides you with. It was interesting to see that outdated setups didn’t make these developers less effective. Many created a lot of important an well-used web content. Often there is a high level of customisation at play here. People still use Dreamweaver with lots of company sanctioned settings and templates. And doing well, so who are we to judge?
The “switched on” developer
These are people that we want, but kind of lost. New web features are interesting to them but they don’t trust in the speed of adoption. Hence they prefer libraries and trust them to deal with backwards compatibility. New programming paradigms and ways to develop are more exciting. Again, nothing that is despicable, but perfectly understandable.
The “I am here to deliver” developer
Pragmatism rules the world of these developers. Again, we found a blind trust in frameworks doing the right thing. In this case, accessibility is something the tool does and doesn’t warrant an extra effort. Not being sued was the driver, not creating accessible products. Still, if your goal is to get paid and move on this is perfectly sensible.
The “the web is broken” developer
I had horrible flashbacks to the beginning of my career witnessing this type of developer. Back then “real developers” scoffed at our attempts to make a “toy language” like JavaScript work in larger teams. The interesting thing is though that they do have a point. If you look at the richness of some of the stacks in closed environments (huge IDEs, Flash…) the web stack can look shoddily put together.
The “I’m here to learn” developer
This was the most annoying group to witness. We should be better at empowering and welcoming new people into our market.
Somehow bitter pills to swallow (2)
User research focused on developers taught us a few things. The caring developer who wants the web to be better is a tiny fraction of the market. For many, this is a job to make a living. And that is OK, we always claimed that the beauty of the web is that everyone can make a living from it. A bit more disheartening is how newcomers perceive the current web development space. They don’t find an empathic, open world, but an avalanche of choice and opinions.
The bigger insight though seems to be one that is sobering. Satisfying developer needs has no direct correlation to creating better user products. Of course there is a trickle-down effect of developers setting up machines for others. And even telling them about it. But there seems to be a problem reaching the bulk of developers instead of catering to a vocal minority. The oft-mentioned concept of getting the hearts and minds of influencers doesn’t seem to pay out as much as we think.
Why learn the basics?
Knowing the basics of web development seems not to be interesting any longer. And that’s not surprising, giving the forgiving nature of the web. Browsers can’t break the web – their forgiveness knows no bounds. A huge part of browser code is to ensure that outdated, terrible code still yields a usable result. Even worse, we can’t even remove non-standard code that many huge web products rely on. So, if it works, why bother learning why? And even when it doesn’t work, there are lots of resources to learn quick fixes. The success of W3Schools and also Stackoverflow isn’t that they tell you why something functions. It is that they give you copy and paste solutions that yield the result you want. And the browser can deal with the rest.
Our focus on constant innovation and making developers more effective also made “knowing the web” not a desirable skill. Instead, listing your interest in abstractions get you eyeballs on job boards. Abstractions promise faster turnaround and great products – after all, big companies use them, right?
There are strong indicators that inventing new technology isn’t the answer to preserving the web. Constant change in the developer space is a double-edged sword. We empower a small group of highly motivated people at the cost of overwhelming others. We innovate to fill a specialist need and call it a success because it is new instead of solving real problems. We don’t find out about real problems as we don’t ask people outside our bubble
So, what can we do as web lovers?
It may be prudent to push technology to the side for the moment and focus on what’s needed for our end users. We can’t expect them to care about all these things, so let’s organise them in level of importance. The first thing we need to do is fill the basic needs.
The “must haves” of any web product are:
- Availability – if it doesn’t load and render or if the domain isn’t available, we are stuffed.
- Accessibility – we are morally and legally required not to block anyone out. And it is good business to make ourselves available
- Security – users trust us by visiting us and give us information. Neither interaction should result in third parties listening in or malware coming down the wire
- Mutability to user needs – people will need to change the font size or use high contrast mode (3% of all Windows users – that’s a lot of people), others will want to use a “simple reader mode” as offered by the browser
Once we covered these basics, we can move on to add more quality to the product.
The next level of delivering on the web can be dependent on environment and won’t get applied to all users.
- Enhanced performance – loading non-essential content on demand, caching static assets on device
- Extended availability – using ServiceWorker to offer content offline, pre-caching next pages
- Customisability – allowing users to change the content to their needs or ask for more content as their connection and setup permits
The last step is to create a thoroughly enjoyable experience.
For a small part of our users we can do even more by fully tapping into the capabilities of their environment.
- Make full use of the platform – push notifications, PWA features, special features of the device…
- Predict use cases and prevent simple mistakes – we can enhance any text entry field these days to allow for human input and use natural language processing to understand them. We can allow scanning of credit cards and picking from browser data to automatically fill forms…
- Offer extended functionality – depending on what the platform has for us available we can log in using biometrics or use payment systems to offer more content.
By concentrating on the must, should and want of our approach, we also can prune the way we talk about our tools.
We love tools
There is no question that tooling of the web is amazing these days. We have so many great things to choose from and we spend a lot of time talking about and how to customise them. We have:
- Browser developer tools
- Editors, extensions and themes
- Terminal emulators and their configuration
- Deployment and optimization toolchains
- Abstractions, Components and libraries
When it comes to building for the web, there is the “trinity of web development”: Editor, Browser and Terminal. Each a chance to talk in length about our favourites and how these – and only these – make you an effective developer.
It is all a bit much…
I’m going to be honest here. I work in this field and I am overwhelmed and even a bit bored of having to chase the next amazing setup. I can’t even imagine what it must be to start out as a developer these days. We overwhelm people who want to start with the web – or just want to do their job. By demanding people to set up complex dev environments we inhibit diverse contribution.
What do I mean by that? The other day I wanted to contribute to an open source project that literally was a single HTML document. I forked it on GitHub and followed the instructions to set up the local development environment. Turns out, the build downloaded 150 megabytes of dependencies and I had to tweak a lot of my setup before I was able to edit a few words in the HTML document. Now, I have a 250Mbit connection and a great laptop. Imagine the disappointment of someone on a limited connection getting that huge barrier to entry. In the end, I edited the file on GitHub in the “raw” view and pushed my change through this way.
Maybe we went too far in assuming developer setups and connectivity. We care quite a bit about end user performance, but contributor performance is a thing you don’t hear much about. In other words, we don’t focus enough on human needs in our tools.
I am going to put on my product management hat right now and concentrate on a the things we can use to make this better. This is a very pragmatic approach looking at the things people use and what to use them for. Everything here is open source and cross platform and thus makes me happy. Everything that follows is also something anyone can contribute to, or – at least – file bugs and feature requests against. So, instead of offering yet another choice, we could contribute to these things and make the web better that way.
Node: the glue
Right off the bat, Node took over the market by storm. The idea of an event based server wasn’t new, but it was daring and it really filled a need. Node, in essence, is JavaScript on the server side. It fixed the issue of “what when JavaScript is turned off” by allowing to render components on the client and the server. YUI was one of the first libraries to play with that idea and a lot of people associated with Node worked at Yahoo at that time.
Nowadays it is fair to say that Node elevated JavaScript from “how does browser X break” to a solution for all kind of web tasks. You can build all kind of services with it. This allows for a highly customisable web infrastructure like servers and build tools. Where in the past you needed to learn C++ or Java or – at a stretch – PHP, now JavaScript is the answer. It should be mentioned though that writing JavaScript for Node is quite different to client-side scripting. But we learned a lot from branching out.
Chromium: the engine
Chromium is an open source, multi-platform browser engine. It does not only power the leading browsers, but also application frameworks like Electron. It also comes in a headless version that allows for automation of web tasks using tools like Puppeteer. It is important to point out that Chromium isn’t Chrome, but an open source project open to contributions from various players, not only Google. Chromium became the dominant browser platform not only because of the browsers built on it but also because it could be embedded easily in other products. And that’s an ongoing thing – the user interface on top of the engine is how browser maker differentiate.
Visual Studio Code: the “hackable” editor
VSCode is one of the big successes of Electron and open source in the web space. Building a text editor with web technology was a painful experience in the past. Quite a few JavaScript based code editors came and went. One big breakthrough was Adobe’s brackets and Github’s Atom editor. Both influenced VSCode, which took the extra step of adding Visual Studio’s Intellisense to a light-weight editor. VSCode is open source, multi platform and based on Electron/Chromium. It is extensible and customisable using JavaScript/TypeScript.
This meant that a community could grow around it quickly and now you’ll find extensions for almost anything. One feature that helps beginners a lot is that it includes a Terminal and Git version control. The former allows you to run build commands without leaving the editor and the latter takes the pain out of version control. Instead of teaching a class to “save your work, go to the Terminal and type these commands”, you see an “M” appear next to the file name. You then tell the class to write a good commit message and click a button. This teaches version control as a human communication tool and not a code sequence to follow. IDEs and editors were black holes in the past with no way to add functionality. VSCode is open, invites feedback and you can mess around with it yourself.
Webhint: testing and best practices
“Best practices” are a weird construct. There are definitely some truths that apply to any product. But the way you work can differ a lot from project to project. That’s why a lot of documentation on “how to do do things right” doesn’t reach the intended audience. It doesn’t apply to them and sometimes can feel like an extra step that doesn’t yield any sensible result. The same applies to “opinionated testing tools” that understandably put a stake in the ground telling people what it needs to be a good product. These are well-intentioned but invite discourse questioning their motives. When a search engine punishes bad code by listing the results lower, that is a mighty stick. But it doesn’t mean people agree with the decision and it invites others to question if a company should have that much power.
There is no doubt though that over the years we learned a lot of things building web projects. And it is not sensible to assume that every new developer would know about them or follow them.
That was the inspiration for webhint, a fully customisable testing tool for the web. Using webhint you can test your web projects for:
- Accessibility issues
- Performance problems
- App readiness
- Compatibility with standards
- Common pitfalls to avoid
- Security issues
Webhint comes in a few flavours. You can either use the online scanner at webhint.io to test your products and get a full report. Or use the hint npm package to include its functionality into your own products or build processes.
Webhint is highly customisable, so instead of having a “this is how the web has to be” you can turn different tests on and off. You can also write your own hints specific to your needs and you can run it on your own servers or behind a Firewall.
Mixing and matching these tools
The great thing about these tools of the open web is that we can mix and match them to provide different audiences with help in their work.
Visual Studio Code and Webhint
Webhint is available as an extension to VSCode . This means you don’t have to test your product in the browser to find out what you did wrong. You learn about best practices and mistakes to avoid while you code. For example, webhint will flag up any image elements without an alt attribute. You don’t only get an error message, but you also get an explanation why this is a problem. This can avoid a lot of problems and speed up your development process. It also means that you don’t get a frustrating error, but the explanation why and how to fix it. That way you learn about best practices on the sly and remember it for the next time without switching context.
Webhint and Browsers
Webhint is also available as an extension for browsers to test your sites in the browser of your choice. You can use it for now in Microsoft Edge , Firefox and Chrome.
Webhint and Node
As mentioned earlier, Webhint is also a npm module which means you can use it in any Node environment, like your own projects or your continuous integration/deployment setup. It can automatically generate reports or act as a gatekeeper.
Elements for Code
We analysed the needs of several groups of developers and found that these mix and match ideas cover a lot of ground. But one thing was still missing. The browser developer tools are a great way to tweak the look and feel of a web site. Once you are happy with all the changes you did to the CSS, the problem was to get them back into your code. Even worse, how do you know which line of code in your source code creates that CSS (if you start f.e. with Sass)?
As the Chromium developer tools are a web app in themselves (several, actually), we thought it a good plan to pull them into VSCode. This is what the Elements for Code extension does.
- Edge/Chromium developer tools inside VS Code
- No need to switch to the browser to try a few things out
- No need to replicate the same functionality in both places
You can see what that means in the following screencast. You can not only tweak the CSS right in the editor with the familiar tools. You can also set breakpoints and watch events.
We are pretty early in this experiment and feedback is welcome!
Edge on Chromium
Now, as mentioned earlier, we are currently in the middle of releasing the Microsoft Edge browser based on Chromium. You can download and install the preview builds as a daily Canary build, a weekly Developer build and a much more stable Beta.
I am excited about this not only because it brings Edge to other platforms than Windows 10, but what it means to the Chromium project. In essence, we are adding a powerful player with lots of users to an important part of the web.
Microsoft reaches a lot of users on the whole spectrum described earlier. This meant that we needed to contribute a few things to Chromium that weren’t there before.
- Internationalisation of browser and developer tools (30% of VSCode users change the language)
- Accessibility enhancements to browser and developer tools
To name but a few…
With lots of developers added to the project we’re also excited about fixing bugs. Chromium is big and it was exciting to help clean a lot of it up.
So, take a look at what Edge is now and give us feedback by clicking the smiley icon. If you have specific feedback for the developer tools, there is also a smiley in those. You’ll create a report with a screenshot, the URL you encountered a problem and a form field to describe your issue. Part of my job is to go through this feedback and triage it accordingly.
Of course we shouldn’t stop innovating the web either. A new way to voice your concerns about it and get support from various browser makers and standards body members is The web we want . Take a look and see what people thought is missing and contribute your own ideas.
In summary: The web is a diverse place
It was exciting and humbling to take a step back and look at the web from a user angle for a change. I hope that you got some inspiration from it what to do next. We have a chance to make it easier for everyone to create quality products for all end users. We have open source platforms with open communication channels that power the web. We accumulated a lot of knowledge, why not add it to our tools to teach the creators of the web in context?