Christian Heilmann

Author Archive

So, you learned JavaScript – what now?

Tuesday, December 5th, 2017

Yesterday, I was asked by the Berlin chapter of Women Techmakers to give a talk at the graduation ceremony of their JavaScript Crash Course. I wanted to give a talk about the next steps you can take now you learned the basics of JavaScript in 2017 instead of repeating old ideas. This is what I came up with.

First of all, congratulations – you chose wisely. JavaScript has evolved from being a “toy language”, “real programmers” laughed at into the language that powers the web and beyond.

For better or worse. JavaScript isn’t a perfect language- if something like that even exists. But it has a few things that speak for it.

JavaScript is everywhere

First of all, it is by now ubiquituous and you can create a lot of amazing things with it.

  • You can use it in a browser environment to make web “things” that are highly interactive. These could be web sites that respond better and are more engaging when Javacript works. They could also be apps that users install and don’t even need to know that you used HTML. CSS and JavaScript to build them. They could even be complex games and VR experiences.
  • You can also go server-side with Node. Then you can use JavaScript to build APIs, services and even full-fledged servers. You can create build processes and automate a lot of boring tasks that in the past needed a server to run on.
  • You can use abstraction frameworks to publish on the web and create binary formats for iOS, Android and others. By starting with JavaScript, you can convert into many other things – something that isn’t sensibly possible the other way around.
  • Or you can go completely wild and build robots that get their instructions in JavaScript. You can automate operating systems. You can write extensions for browsers and you can script other applications.
  • You can publish to and take advantage of NPM, a vast resource of pre-build solutions you can mix and match to build your own – more complex – solutions. This is tempting, but there is also a danger of using too much and using things you don’t understand. So, whilst we’re in a package world with JavaScript, it makes sense to remember what JavaScript is and start there.

Mastering JavaScript isn’t easy, but getting started is simpler than with other languages. You are not dependent on a certain editor and you don’t need to compile it to create something that can run. Most important of all – you don’t need to spend any money to get started. Browsers are free. A lot of editors are free and many even open source. Above all, documentation is online and available to you.

You know the best thing though? I envy you!

I’ve been working with JavaScript more or less since it has been around and I’ve been carrying a lot of ballast with me.

The JavaScript definitive Guide book in comparison with JavaScript-The good parts

There is a running joke where you can see the “Definitive Guide to JavaScript” book next to Douglas Crockford’s “JavaScript – the good parts”. The latter is much, much smaller. This doesn’t mean that JavaScript is bad. It means that the versatility of JavaScript can be its own undoing. JavaScript runs everywhere. Over the years terrible environments left a track record of awful APIs and methods. The standardisation process of JavaScript has been a roller-coaster ride. The Definitive Guide explains that. It is a compendium of all that happened – good and “dear me, why would you even consider doing this in JavaScript?”. The Good Parts book sticks to the syntax and how to write pure JavaScript. Think of The Guide as the whole back catalogue of a band with all their sins of the past. Whereas “The Good Parts” is their hit single.

Starting with a much cleaner slate

And here is where you come in. I don’t even want you to think about the good parts of the language itself. That can come later – if deep-diving on a language’s syntax and constructs tickles your fancy. Right now, I envy you because you have a chance to hit the ground running. And I want you, as a newcomer, to not repeat the mistakes we did to get where we are.

So next up are a few things I want to introduce to you that you can use to take the next step as a JavaScript developer. These things can help you become more effective in what you do and also help you to take part and help the community. You’re very welcome to disregard this advice. You’re also welcome to challenge it – after all, this is what new voices are about. With the speed JavaScript evolves some things I tell you now will become obsolete, for sure. But I for one am excited about these things and I am sure had I had them when my career started, I’d be much further than I am now.

First of all, where can you go to learn about JavaScript?

Unsurprisingly, this is a tricky one. JavaScript is a hot topic and a lot of people get the job to write some without wanting to understand it. This means there are terrible, spammy resources telling you the “simple” way to do something in JavaScript. Do yourself a favour and don’t fall for that siren song. Also be vigilant about “here is the quick solution” answers on the web. Often these are biased solutions that keep getting repeated to reach a quick result. If something sounds too good to be true, often that is exactly what it is.

Mozilla Developer Netwwork

This is where the MDN Web Docs is my weapon of choice. It is a unbiased, open, very well maintained resource. It is not a Mozilla only thing, but many other players add content and help keeping it up-to-date. It is even writable – you can do edits when you find something is wrong.

The great thing about the Web Docs is that it isn’t “just” documentation. It also has code examples and detailed browser support tables for all the things it talks about. Many resources that tried to document the open web came and went. MDN prevailed.

Talking about browser support, Can I use is a well-maintained resource. It doesn’t only show detailed browser support. It also links to the standard documents telling you what should happen. And it shows the quirks and problems in various versions with possible workarounds.

Can I use web site

Browsers are much less of a problem

Talking about browsers, it is a wonderful time in that regard. We’ve had a tricky ride there. In the past, browsers were a black hole and we had no idea what magic made them work. Until browsers committed to following standards we had to know about their ideas how things should work. In essence, our career was dependent on knowing how browsers mess up. That’s easy to do, but in the long run not fulfilling. Nowadays browsers take a much more leading role in defining and inspiring standards. Browsers themselves strive to be evergreen. Non-standardised functionality is behind flags or shipped in “developer editions” or nightly builds.

The best thing is that browsers makers are available for feedback and invite you to file bugs. This has been a massive change and something we as developers should cherish. There is a lot less of “this is how it is, not much we can do” and a lot more “well, that doesn’t work, can we get this fixed”.

Browser makers also understand that developers matter. Furthermore, that a web developer is as much an engineer as somebody writing Java or C++. Engineering needs more than “throw code at us and we make it work”. We need insights into how our code performs and what effects it has. That’s why they all browser makers spend a lot of time building developer tools. These give us the insights we need to write performant and secure JavaScript.

Learning browser developer tools

It is a good idea as a JavaScript developer to get accustomed to browser devtools as much as you can. Every browser provides them and they differ in some regards, but they all give you a lot of insight. You can learn what’s available to you by inspecting objects the browser gives you in the console. You learn what happens when your page gets rendered and where the bottlenecks are. You can inspect, edit and test what the visual output of your code is and see the CSS browsers generate from it.

Moving from console.log() to breakpoint debugging

One thing that helped me a lot is moving away from a console.log() mentality to using breakpoints. Instead of having to ask for each of the things you want to know about you get so much more. Breakpoint debugging has the benefit that the JavaScript engine stops. You can then get a snapshot of what’s happening in the browser. You don’t only get the value of, for example, a variable, but you can also see the effects this change has. And you have a direct way to step through debugging your code without having to change it. One breakpoint is often worth dozens of logging requests.

It is great that we can now debug our JavaScript, but wouldn’t it be better if we didn’t make mistakes in the first place? We’re human, we get tired, we get bored, we get sloppy. That’s why it is good when we get reminders that we’re doing something wrong whilst we’re doing it. In Word processing this is the spelling and grammar check that adds squiggly red lines were we went wrong.

Linting – prevent bugs by not writing wrong code

In development, this is linting. Linters are software that analyses your code while you write it. It executes the partial code in the background and tells you when something is wrong. In the past you needed to install and configure linters yourself. These days, many come as extensions to editors with preconfigured rules. Rules that are the result of years of experts arguing and finding a consensus what makes sense. Google’s developer tools in Chrome give you similar insights. Reading through the results of linters is a good learning exercise. They not only tell you that something is wrong, but also why, what the effects are and how to fix it.

Finding an editor that makes you more effective

This is also where a good editor is a life-saver. As said before, you can use whatever you want to write your JavaScript. For me, I am amazed how much work I get done using Visual Studio Code. This editor is open source and comes with hundreds of extensions to help you with your tasks. It has linting built-in and allows for setting breakpoints in the editor itself. It contains a command line interface to do more hard-core tasks and has Git version control built in. There is something great about not having to jump from editor to browser to terminal. Of course, sooner or later you’ll have to master all three, but why not start with a shortcut?

And guess what this editor is written in? Yes, JavaScript.

When using JavaScript you will find holy wars and ideas of what editor in what configuration is the best. As there are dozens of options in hundreds of variations, you’re welcome to go down that rabbit hole. I move every few years from one environment to another and right now, Visual Studio Code makes me happy. And many others agree. It gives you a jumpstart by providing integration with build processes out of the box. It is cross-platform and lightweight compared to other IDEs with similar feature sets. And you can hack on it and extend it using JavaScript.

Publication and build processes

That said, editors are not the only place where linting can help. It can also be a part of a publication process. Sonar, for example is a tool that requires a URL and than tells you all the things that need improvement. This involves performance, compatibility, security and many more. A service like Sonar can act as a gate-keeper in your release process. A bug found by you is better than one reported by your users. Or, even worse, one that prevents people from using your work without you ever hearing about their problems.

Sonarwhal results for this web page

Release processes are a huge part of what we follow in JavaScript these days. So it is a good idea to have it in the back of your head to get accustomed to those. A common piece of advice is to start your own project and follow best practices as you can’t break others that way. But I am not sure how helpful this is and found it rather dry and frustrating.

Should creating your first project include configuring your own server?

There is no doubt that you learn a lot by starting your own projects and running them in a professional manner. But you don’t need to start that way. I’d argue it makes more sense to start smaller. JSBin, CodePen, Glitch and other hosting communities are there for you to play and learn. They allow you to use preprocessors, frameworks and package managers to build something. You don’t get stuck setting them up and configuring your own computer over and over again. You can see if they work for you without the overhead of having to learn and un-learn something. In a job, as part of a team, it is unlikely that as a junior developer you’d be setting up the environments. So keep that for when it is necessary.

Learning materials are free and in abundance. But what’s good and what’s spam?

Talking about learning, we live in amazing times. There are almost weekly conferences about JavaScript. There are free to attend meetups. There are Slack communities, mailing lists and free online resources. Many books on the subject are free to read online. And almost all the content you pay a lot of money for to see live ends up on YouTube to watch within hours. Frankly, we have so much content that triaging it becomes a necessity. So instead of trying to follow it all, find people you trust who do a good job collecting and commenting. Then take your time to digest what is out there in your own pace. I watch talks in the gym on my phone – the average JavaScript talk burns 500 calories on a cross-trainer.

Contributing to projects and dealing with other people

Instead of starting by owning of a product, it makes much more sense to take part in existing ones. This is what JavaScript is a about these days. We’re not dealing with a language, but we are dealing with a community of developers. Many open source projects have people to help you get involved. You can get started by helping to improve documentation. You can fix simple problems the maintainers are too busy to look at as they got bigger fish to fry. And you can learn a lot by lurking and seeing what people do and how they work with each other. You also learn what to avoid and what kind of developer not to become.

As it can get messy, I’m not beating around the bush there. The success of JavaScript and the general hype around software engineering has an ugly side to it. People get into unhealthy competition and see overworking themselves as a badge of honour. It doesn’t help that many community tools value “fast an often” as a way of contribution over “good and well explained”. You will encounter terrible communication and open hostility to change and discussion.

You’re new here. Be that new person and a better one than those who annoy you

And here is my plea to you. Don’t become that person. Don’t feed the ego of those valuing telling others that they are wrong over working together. Stay calm, be kind and don’t let blanket statements and open dismissal of ideas stop you. There is a lot to choose from. No need to fill your life with more drama, when all you want is to make your mark as a creator.

We have a beautiful technology stack here and if I learned anything that change is the only constant. And today, here and now, I hope that you will be part of a change that we need.

Your happiness is more important than getting into fights over syntax, process or what framework to use. Happy developers build better products. The more diverse we become, the more we can reflect those who use our products.

And with that, I am looking forward to seeing what you do next. Thank you for going on this journey. It did me good, I hope it also will work for you.

Web Truths: Publishing on the web using web standards is easy and amazing

Monday, November 27th, 2017

This is part of the web truths series of posts. A series where we look at true sounding statements that we keep using to have endless discussions instead of moving on. Today I want to talk about the notion of the web as an open publication platform and HTML as the way to publish.

Publishing on the web using web standards is easy and amazing

Those of us who were lucky enough to be around when the web started growing cherish this concept. It was exciting to be able to open a text editor – any text editor – write some tags and see our text come to life in a browser. Adding a H1 here gave us a headline, adding a P described paragraphs and gave free margins. Adding an A allowed us to point to other resources or a target in the same page. Adding an HR gave us a horizontal ruler. The latter later on turned out to be a pain in the backside to style and was generally a bad idea. We learned by doing and we learned by trying things out.

We also smirked at the people who looked down on us as not being “real developers”. These people considered us crazy for relying on a browser we don’t control to render our output. Could be that we were, but at least we didn’t have to use a convoluted IDE and follow a slow build process to get a result. Our work was immediate and satisfaction was only a ctrl+s, shift-tab and ctrl-r (or F5) away.

We also loved the fact that all we need is some hosting space and an FTP client to publish our work to the world.

If we disliked what our server companies did, we moved to the next one. After all, your domain is the thing you send to people, not your IP address. To get people to read what we wrote we published on mailing lists, in forums and on IRC. We linked to each other in webrings and banner exchanges. When Google came around we were the old guard that Google loved. Easy to index, with sensible TITLE and proper text content. Not like Flash pages that tried to trick Google with META keyword spamming or hidden text.

All the good things about HTML and publishing on your own server are still valid. It is a beautiful, free and open way to publish with nobody to answer to. You are your own marketing department and your server is your playground. If you are vigilant and you make sure that you have a lot of time to delete spam and defend yourself against attacks.

But here is the problem with telling this story over and over again. It doesn’t quite work any longer. And it is not as fascinating for people these days as it was for us. I remember the exhiliration of hearing a successful modem handshake and my HTML rendering in Netscape 3. I remember explaining to people that “when it sounds everything is broken, then you are online”. People these days don’t expect to be offline. Often they only experience being offline when they are on the go and the mobile connection dies. Those lucky enough to have a home with a fat wireless connection never put effort in to reach the content. They consume, much like we did when we watched TV.

The same happens to publication. It isn’t about writing the perfect article or blog post. It is about creating something fast that gets a lot of eyeballs. And if you want eyeballs, you keep publishing. Faster and faster, more and more. And as it is hard to create your own, you re-hash what other people have done instead and ride any success train of the day. There is no place for HTML or proper standards publication in this world.

I’m not saying at all that this is a good thing. But it is where we are. The web hasn’t won over Facebook, WhatsApp and other closed environments because it needs more effort to use. You still need to show interest and skill to build a web site. Writing three words and picking a GIF from a collection is easier.

The web we love and explain as the amazing publishing platform will survive. It will be a playground of enthusiasts and specialists. And old people. The disruptive platform of the past hasn’t become the mainstream. Everyone has the potential to be a creator and maker. But the marketing machine of the world wants us to be consumers instead. And the best way to keep people consuming is to lock them into a place that is ridiculously easy to publish in. More importantly you need to give them the feeling of being part of a community of cool people. And the explosive growth of the web and tweaking of search algorithms to show the “new” instead of the “correct” isn’t a group of cool people. It is hard work. There are no likes, kudos, claps or whatever on the web. Adding an immediate feedback channel for people is 90% removing horrible content and spam. The web isn’t a small group of cool people, but mainstream media makes pretty sure to tell us it is full of dangers and wrong information. Better stay where it is safe. In a controlled environment that has very enticing immediate feedback.

If you think this is dark, check out André Staltz’ The Web began dying in 2014, here’s how where he paints a pretty bleak future for the mainstream web. And darn him, there is some pretty good evidence in there that finding web content in the future not published inside Google, Amazon or Facebook products will be close to impossible.

So, yes, publishing on the web is amazing. Nobody denies that. But we’re dealing with a new generation of people who grew up with the web and don’t care about it. It is there, like water is when you open the tap. You don’t think about how it gets there or what is involved until it stops coming out. And that might be the same for the web.

Instead of painting a romantic view of how the open web keeps prevailing, it may be time to tell people more about what their use of closed platforms does. How much they give away for the convenience of publishing something and harvesting some fake internet points. We’re past the format of the publication. We need to get people excited again about owning their data. For our sake and theirs.

Web Truths: the web is broken and backwards compatibility is holding us back

Tuesday, November 21st, 2017

This is part of the web truths series of posts. A series where we look at true sounding statements that we keep using to have endless discussions instead of moving on. Today I want to tackle the issue of the web not moving fast enough for people and clinging on to seemingly terrible ideas from the past.

The web is broken and backwards compatibility is holding us back

This is the counter-argument to the one I discussed in the last post. Much like the exaggurated praise for the web and its distributed nature it has been around for as long as I can remember online discussions.

There is no doubt that many things about the web are sub-optimal. It is also true that carrying the burden of never blocking out old content can slow us down. Yes there are many features of CSS and JavaScript that in hindsight are terrible ideas. And it is true that by sticking to the bleeding edge, you have a lot more fun as a developer. First, there are more things to play with. And, more modern environments also come with better tooling and deeper insights.

There is a problem with this though: as Calvin said it, the problem with the future is that it always turns into the present.

calvin and hobbes strip

So, whenever we embrace new and bleeding edge technology and damn the consequences we create debt. Often arguments against backwards compatibility stem from actions like these. Some standards we have to keep adding to browsers have been rash decisions or based on a need of one player in the W3C at the time.

Breaking changes in any new version of software aren’t ever fun for users and maintainers. This gets worse with how popular your software becomes and how many people use it. And I’d argue that the web is the most used piece of software out there.

Back in the day the argument against the web stack was always Flash. It seemed to be the right thing to use. There was 99% coverage in browsers. It had far advanced tooling in comparison to Firebug (RIP). And there was a sort of built-in code protection. People couldn’t look at and steal your code without jumping any barriers.

Turns out, Flash wasn’t the amazing platform these arguments made it out to be. In the Flash Games Post Mortem keynote at GDC. John Cooney of Kongegrate talks about the story of Indie gaming and Flash.

I love this talk. It shows that Flash and Web developers weren’t that different. Except Flash developers were more pragmatic about wanting to make money. And they had less delusions about their code lasting forever but knew that they had a short window of opportunity.

And this is what this argument boils down to. When it comes to betting on the web, there’s a lot of good faith and wanting to create something lasting involved. If that is your thing it will make you more understanding for the failures of the web as a software platform. And it makes backwards compatibility a no-brainer as this is what ensures the longevity of the web. When Flash changed and the support from the one company that owned it faded, a lot of developers felt forgotten. We now have the problem that a lot of creativity and a lot of work will go away as the platform to execute it is gone. Backwards compatibility ensures that isn’t the case.

If your thing is to release something quick, make some money and know it will go away, the web isn’t as interesting. Even worse, those defending it can come across as evangelical or condescending. But there is nothing wrong with what you want to do. A lot of innovation stems from this approach, and the web can learn from its successes and failures. Much like HTML5 learned a lot from Flash. But that doesn’t mean your approach is better or that the web is broken – it just doesn’t fit your goals. Without the web, Flash wouldn’t have happened the way it has either. Air proved that. The distribution model of the web works. And you can benefit from that without having to replace it.

There are of course some valid arguments for the abandonment of older ideas and non-support of broken platforms. Seeing how fast JavaScript moves, it seems detrimental to the cause to support older browsers. And some of the new ideas we have now solve important performance and security issues.

But all in all, the backwards compatibility of the web is what made it survive all the other platforms set out to replace it. And there will not be a time where we need to run the web in emulators because of it. That is what makes the argument of the web as broken and backwards compatibility holding us back invalid. Of course we can do better, but are we also 100% sure that what we think is amazing now really stands the test of time?

There are more pressing matters to consider:

  • How can we ensure that despite backwards compatibility we get people to upgrade their environments? Seeing that a malware targeted at Windows XP is a huge success in 2017 is more than worrying.
  • How can we enhance older solutions to become better without breaking them? Chrome’s passive Event Listener extension to addEventListener seems to break backwards compatibility . Arrow functions are arguably only syntactic sugar (despite fixing “this”) but will always be just a syntax error for older browsers.
  • How can we make developers embrace newer solutions to old problems that have less side effects? It seems there is a certain point where we stop caring to keep up-to-date and use whatever worked in the past instead.
  • How can we make newer developers embrace the idea of the web as a platform without overloading them with borderline evangelical and philosophical messages? How can we make the web speak for itself?

Web Truths: The web is better than any other platform as it is backwards compatible and fault tolerant

Saturday, November 18th, 2017

This is part of the web truths series of posts. A series where we look at true sounding statements that we keep using to have endless discussions instead of moving on. Today I want to tackle the issue of the web as a publication platform and how we keep repeating its virtues that may not apply to a publisher audience.

The web is better than any other platform as it is backwards compatible and fault tolerant

This has been the mantra of any web standards fan for a very long time. The web gets a lot of praise as it is to a degree the only platform that has future-proofing built in. This isn’t a grandiose statement. We have proof. Web sites older than many of today’s engineers still work in the newest browsers and devices. Many are still available, whilst those gone are often still available in cached form. Both search engines and the fabulous wayback machine take care of that – whether you want it or not. Betting on the web and standards means you have a product consumable now and in the future.

This longevity of the web stems from a few basic principles. Openness, standardisation, fault tolerance and backwards compatibility.

Openness

Openness is the thing that makes the web great. You publish in the open. How your product is consumed depends on what the user can afford – both on a technical and a physical level. You don’t expect your users to have a certain device or browser. You can’t force your users to be able to see or overcome other physical barriers. But as you published in an open format, they can, for example, translate your web site with an online system to read it. They can also zoom into it or even use a screenreader to hear it when they can’t see.

One person’s benefit can be another’s annoyance, though. Not everybody wants to allow others to access and change their content to their needs. Even worse – be able to see and use their code. Clients have always asked us to “protect their content”. But they also wanted to reap the rewards of an open platform. It is our job to make both possible and often this means we need to find a consensus. If you want to dive into a messy debate about this, follow what’s happening around DRM and online video.

Standardisation

Standardisation gave us predictability. Before browsers agreed on standards, web development was a mess. Standards allowed us to predict how something should display. Thus we knew when it was the browser’s fault or ours when things went wrong. Strictly speaking standards weren’t necessary for the web to work. Font tags, center tags, table layouts and all kind of other horrible ideas did an OK job. What standards allow us to do is to write quality code and make our lives easier. We don’t paint with HTML. Instead, we structure documents. We embed extra information and thus enable conversion into other formats. We use CSS to define the look and feel in one central location for thousands of documents.

The biggest benefactors of standards driven development are developers. It is a matter of code quality. Standards-compliant code is easier to read, makes more sense and has predictable outcome.

It also comes with lots of user benefits. A button element is keyboard, touch and mouse accessible and is available even to blind users. A DIV needs a lot of developer love to become an interactive element.

But that doesn’t mean we need to have everything follow standards. If we had enforced that, the web wouldn’t be where it is now. Again, for better or worse. XHTML died because it was too restrictive. HTML5 and lenient parsers were necessary to compete with Flash and to move the web forward.

Backwards compatibility

Backwards compatibilty is another big part of the web platform. We subscribed to the idea of older products being available in the future. That means we need to cater for old technology in newer browsers. Table layouts from long ago need to render as intended. There are even sites these days publishing in that format, like Hacker News. For browser makers, this is a real problem as it means we need to maintain a lot of old code. Code that not only has a diminishing use on the web, but often even is a security or performance issue. Still, we can’t break the web. Anything that goes into a “de facto standard” of web usage becomes a maintenance item. For a horror story on that, just look at all the things that can go in the head of a document. Most of these are non-standard, but people do rely on them.

Fault tolerance

Fault tolerance is a big one, too. From the very beginning web standards like HTML and CSS allow for developer errors. In the design principles of the language the “Priority of Constituencies” states it as such:

In case of conflict, consider users over authors over implementors over specifiers over theoretical purity

This idea is there to protect the user. A mistake made by a developer or a third party piece of code like and ad causing a problem should not block out users. The worrying part is that in a world where we’re asked to deliver more in a shorter amount of time it makes developers sloppy.

The web is great, but not simple to measure or monetise

What we have with the web is an open, distributed platform that grants the users all the rights to convert content to their needs. It makes it easy to publish content as it is forgiving to developer and publisher errors. This is the reason why it grew so fast.

Does this make it better than any other platform or does it make it different? Is longevity always the goal? Do we have to publish everything in the open?

There is no doubt that the web is great and was good for us. But I am getting less and less excited about what’s happening to it right now. Banging on and on about how great the web as a platform is doesn’t help with its problems.

It is hard to monetise something on the web when people either don’t want to pay or block your ads. And the fact that highly intrusive ads and trackers exist is not an excuse for that but a result of it. The more we block, the more aggressive advertising gets. I don’t know anyone who enjoys interstitials and popups. But they must work – or people wouldn’t use them.

The web is not in a good way. Sure, there is an artisinal, indie movement that creates great new and open ways to use it. But the mainstream web is terrible. It is bloated, boringly predictable and seems to try very hard to stay relevant whilst publishers get excited about snapchat and other, more ephemeral platforms.

Even the father of the WWW is worried: Tim Berners-Lee on the future of the web: The system is failing .

If we love the web the way we are happy to say all the time we need to find a solution for that. We can’t pretend everything is great because the platform is sturdy and people could publish in an accessible way. We need to ensure that the output of any way to publish on the web results in a great user experience.

The web isn’t the main target for publishers any longer and not the cool kid on the block. Social media lives on the web, but locks people in a very cleverly woven web of addiction and deceit. We need to concentrate more on what people publish on the web and how publishers manipulate content and users.

Parimal Satyal’s excellent Against a User Hostile Web is a great example how you can convey this message and think further.

In a world of big numbers and fast turnaround longevity isn’t a goal, it is a nice to have. We need to bring the web back to being the first publishing target, not a place to advertise your app or redirect to a social platform.

Web Truths: We need granular control over web APIs, not abstractions

Monday, October 16th, 2017

This is part of the web truths series of posts. A series where we look at true sounding statements that we keep using to have endless discussions instead of moving on. Today I want to tackle the issue of offering new functionality to the web. Should we deliver low-level APIs to functionality to offer granular control? Or should we have abstractions that get people started faster? Or both?

In a perfect scenario, both is the obvious answer. We should have low-level APIs for those working “close to the metal”. And we should offer abstractions based on those APIs that allow for easier access and use.

In reality there is quite a disconnect between the two. There is no question that newer web standards learned a lot from abstractions. For example, jQuery influenced many additions to the DOM specification. When browsers finally got querySelector and classList we expected this to be the end of the need for abstractions. Except, it wasn’t and still isn’t. What abstractions also managed to do is to even out implementation bugs and offer terser syntax. Both of these things resonate well with developers. That’s why we have a whole group of developers that are happy to use an abstraction and trust it to do the right thing for them.

Before we had a standardised web, we had to develop to the whims of browser makers. With the emergence of standards this changed. Web standards were our safeguard. By following them we had a predictable way of debugging. We knew what browsers were supposed to do. Thus we knew when we made a mistake and when it was a bug in the platform. This worked well for a textual and forms driven web. When HTML5 broke into the application space web standards became much more complex. Add the larger browers and platform fragmentation and working towards standards and on the web became much harder. It doesn’t help when some of the standards felt rushed. An API that returns empty string, “possibly” or “maybe” when asked if the current browser can play a video doesn’t fill you with confidence. For outsiders and beginners, web standards are not considered any more the “use this and it will work” approach. They seem convoluted in comparison with other offers and a lot of changes seem to be a lot of work for developers to keep up. Maybe too much work.

Here’s what it boils down to:

  • Abstractions shield developers from a lot of implementation quirks and help them work on what they want to achieve instead
  • Low-level APIs allow for leaner solutions, but expect developers to know them, keep track of changes and to deal in a sensible way with non-supporting environments (see: Progressive Enhancement)

What do developers want?

As web developers in the know, you want to have granular control. We’ve been burnt too often by “magical” abstractions. We want to know what we use and see where it comes from. That way we can create a lot of different solutions and ensure that what we want to standardise works. We also want to be able to fix and replace parts of our solutions when a part becomes problematic. What we don’t want is to be unable to trace back where a certain issue comes from. We also want to ensure that new functionality of the web stays transparent and secure. We achieve this by creating smaller, specialised components that can get mixed and matched.

As new developers who haven’t gone through the pains of the browser wars or don’t need to know how browsers work, things look different. We want to code and reach our goal instead of learning about all the different parts along the way. We want to re-use what works and worry about our product instead. We’re not that bothered about the web as a platform and its future. For us, it is only one form factor to build against and to release products on. Much like iOS is or gaming plaforms are.

This is also where our market is going: we’re not paid to understand what we do – we’re expected to already know. We’re paid to create a viable product in the shortest amount of time and with the least effort.

The problems is that the track record of the web shows that we often have to start over whenever there is a new technology. And that instead of creating web specific functionality we got caught up trying to emulate what other platforms did.

The best case in point here is offline functionality. When HTML5 became the thing and Flash was declared dead we needed a fast solution to offer offline content. AppCache was born, and it looked like a simple solution to the issue. As it turns out, once again what looked too good to be true wasn’t that great at all. A lot of functionality of AppCache was unreliable. In retrospect it also turned out to be more of a security issue than we anticipated.

There was too much “magic” going on that browsers did for us and we didn’t have enough insight as implementers. That’s how Service Workers came about. We wanted to do the right thing and offer a much more granular way of defnining what browsers cache where and when. And we wanted to give developers a chance to intercept network requests and act on them. This is a huge endeavour. In essence we replicate the networking stack of a browser with an API. By now, Service Workers are doing much more than just offline functionality. They also should deal with push notifications and app updates in the background.

This makes Service Workers tougher to work with as they seemed complex. Add to that the lack of support in Safari (which is now changing) and you lost a lot of developer enthusiasm.

There is more use in abstractions like Workbox as they promise you to keep up-to-date whilst the changes in the spec are ironed out. Instead of getting a “here are all the lego bricks, build your own car”, it has a “so you want to build a car, here are some ways to do so” approach.

This is a good thing. Of course we need to define more granular and transparent standards and solutions to build the web on. But there is a reluctance in developers to take part in the definition phase and keep an eye on changes. We can not expect everybody who wants to build for the web to care that much. That is not how the web grew – not everybody had to be a low level engineer or know JavaScript. We should consider that the web outgrew the time where everyone was deeply involved with the standards world.

We need to face the fact that the web has become much more complex than it used to be. We demand a lot from developers if we want them all to keep up to date with standards. Work that often isn’t as appreciated by employers or clients than shipping products is.

This isn’t good. This isn’t maintainable or future facing. And it shouldn’t have come to this. But it is a way of development we allowed to take over. Development has become a pretty exhausting and competitive environment. Deliver fast and in a short cadence. Move fast and break things. If you can re-use something, do it, don’t worry too much if you don’t know what it does or if it is secure to do so. If you don’t deliver it first to market someone else will.

This attitude is not healthy and we’re rubbing ourselves raw following it. It also ensures that diversity in our market it tough to achieve. It is an aggressive game that demands a lot of our time and an unhealthy amount of competitiveness.

We need to find a way to define what’s next on the web and make it available as soon as possible. Waiting for all players to support a new feature makes it hard for developers to use things in production.

Relying on abstractions seems to be the way things are going anyways. That means as standards creators and browser makers we need to work more with abstraction developers. It seems less and less likely that people are ready to give up their time to follow specs as they change and work with functionality behind flags. Sure, at conferences and in our talks everyone gets excited. The hardware and OS configurations we have support all the cool and new. But we need to get faster to market and reach those who aren’t already sold on our ideas.

So, the question isn’t about granular definition of specifications, small parts that work together or abstractions. It is about getting new and sensible, more secure and better performing solutions into production code quicker. And this means we need both. And abstractions should have a faster update cycle to incorporate new APIs under the hood. We should work on abstractions using standards, not patching them.