Christian Heilmann

Author Archive

Assistive technology shouldn’t be a mystery box

Saturday, August 5th, 2023

Web developers nowadays have full insight into how browsers work. The engines are open source, and you have programmatic access to almost anything they do. You even have in-built developer tools allowing you to see the effects your code has. You can see what the render engine does and how much memory gets allocated.

With assistive technology like screenreaders, zooming tools and voice recognition you got nothing. Often, you even don’t have access to them without paying.

Closed source stalled the web for years

For years, closed browsers held back the evolution of the web platform. When I started as a web developer, there was IE, Netscape and Opera. The DOM wasn’t standardised yet and it was a mess. Your job as a web developer was to 90% knowing how which browser failed to show the things you tell it to. Another skill was to know hacks to apply to make them behave. Starting with layout tables and ending with hasLayout and clearfix hacks.

It is no wonder that when Flash came about and promised a simpler and predictable way to build web applications, people flocked to it. We wanted to build software products, not fix the platform whilst doing that.

This has been an ongoing pattern. Every time the platform showed slow-ish improvement due to browser differences, developers acted by building an abstraction. DHTML was the first bit where we created everything in JavaScript detecting support where needed. And later on jQuery became the poster child of abstracting browser bugs and DOM inconvenience away from you, allowing you to achieve more by writing less.

Open source browsers are amazing

Fast forward to now where every browser is based on some open source engine or another. Developers get more insight into why things don’t work and can contribute patches. Abstractions are used to increase developer convenience and to offer more full stack functionality than only showing a page. The platform thrives and there are new and exciting innovations happening almost every week. Adoption and implementation of these features is now in a several months timeframe rather than years. A big step here was to separate browsers from the OS, so you don’t need to wait for a new version of Windows to get a new browser version.

Assistive technology is amazing, but we don’t know why

And then we look at assistive technology and things go dark and confusing. If you ask most web developers how screenreaders work and what to do to enable voice recognition things get awry quick. You either hear outdated information or that people haven’t even considered looking at that.

Being accessible isn’t a nice to have. It is a necessity for software and often even a legal requirement to be allowed to release a product.

At Microsoft, this was a big part of my job. Making sure that our software is accessible to all. And this is where things got annoying quick.

There are ways to make things accessible and the best thing to start with is to use semantic HTML. When you use a sensible heading structure and landmarks, people can navigate the document. When you use a button element instead of a DIV, it becomes keyboard and mouse accessible. At least that’s the idea. There are several browser bugs and engine maker decisions that put a spanner in the works. But on the whole, this is a great start. When things get more problematic, you can use ARIA to describe functionality. And hope that assistive technology understands it.

I’ve been blessed with a dedicated and skilled team of people who bent over backwards to make things accessible. The most frustrating thing though was that we had a ton of “won’t fix” bugs that we could not do anything about. We did all according to spec, but the assistive technology wouldn’t do anything with the information we provided. We had no way to fix that as there was no insight into how the software worked. We couldn’t access it to verify ourselves and there was no communication channel where we could file bugs.

We could talk to the Windows Narrator team and fix issues there. We filed bugs for MacOS Voiceover with varying success of getting answers. We even submitted a patch for Orca that never got merged. And when it came to JAWS, well, there was a just a wall of silence.

Ironically, assistive technology isn’t very accessible

Every accessibility advocate will tell you that nothing beats testing with real assistive technology. That makes sense. Much like you can simulate mobile devices but you will find most of the issues when you use the real thing. But assistive technology is not easy to use. I keep seeing developers use a screenreader whilst looking at the screen and tabbing to move around. Screenreaders have complex keyboard shortcuts to navigate. These are muscle memory to their users, but quite a challenge to get into only to test your own products.

Then there’s the price tag. Dragon natural speaking ranges from 70 to 320 Euro. JAWS is a $95 a year licence in the US only, or a $1110 license for the home edition or $1475 professional license. You can also get a 90 day license for $290. So, I could use a VPN to hack around that or keep resetting my machine date and time, much like we did with shareware in the mid-90s?

I’ve been to a lot of accessibility conferences and kicked off the subject of the price of assistive technology. Often it is dismissed as not a problem as people with disabilities do get them paid by their employers or government benefit programs. Sounds like a great way to make money by keeping software closed and proprietary, right?

Well, NVDA showed that there is a counter movement. It is open source, built by developers who also rely on it to use a computer and responsive to feedback. Excellent, now we only need it to be cross-platform.

When you look at the annual screenreader survey, JAWS has 53.7% of the market, followed by NVDA at 30%. The assistive tools that come for free with the operating system are hardly used. Voiceover on MacOS is 6.5% of the users and Microsoft Narrator even 0.5%.

The reason why these newer tools don’t seem to get as much adoption is that people used JAWS for a long time and are used to it. Understandable, considering how much work it is to get accustomed to it. But it means that I as a developer who wants to do the right thing and create accessible content have no way to do so. If JAWS or Dragon Natural Speaking doesn’t understand the correct markup or code I create, everybody loses.

Access needs to me more transparent

I really would love open source solutions to disrupt the assistive technology market. I’d love people who can’t see getting all they need from the operating system and not having to fork over a lot of money just to access content. I want Narrator and Voiceover and Orca to be the leaders in innovation and user convenience. NVDA could be an open source solution for others to build their solutions on, much like Webkit and Chromium is for browsers.

But there is not much happening in that space. Maybe because it is cooler to build yet another JavaScript component framework. Or because assistive technology has always been a dark matter area. There is not much information how these tools work, and if my experience as a developer of over 20 years is anything to go by, it might be that this is by design. Early assistive technology had to do a lot of hacking as the operating systems didn’t have standardised accessibility APIs yet. So maybe there are decades of spaghetti code held together by chewing gum and unicorn tears people don’t want to show the world.

GitHub Copilot is a great example for in-context AI chat

Wednesday, August 2nd, 2023

GitHub Copilot chat in-context with a highlighted piece of code

Generative AI is only as good as its integration in your work environment. GitHub Copilot Chat in Visual Studio Code is a great example how to do that right.

Recording showing the interaction in Visual Studio Code

You can highlight code in the editor and press `CMD + I` on Mac or `Ctrl + I` on Linux/Windows and you get a text box to chat about this piece of code. For example, you can ask Copilot to explain it.

If you ask Copilot to modify the code, for example by asking it to add step-by-step instructions, it won’t replace the code immediately, but give you a diff view to see and change the recommendations before applying them.

Together with an extension to verify the validity of your code, like webhint this becomes a safe and thoroughly enjoyable way of working.

Generative AI is amazing – but good UX makes it even better.

Five things about public speaking

Monday, July 31st, 2023

Still image of the video of me explaining three things to do when giving a talk

As part of the WeAreDevelopers World Congress 2023, we did a quick session on starting out as a public speaker. To this end, I was asked to do a “Five things about public speaking” video. Here is the video and a write-up about the tips.

In the world of public speaking, engaging, informing, and delighting an audience is both an art and a science. If you’re a novice speaker ready to step onto the stage and into the spotlight, here are some tips.

Share Your Story

Every aspect of your talk should reflect who you are as an individual. Your slides, while offering informative visuals, are merely wallpaper lending a backdrop to your core message. Therefore, presentation success lies largely in your personal connection to the topic at hand.

Share your enthusiasm by explaining why the subject excites you, how you discovered it, and what resources you’ve used—all of which your audience can use later when they want do learn more about the topic.

How you learned about a topic, why it matters to you, and why it should matter to the audience, is the best way to bring something across.

Dealing with Nerves

While it’s perfectly natural to be nervous before a talk, it’s crucial to manage these emotions productively. Use this energy to fuel your performance on stage. If the anxiety feels overwhelming, open up about it—authenticity resonates with people.

Redirect your focus from potential judgement to your fundamental purpose: to show excitement and share knowledge.

Preparing for your Talk

Primarily, prepare by conducting thorough research to master your content. Find something current and relative to your topic to serve as a hook for your audience, demonstrating the immediacy and relevance of what you’re discussing.

Secure your presentation by making sure all your demos and materials function offline and are accessible to your audience afterwards. Familiarise yourself with the technological equipment available to avoid overcomplicating or failing demonstrations.

It is highly important to learn about your audience and cater your presentation to their needs. Aim to provide them with a key takeaway they can use to impress their superiors.

Do’s and Don’ts

Do’s

1. Tell Stories. Make the subject human and exciting, and point to materials where your audience can learn more.
2. Limit Yourself. Instead of overwhelming the audience with excessive amounts of information, focus on delivering one key point exceptionally well.
3. Use Clear and Simple Language. Avoid showing off your mastery over sophisticated vocabulary and instead strive to be comprehensible to your audience.

Don’ts

1. Take all the credit. Give credit where it’s due, acknowledging the people from whom you’ve learned.
2. Read out your slides to the audience. You’re not an audience member but the show-runner. The human factor in a presentation is what can’t be digested from simply reading a slide.
3. Try to be funny for the sake of being funny. Humor is great, but only when it doesn’t become a distraction from your main message.

Harness the Unexpected

During one of my keynotes on machine learning, a power outage left me delivering my message in the dark, and later, my slides appearing in an unintended random order. Instead of panicking, I leveraged these mishaps to emphasise the unreliability of machines—ironically, the exact message I had come to share.

Remember, when unexpected situations arise during your talk, flip them to your advantage. Owning your story and being an expert on your content equips you for effectively dealing with any hurdles that may occur on stage.

In conclusion, successful public speaking requires substantial preparation, an understanding of your audience, and most importantly, a genuine passion for your topic. Serve as a guide to your audience by threading a story throughout your presentation, and never lose sight of why the subject holds significance for both you and your listeners.

We are looking for presenters!

If you want to start as a public speaker, remember that WeAreDevelopers has a “live” video series and we are looking for presenters. Check out the WeAreDevelopers live site and the call for papers .

This post has partly been created using VideoTap .

Browser-Entwicklerwerkzeuge für Nicht-Entwickler:innen – neuer Kurs auf LinkedIn Learning

Wednesday, July 19th, 2023

Ab heute ist mein erster deutscher Kurs auf LinkedIn Learning Browser Entwicklerwerkzeuge für Nicht-Entwickler:innen live!

In dem circa eine Stunde langem Kurs erkläre ich, wie man die Entwicklerwerkzeuge in den gängigen Browsern benutzt, um Webinhalte zu analysieren und zu verändern. Der Kurs richtet sich an alle, die sich für Webinhalte interessieren, aber keine Entwickler:innen sind. Stattdessen zeige ich wie man:

  • Bilder auf Webseiten herunterladen kann (auch wenn es keinen Download-Button gibt)
  • Videos einfacher anzeigen lassen kann
  • Nervige Elemente auf Webseiten ausblenden kann
  • Webseiten auf verschiedenen Geräten simulieren kann um eine einfachere Darstellung zu bekommen
  • Funktionalität, die nur auf dem Handy angeboten wird auch auf dem Rechner zu nutzen
  • Webseiten auf Fehler untersuchen kann
  • Cookies anzeigen und verändern kann
  • und vieles mehr

Hier sind alle Videos mit den jeweiligen Zeiten:

  • 1. Was sind Browser-Entwicklerwerkzeuge?
    • Entwicklerwerkzeuge öffnen (1m 22s)
    • Worauf kann ich zugreifen? (1m 37s)
    • Ist das überhaupt erlaubt? (48s)
  • 2. Wie benutze ich die Entwicklerwerkzeuge?
    • Navigation in den Entwicklerwerkzeugen (2m 49s)
    • Ein- und Ausschalten verschiedener Werkzeuge (1m 13s)
  • 3. Entwicklerwerkzeuge den eigenen Bedürfnissen anpassen
    • Wo sollen die Werkzeuge angezeigt werden? (1m 4s)
    • Schriftgröße und Farbe (1m 58s)
  • 4. Werkzeuge für verschiedene Aufgaben
    • Webinhalte mit dem Inspektor analysieren (6m 13s)
    • Webinhalte mit dem Elemente-Werkzeug verändern (7m 38s)
    • Datenanalyse und einfacher Zugang zu Bildern und Videos mit dem Netzwerk-Werkzeug (5m 39s)
    • Verschiedene Geräte simulieren (9m 56s)
    • Screenshots erstellen (4m 2s)
    • Probleme in Webinhalten erkennen und analysieren (4m 54s)
    • Cookies anzeigen und verändern (2m 53s)

  • 5. Königsdisziplin: Webinhalte per Konsole
    • Was ist die Konsole? (4m 3s)
    • Zugang zu Webinhalten mittels der Konsole (3m 27s)
    • Scriptbeispiele: Bilder (4m 7s)
    • Scriptbeispiele: Links und URLs (3m 21s)
    • Tipps zur Konsole (1m 41s)

  • Abschluss
    • Nächste Schritte (1m 13s)

What if the Nazis had the internet and social media?

Wednesday, July 19th, 2023

Book cover of the NSA novel with an eye as an icon in a nazi propaganda styled cover

Going through the bargain bin at a local store I came across the novel NSA by Andreas Eschbach. It is based on the intriguing idea what would have happened if the Nazis had access to the internet, social media, mobile devices and card payment systems. The NSA is a state-owned department of the government before the Nazi takeover that is data-mining the internet and storing all the information.

One of the first bits in the book is a fast forward to when the Nazis had taken over and WW2 is already in its desperate end phase. The NSA department tries to show off to the Nazi regime by proving that they can find out who is hiding Jews. They do this by tracking all the food people bought over a period of time and how many people live in their flats. A huge discrepancy in those numbers indicates that there are probably more people living there than are in the official registry. The Nazis then go on to verify this by dispatching inspectors to a house in Amsterdam. At first they don’t find anyone, but after adding another datapoint by looking up the blueprints of the house they find a hidden family and send them to the death camps. After burning the daughter’s diary as its contents should be a problem if it gets distributed.

Picture of the radio

The book also goes on to describe how the Nazis used computers, mobile devices and online forums to track people, spread misinformation and fire up groups to gang up on people who write about topics that aren’t about purity of race or nationalism.

It is a chilling read, and – if you think about it – not far from what has happened, albeit only with other media. The Nazis were excellent at using Radio and TV to hide the sheer evil they represent with positive messages that pandered to people’s pride. This went as far as creating and distributing highly affordable radio sets into homes called Volksempfänger.

In essence, controlling the media means controlling the masses. And the book describes how having information is as powerful as having heavy weapons. That’s no new insight, of course. George Orwell’s 1984 had similar ideas, after all, and many people, including me, have warned about oversharing for quite a while.

Anti-social media

Why am I writing about this? Because we are not far from a world like that. Social media has become anything but social. People either drown in a river of generated and opinionated content or recede into their own, smaller bubbles where they can talk with likeminded folk without fear of being ganged up on by louts and trolls.

The big elephant in the room, of course, is Twitter and what happens to it now that a single person was able to buy it and change it to their ideals. A lot of people who have been preaching about privacy and evil corporations are now quick to utterly disregard it and move away to other places. None of these places having the same size or impact though. Maybe it is a good thing that Twitter’s monopoly is dwindling, but it is dangerous to assume that people who just want a daily information fix will stop using it and move to other platforms. Platforms that appear much harder to use or only get rolled out in certain locales.

Poster advertising the volksempfaenger radio with the caption that all of germany listens to the fuhrer with the twitter logo overlaid

Twitter has become rough and annoying. It is easy to feel smug about that and wave a huge “we told you so” banner, but that does not change the fact that people still read Twitter and if the calmer and more enlightened voices leave, the hooligans and their puppet masters take over.

The other day someone told me that Twitter is a fascist platform and anyone still using it is a fascist. What a bunch of elitist and defeatist nonsense. If you march alongside fascists in the street without distancing yourself I agree – that makes you a supporter. If you donate money to a platform or a cause that is inherently against people’s rights or freedoms, that’s even worse. I won’t pay for Twitter, that much is for sure. But I keep using it the way I always used it. To me, it’s a publication channel. I don’t go there for information, as the algorithmic timeline made that pointless some time ago. I do comment on people who attack others or point out wrong information and I use it to direct message people and answer incoming messages.

Leaving a place when it gets rough and people spout hatred instead of standing up to them is not that much better than endorsing it. The fact remains that Twitter, much like Facebook, has a huge impact in the media and people do use it. And they are being manipulated by others on these platforms. I see Twitter as a pub that has had an influx of rough folk that shout racist and sexist paroles with an innkeeper that’s asleep at the wheel. If that pub is your hangout, it’s up to you to stay there and work against that takeover. Much like football clubs need to distance themselves from hooligans to ensure that fans of the sport and the team can still safely attend a match.

Fascism works through intimidation. Civil courage is needed to prevent it from becoming the norm.