Christian Heilmann

Author Archive

Social media platforms should make it easier to add alternative text using an [Alt: description] syntax

Wednesday, November 13th, 2024

Adding alternative texts to images on social platforms is not a “nice thing to have” but important to not lock people out. That’s why it is a shame that it is quite tricky to do it across different platforms. Personally I use Twitter, BlueSky, Mastodon and LinkedIn and the following video shows just how much of an overhead this is.

Adding alternative text to images on various platforms

Sure, I could use mass posting tools like Buffer, but I don’t want to. One thing I have seen people do when others do not add alternative text is answer in the thread with something like:

[Alt: An annoyed user with thick glasses screaming at a laptop]

Why is this not a part of social media platforms? Sure, the editors showing the image bigger and offering a text box below give more context, but they mean yet another interaction which is why people don’t add alternative texts. People tend to drag images in or paste them after they’ve written the post. It feels like a lot of extra work having to go to a different editor, when you’re already in a text box.

What about several images?

This gets a bit trickier, but not insurmountable. Just allow a few of them in succession with a linebreak:

[Alt: An annoyed user with thick glasses screaming at a laptop]
[Alt: A beautiful sunset on a beach with frolicking seals]
[Alt: Elmo of Sesame Street saying that he hopes that you choose to be kind]

Maybe one of the newer platforms could lead the way here. It feels bad to see a huge amount of social posts not having any alternative texts at all – which makes them a lot less social.

Exif by magic – the forgotten extra information in JPEG and TIFF files

Friday, November 1st, 2024

I just shot a ton of pictures on vacation and was amazed to see just how much extra data our mobile phones store in images.

This exif data in JPG and TIFF files can be a privacy issue, which I pointed out in my TEDx talk some time ago and even created a tool to remove the extra information in-browser without the need for any server.

However, it can also be a great thing to use in your products.

You can access EXIF information in many different ways.

What data can you get from an image?

Take the following photo:

A mountain of clay in the sea in corfu

If you use any of the solutions above, you can get the following information from the image. For example, in PHP you can do a:

$file = './Corfu-2024-126.jpg';
$exif = exif_read_data($file);
print_r($exif);

This results in:

Array
(
    [FileName] => Corfu-2024-126.jpg
    [FileDateTime] => 1730189825
    [FileSize] => 3497266
    [FileType] => 2
    [MimeType] => image/jpeg
    [SectionsFound] => ANY_TAG, IFD0, THUMBNAIL, EXIF, GPS, INTEROP
    [COMPUTED] => Array
        (
            [html] => width="4032" height="2268"
            [Height] => 2268
            [Width] => 4032
            [IsColor] => 1
            [ByteOrderMotorola] => 0
            [ApertureFNumber] => f/1.9
            [FocusDistance] => 11.92m
            [Thumbnail.FileType] => 2
            [Thumbnail.MimeType] => image/jpeg
        )
 
    [Make] => Google
    [Model] => Pixel 7 Pro
    [Orientation] => 1
    [XResolution] => 72/1
    [YResolution] => 72/1
    [ResolutionUnit] => 2
    [Software] => HDR+ 1.0.641377693zd
    [DateTime] => 2024:10:09 16:37:40
    [YCbCrPositioning] => 1
    [Exif_IFD_Pointer] => 222
    [GPS_IFD_Pointer] => 960
    [THUMBNAIL] => Array
        (
            [Compression] => 6
            [Orientation] => 1
            [XResolution] => 72/1
            [YResolution] => 72/1
            [ResolutionUnit] => 2
            [JPEGInterchangeFormat] => 1303
            [JPEGInterchangeFormatLength] => 24473
        )
 
    [ExposureTime] => 989/1000000
    [FNumber] => 185/100
    [ExposureProgram] => 2
    [ISOSpeedRatings] => 45
    [ExifVersion] => 0232
    [DateTimeOriginal] => 2024:10:09 16:37:40
    [DateTimeDigitized] => 2024:10:09 16:37:40
    [UndefinedTag:0x9010] => +03:00
    [UndefinedTag:0x9011] => +03:00
    [UndefinedTag:0x9012] => +03:00
    [ComponentsConfiguration] => 
    [ShutterSpeedValue] => 998/100
    [ApertureValue] => 178/100
    [BrightnessValue] => 791/100
    [ExposureBiasValue] => 0/6
    [MaxApertureValue] => 178/100
    [SubjectDistance] => 11921/1000
    [MeteringMode] => 2
    [Flash] => 16
    [FocalLength] => 6810/1000
    [SubSecTime] => 251
    [SubSecTimeOriginal] => 251
    [SubSecTimeDigitized] => 251
    [FlashPixVersion] => 0100
    [ColorSpace] => 1
    [ExifImageWidth] => 4032
    [ExifImageLength] => 2268
    [InteroperabilityOffset] => 930
    [SensingMethod] => 2
    [SceneType] => 
    [CustomRendered] => 1
    [ExposureMode] => 0
    [WhiteBalance] => 0
    [DigitalZoomRatio] => 0/1
    [FocalLengthIn35mmFilm] => 24
    [SceneCaptureType] => 0
    [Contrast] => 0
    [Saturation] => 0
    [Sharpness] => 0
    [SubjectDistanceRange] => 3
    [UndefinedTag:0xA433] => Google
    [UndefinedTag:0xA434] => Pixel 7 Pro back camera 6.81mm f/1.85
    [UndefinedTag:0xA460] => 3
    [GPSVersion] => 
    [GPSLatitudeRef] => N
    [GPSLatitude] => Array
        (
            [0] => 39/1
            [1] => 47/1
            [2] => 4862/100
        )
 
    [GPSLongitudeRef] => E
    [GPSLongitude] => Array
        (
            [0] => 19/1
            [1] => 41/1
            [2] => 5362/100
        )
 
    [GPSAltitudeRef] => 
    [GPSAltitude] => 4200/100
    [GPSTimeStamp] => Array
        (
            [0] => 13/1
            [1] => 37/1
            [2] => 38/1
        )
 
    [GPSImgDirectionRef] => M
    [GPSImgDirection] => 76/1
    [GPSDateStamp] => 2024:10:09
    [InterOperabilityIndex] => R98
    [InterOperabilityVersion] => 0100
)

This is a ton of information to play with. The classic is using the GPS data to show images on a map . If you click on the different thumbnails, you can see the map moving to where the photo was taken.

Map mashup moving a map around to where the photo was taken.

To get this information, all you need to do is read the data and then convert it.

    $exif = exif_read_data($file);
    $lat = $exif['GPSLatitude'];
    $lon = $exif['GPSLongitude'];
    $latref = $exif['GPSLatitudeRef'];
    $lonref = $exif['GPSLongitudeRef'];

One thing I had not done before though was reading the altitude information. This allows you, for example, to show the images on the height they were taken.

Showing the elevation of different images

As the data is returned in a format like “4200/100” you need to do some parsing. This one creates an array of all images with the image name and its height in meters.

$max = 0;
$all = array();
foreach ($jpgs as $jpg) {
    $gps = exif_read_data($jpg);
    if (isset($gps['GPSAltitude'])) {
        $parts = explode('/', $gps['GPSAltitude']);
        $height = round($parts[0] / $parts[1]);
        $jpgs[$jpg]["height"] = $height;
        if ($height > $max) {
            $max = $height;
        }
    }
    array_push($all, array($jpg, $height));
}

Using the embedded thumbnail information

One interesting use case is to use the embedded thumbnail information to avoid having to create thumbnails. If you check the gallery or the height example and you keep developer tools open, you can see that whilst the images are all a few megabytes, the page only reads a few hundred kilobytes.

The gallery with browser developer tools open, showing that only 500 kilobytes were loaded whilst there are several megabytes of images.

This is because we don’t load the images, or created thumbnails from them, but I use the exif thumbnail information in the JPG file instead:

[COMPUTED] => Array
    (
        [html] => width="4032" height="2268"
        [Height] => 2268
        [Width] => 4032
        [IsColor] => 1
        [ByteOrderMotorola] => 0
        [ApertureFNumber] => f/1.9
        [FocusDistance] => 11.92m
        [Thumbnail.FileType] => 2
        [Thumbnail.MimeType] => image/jpeg
    )
…
[THUMBNAIL] => Array
    (
        [Compression] => 6
        [Orientation] => 1
        [XResolution] => 72/1
        [YResolution] => 72/1
        [ResolutionUnit] => 2
        [JPEGInterchangeFormat] => 1303
        [JPEGInterchangeFormatLength] => 24473
    )

Originally, this thumbnail was meant to be displayed on screens built into digital cameras to have a faster preview. We can, however, also use this to preview the image without having to load it in full. In PHP this is the exif_thumbnail() method.

$image = exif_thumbnail('Corfu-2024-991.jpg', $width, $height, $type);
if ($image !== false) {
    header('Content-type: ' .image_type_to_mime_type($type));
    echo $image;
    exit;
} else {
    echo 'No thumbnail available';
}

This will not load the whole JPG file, but only as much as it needs to get the thumbnail. This is not news. Flickr, for example, used this in the early 2000s to show you a thumbnail of the image immediately when you dragged it into the browser and then started to upload the rest. However, I have not seen it used lately. Maybe because it is too easy to create images on the cloud.

Playing with Exif is fun. And why not – after all, our phones and cameras add this information anyway.

I just pulled a 2006 and uploaded my holiday photos to Flickr with a Creative Commons Licence

Friday, October 25th, 2024

I just returned from a holiday on the gorgeous island of Corfu in Greece and spent quite some time taking photos. Instead of releasing those piecemeal on various social media channels, I thought it would be fun to go back to our ways of early social media, and put them all up on Flickr with a CC licence so you can use them. Some make great materials for image manipulation, others are great backdrops for your video calls.

You can see them all on Flickr :

Overview of the six albums on Flickr.

Here are the albums:

For those not on Flickr, you can also get them as Zips:

I really miss the early days of social media and especially the Flickr community. I was lucky enough to have worked with the people who built Flickr and then joined Yahoo and the community aspect of the product was what really made it work. You uploaded a few high quality photos and people would tag themselves and others in them. People would add descriptions and comment in full sentences and not just likes. And the thing that made Flickr really special was that everything was API driven and it was incredibly easy to embed the images in other products and create mashups.

The idea was to add to a web of data and allow your creative output to be re-used, mixed and added to other materials. Not to feed one addiction machine and not even have access to your own work a few seconds later. It is a shame that we lost that, and the bigger issue is that any system that allows for tagging or adding comments drowns in spam and AI slop within seconds now.

We skipped the “social” part of Social Media quite some time ago. But hey, maybe you have some joy seeing how gorgeous this island is!

Let’s bring back browsing

Sunday, September 15th, 2024

When the web started one of the best parts about it was the naming of things. To “surf the web” implied fun and adventure and to “browse” implied serendipity. And we seem to have lost that. Let’s go back.

When I discovered the internet it was pretty much just taking off. I didn’t go to university, so I had no means of access from the get-go. I had a modem, a Commodore 64 and access to BBSes. Then I got an email and newsgroup access. And when I started working at a radio station as a newscaster, I had internet access with a browser. That was 1996 or so, and soon I had my own internet access at home. Still with a modem and paid by the minute. Many Americans can’t understand that issue, as local calls were free, but in Europe we got online, surfed a bit and disconnected. Then you checked the browser cache for pictures to keep. It was arduous, expensive but also, and maybe because of that, exciting. It gave you a feeling of taking part in a new world of publishing. And that’s why it became my career.

There were no big search engines, there were a few portals and news sites, but most of the discovery of web sites was word of mouth on mailing lists, IRC, newsgroups and of course links from one site to another. Getting linked to by others was amazing and linking to others showed that you cared and also showed proof that others also cover the things you wrote about. Web rings and banner exchanges were also great tools and I spent a lot of time making that 400×40 pixel GIF less than 10k.

Finding information on the web was a journey, an adventure. And people wrote about the most random things, went down many rabbit-holes and of course also published things nuttier than squirrel droppings. But here’s the thing: for the first time other than fanzines and underground magazines people couldn’t stop you from publishing. And people wrote what they wrote because they were genuinely bonkers, and not sponsored to manipulate elections.

Nowadays the web has taken over the mantle of most in your face medium trying to force you to consume and purchase things. And it “does the research for you” and pushes you into bubbles. Spending time aimlessly browsing for content is touted as inefficient. Operating systems and browsers come with “amazing AI features” that give you summaries of content instead of allowing you to get your own impressions and draw your own conclusions.

Sure, on the surface this seems great, but it feels like we’re pushed into a world of faster and faster consumption without allowing us and our minds to wander.

The journey towards information is important. Humans retain information better they had to put effort in to get. Aimlessly browsing to find things you may not have heard of yet is as important as discovery is exciting.

I love getting lost on Wikipedia, going from one weird fact to another.

I used to love that on YouTube before everyone and their pet wanted to become an influencer and follow formulaic and manipulative patterns to create their content.

I do enjoy Spotify’s Radio and mixes feature that allows me to find music closely related to bands I love and didn’t know about.

Amazon can get in the sea. The start screen of my Fire TV right now shows me an ad for shower gel and all the programs that look exciting are in reality ads for other streaming services. That’s just TV, but less honest about it. Which is annoying as the x-ray feature of the Amazon Prime player is great. It adds IMDB information to every scene and you can look up where you know that actor from and what that song is, that is currently playing. You know, things you browsed for in the past. Some programs even use it as an Easter Egg, for example The Boys. More of that, less of “this is in cinemas now” nonsense.

We need to browse more, find things we haven’t looked for and discard or embrace them. You don’t often go to a clothes shop to buy one special item. It’s fun to try on a few things and maybe find a new style. You don’t go to a book store and buy one special book. You compare, you try, you might see that you missed a new release of an author you liked. You spend time looking around and taking things in. And that can spark creativity. Or, as the Beastie Boys put it:

Mike D and AdRocck of the Beastie Boys on stage with the caption "fucking around became our creative process"

Let’s be browsers again, let’s embrace the weirdness of the web, a library curated by racoons on drugs. In the newsletter I curate, I have a section called “procrastination corner / weird wide web” where I collect wonderful finds of my browsing. I love that they still exist and I also keep an archive.

So let’s keep looking around for the strange, the overly detailed and the just bonkers web. And – hey why not – start contributing to it. You can publish, nobody is stopping you.

A billion new developers thanks to AI?

Thursday, September 12th, 2024

This is a translation of my German article for the AI mag.

Demetris Cheatham of GitHub on stage with the 1 billion developer road map

At the WeAreDeveloper World Congress in Berlin in July, GitHub announced that the company will use artificial intelligence and assistants to turn a billion people into developers in a very short time. Amazon’s Cloud CEO, on the other hand, explained in an internal fireside chat that soon no one will have to develop software anymore, as machines can do it better anyway. Two big opposing statements, and you have to ask yourself who will be right and what that means for developers and those who want to enter this market. So let’s take a quick look back, a look at the current situation and what may come.

Being a software developer in the past

I myself have been a professional, well-paid and sought-after developer since 1995. I worked for several years at Yahoo, Mozilla and Microsoft and worked with Google on the Chromium project. At the beginning of my career, it was frowned upon that software that I wrote during my working hours and on company computers should be made available to others for free, and much of what I worked on was solely for internal use.

Open Source as an introduction

But soon Open Source came around and changed everything. First it was a tool and idea only for geeks, but later it proved itself as a sensible approach to software development. The cloud runs mostly on Linux, Android is numerically superior to iOS, and a huge part of the web is based on WordPress.

Open Source and Creative Commons were a thing that just made sense to me. I didn’t want to be the only one who understood and edited the software. I wanted others to look at my work, check it, and take over when I no longer had time or muse to keep this project going.

Open Source allowed thousands of developers to start their careers. As a lead developer, it meant that I didn’t have to look for new employees, but could hire them from inside the project. I could see who used or contributed to the project. I already had insight not only into what these new employees are technically capable of, but also how they documented their work, dealt with criticism, and how they communicate in the team.

When I started with Open Source, Microsoft was still the evil empire. When I left Mozilla to join Microsoft in 2015, the main reason was the promise that I would help bury Internet Explorer, a dream I had had as a web developer for a long time. At the same time, Microsoft released Visual Studio Code. An Open Source code editor that would completely revolutionise the developer world in a very short time.

When rumours started that Microsoft would buy GitHub, grumblings went through the developer community and many predicted that this would be the end of the platform. I saw it differently. GitHub was a wonderful platform that simplified version control and allowed anyone to create a community around their software product in a very short time. But it was a startup from San Francisco, and many traditional European companies would never put their software or data there. With Microsoft as the corporation behind it, that was a completely different story.

What I’m trying to say is that Open Source has long since made it possible to democratise software development and allow anyone to start a new career as a developer. Of course, the web is the other big invention that helped with this before, but now you can use professional tools that the big players in the market use for free and also collaborate on them.

So far, so good. But then came the AI ​​hype and suddenly we are back in the middle of a boom reminiscent of the .com bubble at the beginning of the millennium.

Welcome to the world of AI hype

The idea of ​​artificial intelligence is nothing new, but there are currently only two major differences:

  • Computers are fast enough to provide the power that AI systems need
  • ChatGPT, LLMs and RAG have made AI accessible to everyone and are currently being used everywhere, whether it makes sense or not.

Conversing with a seemingly intelligent machine and obtaining information this way is a dream for anyone who grew up with Star Trek. It is also a major change in the general approach to computers and knowledge. While people used to read books and then find websites via portals and search engines, today they ask the machine and get an answer immediately. And if the answer is not right, they can ask for more information.

Software as a mass product generated by AI?

Whenever there is a change in user behaviour, CEOs of large companies turn into fortune tellers predicting the future. When smartphones came along, everything had to be an app, because only these could provide the end user with the best service. When digital assistants such as Siri, Cortana, Bixby, Alexa and so on were introduced, the prediction was that soon there would be no more apps, but that these assistants will be able to fulfil all our wishes. The model here was WeChat in China, which really is the solution for everything there. However, this is also a market where the free Internet is not available.

Now many people predict that every software solution could be an extension for ChatGPT.

Every time that happens, there is immediately a marketplace where you can offer your extensions or apps. These marketplaces soon turn into digital rubbish heaps, as companies create hundreds of apps automatically, digital attackers present viruses and trojans as legitimate offers, and there are hundreds of cheap copies of every successful product.

In other words, software is becoming a mass product and many people are being fooled into believing that they, too, could become millionaires tomorrow with the killer app.

Often, however, only the market operators benefit from successful offers, and in the case of AI extensions, there have recently been a lot of cases where successful ideas were simply offered in the system itself, and the entrepreneur suddenly saw all users disappear. However, this is nothing new, as it has often been the case with browser plugins and developer environments.

If you look at it from the outside, it is similar to streaming services. In the past, you bought the CD or DVD, but today you have immediate access to everything. But you also have no claim to the content and cannot rely on finding it again if you want to watch it once more. Just like you don’t always get the same answer from ChatGPT, but sometimes odd answers. The correct thing wasn’t available, so here’s, well, something.

Whenever a new technology is supposed to conquer the market, you hear the same statements. One is that it will very soon be possible to create great software solutions without having to program a single line. This was already the case in the days of Visual Basic or later with WYSIWYG (“What You See Is What You Get”) environments such as Frontpage or Dreamweaver. Today there are a whole lot of “low code” or “no code” solutions with the same promises, which make it easier to create products, but also deliver highly unoptimised results.

GPT demo turning a hand drawn app on paper into HTML, CSS and JS

Of course, this was predictable to be a statement in the AI ​​field and one of the first “Wow” presentations from ChatGPT created a web application from a design scribbled on paper. Later, “Devin” was the first fully effective software developer as an AI. Both brought a lot of big headlines and applause, but with Devin in particular it quickly became clear that it was a nice presentation, but not really a solution.

Who needs developers?

Whether we even need developers anymore depends on what we want to create. ChatGPT’s “From Paper to Code” demo application was a website that displays a joke at the touch of a button. Nobody needs this application, and it is feels much more like a lesson of a programming course. And even as an interview question, this app would be 15 years too late to test the knowledge of candidates.

If our job is to create solutions like this, we don’t need a professional developer. But we don’t need AI either, because low- and no-code products could do those for years.

It is true that a lot of the work you do as a developer is based on existing products. And if it really is just about assembling existing components, then an AI can do that, too.

However, there are also a lot of problems that require more complex, human solutions, and for those, we need trained developers. Throughout my career, I have noticed more and more that writing a program is the smallest part of the work. Rather, it is about developing software that is understandable and accessible to humans, and that is a task that AI cannot do for us. Accessibility and usability cannot be automated, no matter what advertising for great new software promises.

How do developers learn?

For every job, you need the right tool. In the case of software, that is the development environment. It should make it easy for me to write code, find errors, make changes and – if possible – immediately see the results. A good development environment tells me while I am writing that I am making a mistake or how to use a method. Similar to how a word processor underlines errors whilst I type.

If I want to learn about syntax, names of methods or how to approach a problem, I can consult documentation. Books, online documentation, courses, and also videos. And there is a lot of it available. It is almost a full-time job to distinguish the good from the bad.

And that is why there are forums and social media on which you can exchange ideas.

When GitHub came up with the idea of​ GitHub Copilot for VS Code, I was immediately hooked and from day one I was a tester, helping to find bugs and request new functionality.

The great thing was that I didn’t have to go to a website like ChatGPT to ask questions about programming. Instead, it happened inside my development environment, as suggestions on how to continue the feature I was just starting. I can also highlight part of the source code and ask the AI ​​what it’s all about. I used to do this on forums or as a comment on GitHub. I learned whilst I was programming, and thus created a lot more in less time. I could also tell the system to only refer to the current project and not to give me some result from the internet. Furthermore, the system learns from me what I expect. The more I used Copilot, the more it gave me suggestions in a format I would have written anyways. It started to mimic my style, instead of offering random suggestions.

In other words, the research tasks are automated and part of the work. And that’s where GitHub has a clear advantage over others, which is why they have the chance to fulfil the big task of turning a billion people into developers.

GitHub is where I store source code, I can edit the code in the browser with one keypress, and I have access to a huge number of experts who also communicate on the same platform. All the learning steps inside one environment. There are more players that offer that now, but GitHub has the advantage of being a huge community as well as a platform.

But the technical part of development is only a fraction of the task. A large part of my job as a developer is to filter and convert data. You never get perfect data, and good software is written defensively, testing the input, expecting false information and filtering it. And that’s where AI in its current marketing form is a real problem.

AI creates naive code

If you present a great new solution to the world, it can not make any mistakes. And that’s a problem with the AI ​​hype at the moment. Instead of a chat bot not giving me an answer, or simply admitting that more information is needed, most systems return something. In chat and when creating images or videos, these are so-called “hallucinations”. In code generation, these are either the first results from the official documentation, or those that were chosen as the best by most developers. And that is not the best solution, but the simplest.

Many AI code generators are based on data from forums such as Stack Overflow, Reddit, official documentation, and personal blogs of well-known developers. The problem is that most of the time a solution is shown that represents the simplest and fastest result, and not the safest or optimised one.

For decades, I have written courses and books, and every publisher or platform wanted exactly that: give the participant a quick, positive experience by showing a simple example without pointing out all the nuances right away.

These examples were also the ones that were voted as the best by the community on forums, because they are simple and give an immediate result. Forum participants did not want the “why”, only the “how”. And these are the code examples that an AI chat bot shows as the first result. And even if users tell the bot that this is not a good result, the underlying model is not changed because that would be too expensive and time-consuming.

A lack of transparency

The hardest thing is to find out where the bot got the solution it is offering. Instead of laying its cards on the table, the AI ​​sector is currently thriving on stealth. When millions of investments are at stake, people like to hide what makes their product special. It actually started well. OpenAI, as a prime example, was initially open and changed later to inspire higher investments. But it would be in the interest of the end users to know which data the models are based on so that you, as the original developer, could explain why it is a bad example or update and fix known security or performance problems. For example, I know which of my Open Source Github repositories have been read and taken over by AI bots, and some of them were very successful, but only because they were a funny trick or a very dirty shortcut.

Who owns the source code?

There is currently an arms race on the Internet about how to protect your open content from being taken over by AI bots. A lot of companies have already been sued for, for example, ingesting YouTube content without paying attention to the license or asking the owner. While many developers releasing their work as Open Source have no problem with others building on it, it is a different matter when a machine comes along and uses your code in chatbot answers as part of a paid-for service without context or accreditation. There are a lot of blocker lists that are supposed to protect your own blog or source code repository from getting indexed. However, AI ​​providers do not identify their crawler bots and mask themselves as normal browsers. After all, it’s about being able to offer the most data, not about ethics or adhering to licenses.

In a lecture at Stanford, the former CEO of Google recently explained without any beating around the bush that it’s totally OK to steal content when it comes to innovation and getting to market quickly. Entrepreneurs shouldn’t worry about it, but leave it to the lawyers. Brave new world.

Europe as a second-class market?

Europe has many rules and laws that can be considered detrimental to some Silicon Valley startups, and in my work with American companies I have spent a lot of time explaining GDPR and similar things to my colleagues and apologising for not being able to show user information because it is illegal to record it in Germany without their knowledge. That’s good, the privacy of our users and their security is the most important thing. But that just doesn’t fit in the world of explosive growth and rapid software distribution. We are currently at a crossroads where more and more AI systems and products are either not being offered at all or months later in Europe.

Politics isn’t helping either. Historically, Europe has always had many Open Source companies and developers, but with the cut in OSS funding from the European Union, many of these providers will have to find other ways to pay the bills. And that will make it difficult to compete against companies in other countries with fewer laws when it comes to finding investors.

In general, the problem is still that many people think that Open Source is free. An old idiom goes that OSS is “free as in a puppy”, so if you get a free puppy, that’s great, but you also have to take care of it. You have to train the animal and there may be accidents on the carpet.

One of these accidents recently shook the OSS world. An important Open Source component in almost all systems, xz, was almost replaced by malware that could have infected all Linux machines. The problem was that the original developer no longer had time to maintain the product and handed it over to a maintainer. This is completely normal behaviour in the OSS world. But the maintainer turned out to be someone who planned to replace the component with malware and took his time to cover his intent. We have to ask ourselves now how to ensure maintenance of system-relevant components without worrying about similar security concerns in the future. And that will be difficult without financial support.

The European Artificial Intelligence Act (AI Act) came into force on August 1st and is intended to regulate the AI ​​world, bring more transparency and allow European companies to be well positioned in international competition. However, it also poses a major problem for Open Source offerings, as these are exempt. One of the reasons was security, as open systems are easier to attack and can be used for nefarious reasons as well as legitimate ones without any feedback or ask for permission.

Security through obscurity?

In the IT security sector, there is a thing that has always been a lie: security through obscurity. Just because you can’t analyse closed source systems directly doesn’t mean that they are more secure.

Recently, there have been increasing reports that all closed AI systems have been attacked and data has been lost. Many code generators have also been used via prompt injection to offer unsafe code to end users and thus install malware. Microsoft in particular has been in the crossfire of the media and has now even made bonus payments dependent on the impact employees have on the company’s security. Interestingly, this came a few months after many of the security experts were laid off in the 11,000 employee layoff wave.

These and other problems such as the Azure Masterkey loss and the Crowdstrike outage have also damaged developers’ trust in the cloud and large companies, and almost all lectures or articles about AI warn against relying on just one provider. Which of course also means that you either have to spend more or rely on locally installed systems. These would then have to be Open Source.

What can happen now…

GitHub has set itself an ambitious task and is well positioned to achieve it. The only question is what a “developer” really is in the age of AI. Discussing this requires a whole separate article, as there are many facets to this.

What most companies are hiding, however, is that the AI ​​business model does not work right now. Most companies are currently paying extra – the revenue of various Copilots and systems is not enough to cover the computation cost. The technical costs are insanely high. Before the GenAI revolution, almost all large companies advertised that they would soon be “carbon neutral” or only dependent on green energy, but this has not been the case quite some time and all have become suspiciously silent about the topic. Generative AI is currently an insane waste of energy – every image created requires as much electricity as charging a cell phone.

Therefore, the AI-on-device idea will become more and more interesting. Instead of hosting the models in the cloud, all Open Source AI models can also be used locally, and Google, for example, is already toying with integrating Gemini into Chrome. There are also some Open Source projects that offer AI chat systems without cloud dependency.

In general, however, these are very interesting times, and the market always needs more developers. I don’t think developers can be replaced yet, and I do think that intelligent and easily accessible development environments give a lot of new people the chance to get involved.

The question is, how do I turn these newbies into developers who can also be proud of their work, and what can we do to make the next learning steps appealing to them after the AI ​​says “take this and everything will work”.

Link resources: