Christian Heilmann

Author Archive

TTMMHTM – Apps, Competitions, libraries, how to run a company and more

Friday, August 6th, 2010

Things that made me happy this morning (a lot this time as I was offline for four days and had 670 unread Google Reader items)

UK Government says no to upgrading IE6 – who is to blame?

Thursday, August 5th, 2010

Back in June Dan Frydman of Inigo Media Ltd submitted a petition to the UK government to encourage government departments to upgrade from IE6 and 6223 people signed it.

A short time ago we got an answer by her Majesty’s government which was a no – of course.

Government says no

Disregarding the horrible PR mumbo-jumbo re-assuring us that the government takes security serious (when they are not leaving personal data files on trains) it gets actually interesting:

Complex software will always have vulnerabilities and motivated adversaries will always work to discover and take advantage of them. There is no evidence that upgrading away from the latest fully patched versions of Internet Explorer to other browsers will make users more secure. Regular software patching and updating will help defend against the latest threats.

This of course is a wonderful example of stating the bleeding obvious, but it is interesting that there is “no evidence that upgrading IE6 makes computers more secure”. I wonder why Microsoft then keeps advertising that IE8 is more secure? True, IE6 can get all the patches for massive attacks but phishing warnings and other interface changes in latest browsers do not get added. So we protect users under the hood but we still leave the barn door wide open for social engineering attacks. A malware warning like Firefox, Chrome or more modern IEs have would help there (unless it gets removed when it affects advertising). If there actually is no proof it would be a good opportunity for Apple, Google and Mozilla to collect some numbers and publish them – not on blogs or other “in crowd” media but in the magazines read by the people who make IT decisions for governments and large corporates.

Security patching not an issue?

The government statement then continues to stress the great relationship they have with Redmond for security related matters:

The Government continues to work with Microsoft and other internet browser suppliers to understand the security of the products used by HMG, including Internet Explorer and we welcome the work that Microsoft are continuing do on delivering security solutions which are deployed as quickly as possible to all Internet Explorer users.

There is a distinct lack of information about what they are – both the other suppliers or the measures. My guess is that Google is starting to approach governments with Chrome and the online office suite. Let’s note down though one thing here: that there is no problem to deploy fixes very quickly to all IE users – we will go back to that.

No centralised security mandate?

Each Department is responsible for managing the risks to its IT systems based on Government Information Assurance policy and technical advice from CESG, the National Technical Authority for Information Assurance. Part of this advice is that regular software patching and updating will help defend against the latest threats. It is for individual departments to make the decision on how best to manage the risk based on this clear guidance.

So, wait – beforehand we were told that there is continuous patching with ease as Microsoft helps a lot and now we learn that it is up to the department to really follow that advice. It is not a mandate, but only a guidance. This means actually that there are probably terribly outdated IE6 in use as changing the IT infrastructure is quite low on the list of priorities for a lot of departments when there are people in the waiting rooms complaining. Which means that if upgrading and patching is not centrally mandated there is no chance we’ll ever have a secure and homogenous IT environment in government bodies.

A departmental decision?

Public sector organisations are free to identify software that supports their business needs as long as it adheres to appropriate standards. Also, the cost-effectiveness of system upgrade depends on the circumstances of the individual department’s requirements.

Which means that a department could switch to other software – especially when they could save money? The catch here is “appropriate standards” which probably means a EULA. Or what, exactly? The other big “oh well, we really can’t do that, can we” here is the cost-effectiveness of a system upgrade. In many cases of Microsoft systems this probably means that the hardware in use is not up to scratch to support other OSes than Windows 2000 or XP1.

Upgrading is an issue?

It is not straightforward for HMG departments to upgrade IE versions on their systems. Upgrading these systems to IE8 can be a very large operation, taking weeks to test and roll out to all users.

How so? Earlier we heard that patching IE is not an issue, so how is replacing IE an issue? Unless of course we’d own up here and admit that it is the infrastructure and the hardware that was defined and set in stone around the millennium when all were scared about Y2K and believed that the IE6/XP Suite will never have to be upgraded.

No time for testing?

The other issue seems to be that testing our systems is hard:

To test all the web applications currently used by HMG departments can take months at significant potential cost to the taxpayer. It is therefore more cost effective in many cases to continue to use IE6 and rely on other measures, such as firewalls and malware scanning software, to further protect public sector internet users.

This to me says that there are systems that were built in a short-sighted manner a long time ago – for IE6 and windows 2000 when they were the new black and every consultant got his Microsoft certification training and out of a sudden was a real expert who can predict the future of the next 10 years. So instead of fixing and replacing the rotten core of the system we add new doors with shiny hinges and a security guard before it and it will be fine. This is like hiring a bouncer for a club where people fight on the dance floor.

The fascinating part of the firewall and malware scanning software is that it makes the life of the end users even more hell than surfing with IE6 already is. One of my favourite things when I switched to Mac/Linux is that my processor can now deal with stuff I want to do rather than analysing my traffic and incoming requests and that I can work without being interrupted by a “scanning all your files, come back in 2 hours” message.

Who is to blame?

The answer of the government was not only predictable, but (in a very shortsighted and limited view) also understandable. Nobody wants to own up having been cheated. And consultants telling people that a network will never have to change do cheat people – no software is 100% future-proof and you cannot run an office on 10 year old hardware without upgrading. The speed of innovation and wealth of information we encounter these days can not be easily consumed on systems that were meant to be used when having a 100kb JPG on the homepage was a huge decision and meant you lost 1/3 of your visitors.

Funnily enough the easiest and favourite target of web geeks in this issue – Microsoft – is not to blame. They do offer a simple way to make their new software support IE6 with a meta tag or – much more appropriate – with a header send by the server (IIS in this case). So the argument that software built for IE6 has to be tested by every department on IE8 is moot as Microsoft solved that issue for us. That the government probably didn’t even know about that option is where it gets interesting:

Reactions like this to an obvious upgrade are our fault

To a degree I have to say after all my years on the web and as a developer, writer, blogger and editor we are the first to blame for no movement in large corporations and the government.

When luminaries of the web design and web development world only showcase things made up to use a certain new technique instead of real world examples it is not surprising that developers working for government agencies don’t get sent to conferences or get their books.

When famous designers say that working for a large company or government is “boring work” and “that there is no point for a creative person to deal with politics in companies” then I really wonder if we have become self-sustaining and complacent. We moved on from shaking the foundations of web development and making people understand the massive opportunity the web as a media and the open web technologies as tools represent to inventing for ourselves rather than for the end user. What will have more users who are much more frustrated when something doesn’t work? The readers of a famous design blog or people who have to pay their council tax online?

When industrial grade research information and tools from companies like Yahoo, Google and Microsoft are never read or – even worse – reproduced in a shinier but less consistent manner by one man army companies and considered to be better (until the one man army is bored of it a month later and never updates) then there is no wonder that other companies don’t believe in these solutions either. Furthermore it means that these companies – who really formed and run the internet as we know it now – will stop sharing their tricks or spending time and money writing them down in a manner that makes sense for people not on the inside.

Shifting our focus

The only way that I can see how responses like the one from the UK government can be prevented in the future is by shifting our focus:

  • Instead of design prototypes and made-up web sites to show a certain technique let’s demand real production case studies and their effects (I remember one @media where the redesign of blogger was shown and how much traffic shifting to CSS saved the company – more of that, please).
  • Ask Microsoft to invite experts, host videos and tutorials of experts with modern solutions and distribute them on their network of clients
  • Make a massive comparison of government web sites and praise what some have done well (nothing works better than competitiveness)
  • Collect success stories of switching to open source solutions and how it saved money and time
  • Take a horrible IE6 only solution and show what it could look and work like if HTML5 and CSS3 were supported
  • Stop plotting shiny pixels on canvas elements and call it a cool HTML5 solution and instead build a complex online form or spreadsheet system using all of the goodies of HTML5
  • Stop applauding people for redesigns of their blog and instead shift people into the limelight who made a difference in an environment like large financial systems or local government

I’ve had these and other points in 1:1 discussions for years now and I yet have to see movement in these areas. Right now, we are happily thinking we innovate and push the envelope where in reality we are making each other go “Oooohhhh” while a large chunk of the audience that could benefit from our knowledge is stuck with really poor experiences on the web. I’d like to pay my council tax on my mobile phone’s browser and get notified when I need to do it – right now there is no way to do that.

Adding a world globe and location information to your site with YQL

Sunday, August 1st, 2010

Whilst looking around the open tables in YQL I found a table with earthquake information released by the United States Geological Survey. One thing the RSS feeds returned from that service had was quite a cool picture of Earth with the location as a star:

Example of the globes rendered by the USGS web service

Looking at the source I realised that the image URL has a certain logic to it:

http://earthquake.usgs.gov/images/globes/50_40.jpg

The first number is the latitude, the second the longitude of the location. Each of them need to be multiples of 5 to result in an image. Try it out by changing the values.

Using this, I put together an open YQL table to render some HTML that shows a the globe image and the information the Yahoo GeoPlanet web service has available about that location.

You can use the table with the following YQL statement:

select * from geo.globeimage where place="sfo" and type="data" and location="true"

Open this in the console here or see the results as XML.

The different parameters are:

place
The geographical location, like SFO for San Francisco Airport or London, UK for London, England
type
the type of the image. If you provide data as the parameter the image gets returned as inline data. This renders the badge much faster as the image doesn’t need to get loaded from the USGS server.
location
A Boolean if want to show the list of location information or not

The above statement would render the following HTML:

sfo
  • Name: San Francisco International Airport
  • Placetype: Airport
  • Country: United States
  • Latitude: 37.614761
  • Longitude: -122.391876
  • WOEID: 12521721

In order to use this without going through YQL, I’ve put together a small JavaScript:

globebadge.init({
element:’ID or reference of element to add the badge to‘,
location:’the geographical location you want to show‘,
showlist:true or false – if set to true the script displays the place information as an HTML list.
});

For example:

globebadge.init({
element:'badge',
location:'Batman',
showlist:true
});

This will render in your browser like the following image:

globebadge

You can find the source of the badge script on GitHub:

Notice that I am testing for the browser. If we have IE6 I do not return the image as a data URI, otherwise I do.

If you want to see it in action and try it out with a few locations, check out the demo page for Geoglobes.

You can see the globeimage open table for YQL at the YQL table repository:

Another example how you can find cool stuff and then turn it into a web service with YQL :)

The Hackday Toolbox – getting you started faster

Thursday, July 29th, 2010

Just having spent a lot of time at the amazing open hack day in Bangalore, India I found that most of the questions about starting a hack using Yahoo technology revolved around a few issues:

  • How do I access data on the web/from web services?
  • How do I use YQL from JavaScript or PHP?
  • How do I display information I received from YQL with PHP or JavaScript?
  • How do I get the location of the user and how do I analyse content for geographical locations?
  • How do I access oAuth authenticated information of Yahoo?
  • How do I set up PHP and where can I see errors?

So, to avoid having to repeat myself again I put together The Hackday Toolbox which contains sample code that deals with all these issues.

The Hackday Toolbox

The hackday toolbox contains:

  • An introduction to installing and using PHP with MAMP/XAMPP and debugging it
  • YQLGeo for all your geo and location needs
  • Demos of querying YQL in JavaScript, YUI3 and PHP
  • Demos to display YQL data
  • Authenticated example to access the Yahoo Firehose
  • Rendering Yahoo Geoplanet data as a map

You can download the Hackday Toolbox on GitHub or try the examples.

The toolbox is BSD licensed, so if you want to add Java/Ruby/Python/Heskell/Pascal/Logo/Fortran/6502 Assembly code examples, please do so.

I put my hack in a box…

Yahoo Open Hack Day India 2010 – here are the winners and some hacks that impressed me

Monday, July 26th, 2010

The third open hack day in India is over and here I am in my hotel room waiting to be picked up for dinner before flying back to London. The last three days were a blur. I talked a lot about hacking, explained technologies and saw a massive avalanche of interest and questions and people taking photos of me.

Open Hack Day Bangalore 2010 by Christian HeilmannOpen Hack Day Bangalore 2010 by Christian HeilmannChristian Heilman by Balaji Narayanan by Sourav BasuOpen Hack India 2010 by Subramanyan GuhanOpen Hack Day Bangalore 2010 by Christian HeilmannOpen Hack Day Bangalore 2010 by Christian HeilmannOpenhackday by

Describing the hack day would be such a collection of superlatives – it is almost untrue. There will be an official list with all the cool numbers, but let’s just say we crammed the hotel with hackers and broke the records of all the other 12 open hack days. The amount of hacks delivered were more than last year’s India hack day and the London one together!

As a passionate speaker and trainer, I have to say I was very happy coming here. Events like these show that there is still a lot of drive and innovation in the idea of hack days. After 24 hours of hacking and a few hours of judging, we had the winning hacks of this year:

  • Github Badges (source) by Brian Guthrie, Tejas Dinkar and Mark Needham are a collection giving warcraft/xbox style achievement badges for github achievements.
  • Quizr by Prateek Dayal and Hemant Kumar is a quiz generator using Wikipedia and Flickr. The generated quizzes get pushed out to all the computers in the room live via HTML5 WebSockets.
  • FlickrSubz by BabuSrithar, Sudeep Nayak and Parashuram enables realtime closed-captioning in multiple languages for videos on Flickr. The hack utilizes a speech recognition engine (Julius for linux, WSAPI for windows) to display subtitles in the chosen language (translate API) for videos on Flickr via a GreaseMonkey script.
  • ChromYQLip (pronounced as Chromy-Clip) by Markandey Singh is a chrome extension for page scraping. Select some text on a page and click the extension icon and it will populate the URL and xpath of selection. Click “getmashup” to get a lightweight page which loads your content. A Sample URL and XPath for advanced mashup building is URL=”http://twitpic.com/photos/$1” Path=”//div[@id=”image-”]/div/div[1]/a” which results in $1 to become a form field to enter the TwitPic user name.
  • Communicator by Mohan Gupta, Sri Ram and Roshan is an API to include a real-time communication widget on any webpage. All the users viewing that page can discuss , talk and collaborate on the content of the page in real time.
  • Chirpshire by Preetham Venkky, Rohit Talukdar, Puneet Jaiswal and Mohd. Amjed allows you to gain belts and grab badges for tweeting regularly and without using automation apps. Businesses can use this service to spread a meme. This could be a # hashtag or a physical location check-in.
  • Shop Green by Nidhi Chaudhary and Anurag Jain is an interesting concept to allow sellers to print 2D barcodes for their products and buyers to simply scan them with their mobile phone and pay on the phone. No need for paper bills any longer. All the payments are made with PayPal.
  • Democracy Tools by Ankur Patel, Ankur Gupta and Yatin Kumbhare did quite a job of scraping all kind of government sites to collect data to answer the following questions: Who is your Leader? Where is your Constituency? Is there a government Website Search Engine? What is Media’s Opinion about your Leader? Another hack that did something similar is RepMeter
  • How Much Time Will This Landmark Take Me? by Susheel was a terribly clever hack that analysed the EXIF data in flickr photos to see how long it took people to take photos at a certain landmark. That can give you an insight into how long it will take you to look at that landmark on your next trip.
  • Nirvana – your late night path back home is a mashup that allows people to tweet where the police currently does alcohol tests – in case you want to avoid that driving home.

Here are a few other hacks that stood out for me. There were a lot more but as the hackers failed to submit real links for me to check (even after I pointed this out repeatedly in the keynote) I cannot verify if they really work.

  • PixMos by Adarsh Ramamurthy and Amod Kumar Pandey creates photomosaics from Flickr photos in PHP -
  • HCards++ is a hack to create online business cards with validation of user’s identities.
  • Find me a teacher! is an app to connect teachers and students and do all the payments with the PayPal API. The danger is that will sooner or later be used for prostitution I guess.
  • Bird on a mission by Prashanth R, Sumanth J, Tabrez Pasha and Umesh Rao Nis a simple way to alert people of problems in certain locations by sending a Tweet.
  • The five minute mentor by Antano Solar John, Niranjan Prithviraj and Ravishankar is an impressive neurolinguistic analysis tool for texts
  • HackerBox by Saurabh Narula, Akash Mohapatra and Abhinav Mehta is a competitor for our Hack Tracker written in Flex allowing hackers to upload and showcase their hacks.
  • Flickr Commerce by Lakshman, Ashok, Shabda and Javed is an app to take your Flickr photos and offer them for sale using the Paypal merchant API. This is such an obvious thing Flickr lacks and it is very professionally done.
  • SetFlickrLocation by Charul Modi and Janak Chandarana is an interface that adds location information to Flickr photos when the user hasn’t geolocated it by analysing the title and description. This would be much cooler as a GreaseMonkey script – I might have a go at that.
  • EZCraig by Amit Agarwal and Prateek Agarwal scrapes Craigslist and offers it as interconnected dropdowns instead of clicks and reloads.
  • Unlock your code by Saurabh Narula, Akash Mohapatra and Abhinav Mehtais a Downloadable OSX application that allows you to take screenshots from code and annotate it. It then creates a PDF from the annotated code for documentation purposes.
  • The Path Finder by Sreenidhi, Richie, Ullas
    and Suhasis a very clever way to get walking directions from maps when you don’t have a GPS enabled phone. You can send an SMS to a service and it does the map lookup for you. Clever.
  • Prettylicious by Manish Agravat, Baljeetsingh Sucharia and Ramjee Ganti is a more beautiful interface for delicious that adds content from Google to links when there is no description. Also check the domain – how win is that?
  • Trialtool by Parashuram, Babu Srithar, Suryanarayan and Santosh SRis a JSFiddle style sandbox for trying out JavaScript live in the browser. For this hack, the makers imported all the YUI examples. You can see it in action on GitHub
  • SearchItIn by Vivek Rp.
    and Shyam S. is an interface for people who are not savvy of the options in Yahoo Search to search the content of files like XLS, DOC and FLV. The thing that annoys me with the hack is that it only shows Yahoo in a lightbox on submission. If you wanted to create a new experience show the results as Word icons with the description next to them instead. Using Ajax and BOSS this can be easily done.
  • HTML5 gets Flickry are some nice Canvas and HTML 5 demos using Flickr output and Yahoo image search.

If copying is the best kind of flattery then I must be very flattered: Awesome Image Search is an image search clone of GooHooBi and Mobile Hack 420 was re-publishing the FIFA 2010 hack with mobile phone and price data. Nothing to it – I told people to use what works. :)

You can see more at the list of all hacks. If you read through it, you will see that there were a few patterns emerging: a lot of hacks dealt with traffic, the elections in India, language translation and transliteration and speech recognition and synthesis. The latter of course is triggered by the Windows 7 APIs for these tasks and sadly enough all the hacks showed that the success rate of these technologies is still abysmal. It was also pretty funny to see a lot of presenters speaking into their laptop cameras – most laptops actually have the microphone next to the keyboard and not on top of the screen :)

There were a few hacks that tried to simplify the YQL language by mapping it to simpler key:value pairs or a “natural language interface”. This is such a classical developer thing to do. The sad truth is that all of these systems will soon run into restrictions or have to become more complex again.

All in all I am a bit beat now, so maybe more later – I just wanted to get this list out to you.