So here goes – this is the PHP source of icant.co.uk (without the HTML which is more or less done with YUI grids builder
// get all the feeds to grab the data
$feeds = array(
‘http://feeds2.feedburner.com/wait-till-i/gwZf’,
‘http://feeds.delicious.com/v2/rss/codepo8/myvideos?count=15’,
‘http://feeds.delicious.com/v2/rss/codepo8/sandbox’,
‘http://feeds.delicious.com/v2/rss/codepo8/icantarticles’,
‘http://www.slideshare.net/rss/user/cheilmann’
);
// assemble the YQL statement
$yql = ‘select meta.views,content.thumbnail,content.description,title,’.
‘link,description from rss where url in ‘;
$yql .= “(‘” . join($feeds,”’,’”) . “’)”;
// get the feeds and populate the data to echo out in the HTML
$feeds = renderFeeds($url);
$blog = $feeds[‘blog’];
$videos = $feeds[‘videos’];
$articles = $feeds[‘articles’];
$presentations = $feeds[‘slides’];
// this function loads all the feeds and turns them into HTML
function renderFeeds($url){
// grab the content from YQL via cURL
$c = getStuff($url);
// as the content comes back as JSON, turn it into PHP objects
$x = json_decode($c);
// reset counter for videos and presentations
$count = 0;
$vidcount = 0;
// start new array to return
$out = array();
// loop over YQL results, if they exist.
if($x->query->results->item){
foreach($x->query->results->item as $i){
// if the link comes from the blog, add to the blog HTML
if(strstr($i->link,’wait-till-i’)){
$out[‘blog’] .= ‘
// grab the books from my blog
$yql = ‘select * from html where url=’.
‘”http://wait-till-i.com/books/”’.
’ and xpath=”//div[@class=’entry’]”’;
$books = renderHTML($root.urlencode($yql).’&format=xml&diagnostics=false’);
// this is a quick and dirty solution for the HTML output
function renderHTML($url){
// pull the information from YQL
$c = getStuff($url);
// check that something came back
if(strstr($c,’<’)){
// remove all the XML parts
$c = preg_replace(“/.*|.*/”,’‘,$c);
$c = preg_replace(“/
” encoding=”UTF-8”?>/”,’‘,$c);
// remove all comments
$c = preg_replace(“//”,’‘,$c);
}
// send it back
return $c;
}
// a simple cURL function to get information
function getstuff($url){
$curl_handle = curl_init();
curl_setopt($curl_handle, CURLOPT_URL, $url);
curl_setopt($curl_handle, CURLOPT_CONNECTTIMEOUT, 2);
curl_setopt($curl_handle, CURLOPT_RETURNTRANSFER, 1);
$buffer = curl_exec($curl_handle);
curl_close($curl_handle);
if (empty($buffer)){
return ‘Error retrieving data, please try later.’;
} else {
return $buffer;
}
I am right now very excited about the new Placemaker beta – a location extraction web service released at Where2.0. Using Placemaker you can find all the geographical locations in a feed or a text or a web url and you get them back as an array of places.
As a demo I took the Yahoo News feed and ran it through Placemaker. The resulting places are plotted on a map and the map moves from location to location when you hover over the news items.
Getting the data from the data feed and running it through placemaker is very straight forward. I explained the basic principle in this blog post on the Yahoo Developer Network blog. The only thing to think about is to define the input and output types correctly:
You’ll also notice that the elements are namespaced and the names of the locations in CDATA blocks, both things I hate with a passion. Not because they don’t make sense, but because simplexml can be drag to make understand them.
What I wanted to do with this data was twofold: create a JSON array of geo locations to plot on a map and a display of the news content. This is the PHP that does that:
$places = simplexml_load_string($results, ‘SimpleXMLElement’, LIBXML_NOCDATA);
// if there are elements found
if($places->channel->item){
// start a JSON array
$output .= ‘[‘;
// start the HTML output
$html = ‘
‘;
// set the counter – this will be needed to link news
// items and map markers
$count = 0;
// loop over RSS items
foreach($places->channel->item as $p){
// set inner counter (as there are more locations per news item)
$innercount = 0;
// start the HTML list item and give it an ID with the counter
// value
$html .= ‘
// all child elements with the defined namespace
$locs = $p->children(‘http://wherein.yahooapis.com/v1/cle’);
// check that there is a location sub-element in this item
if($locs->contentlocation){
// if there is one, add a class to the LI
$html .= ’ class=”haslocation”’;
// start an array for displaying of the locations under the
// news items
$dlocs = array();
// loop over all the places found for this item
foreach($locs->contentlocation->place as $pl){
// append a new JS object with the location data
// and a unique ID to the locations array
$locations[] = ‘{name.’”,”title” title=”“>name“:”’.
preg_replace(‘/n+/’,’‘,addslashes($p->title)).
‘”,latitude” title=”“>lat.
‘”,longitude.’”,”id”:”m” title=”“>lon‘.
$count.’x’.$innercount.’”}’;
// add the location name to the display locations array
$dlocs[] = $pl->name;
// increase the inner count to ensure that every marker has
// a unique ID
$innercount++;
}
The JavaScript to show the map is pretty straight forward and more or less the demo example of the maps API:
// will be called with the array assembled in PHP
function placeonmap(o){
// if there are locations
if(o.length > 0){
// create a new geopoints array to hold all locations
// this is needed to determine the original zoom
// level of the map
var geopoints = [];
// add map with controls
var map = new YMap(document.getElementById(‘map’));
map.addZoomLong();
map.addPanControl();
// loop over locations
for(var i=0;i
// define a new geopoint and store it in the array
var point = new YGeoPoint(o[i].lat,o[i].lon);
geopoints.push(point);
// create a new marker and give it the unique
// id defined in the PHP. Pop up the title of
// the news item and the name of the location when the
// user hovers over the marker
var newMarker = new YMarker(point,o[i].id);
newMarker.addAutoExpand(o[i].title + ‘(‘+o[i].name+’)’);
map.addOverlay(newMarker);
}
// define best zoom level and show map
var zac = map.getBestZoomAndCenter(geopoints);
map.drawZoomAndCenter(zac.YGeoPoint,zac.zoomLevel);
}
// add a mouseover handler to the list of results YAHOO.util.Event.on(‘news’,’mouseover’,function(e){
// remove the “news” text of the ID of the current target
// as we named the list items news0 to news19
var id = YAHOO.util.Event.getTarget(e).id.replace(‘news’,’‘);
// if there is still something left we have one of the news
// items
if(id!==’‘){
// get the first marker with the ID we defined in the loop.
var marker = map.getMarkerObject(‘m’+id+’x0’);
// if there is one, pan the map there and show the message
// attached to it.
if(marker){
map.panToLatLon(marker.YGeoPoint);
marker.openAutoExpand();
}
}
});
}
// call placeonmap with the JSON array
placeonmap();
That’s pretty much it. I am sure it can be refined, but it is amazing how easy it is to get geo information into any text with Placemaker.
I am right now teaching a four day class of DOM and Ajax in Sunnyvale, California and also do some tech editing for Scriptin with JavaScript and Ajax by Charles Wyke-Smith and I find one thing that is pretty worrying: easy examples of web development practices are dangerous to show these days.
I’m talking about practices that make it easy to get quick results and give readers and attendees “I am getting this – this is easy” fuzzy warm feelings.
One very obvious example is form validation and re-rendering of a form using PHP_SELF and displaying user data using $_GET or $_POST. Unfiltered they are a free invitation for any XSS attack and will turn your server into a spam-hub or bot-net drone. Explaining countermeasures of XSS normally is out of scope for an example that only shows how a form would work that you enhance progressively.
The same applies to simply outdated ideas like onevent handlers. It is easy to show an example that uses a few onclick handlers, but explaining event handling really well takes a bit of time. Again, this is something that really does not fit in the scope of a DOM course.
I do however think that it is important to get it in there, as there is no such thing as knowing one technology in the web development stack and being able to use it. There’s a lot of overlap with other areas and in order to be a good developer and play well with others you need to be aware of your effects and areas of overlap with your colleagues’ skill-sets.
The other extreme I find myself doing is being too over-cautious. I went through the tough times of the first browser wars and got a deep-rooted mistrust towards anything some browser tells me is OK to do and use. However, I get the feeling that it doesn’t really matter any more if Internet Explorer has a problem with name vs. ID or whatever other shenanigans we have to be aware of when we build things from scratch.
I do get the distinct feeling that not building on top of a good client-side library is simply a waste of time these days. Libraries allow us to write code, not to work around bugs and wonder what other safety measure we have to put in.
That’s why I started asking people in my courses to use Firefox with Firebug and use a good text editor to code along. Today I managed to breeze through how to write HTML that is ready for internationalisation and works with assistive technology, over simple DOM access to the document and at the end writing a validation script for a form using generated DOM content. By concentrating on how things are meant to work instead of debugging random issues I managed to get the students to reach far into the matter in a day – even those who never touched JavaScript before.
Maybe it is time to get beginners accustomed to a market that builds on working solutions and benefits from browser abstraction via libraries than teaching developing from total scratch – bad browsers and bad people taking advantage of any technology to gain access or spam us seem to have made this way of working redundant.
Tags: accessibility, bash, curl, hackday, php, security Posted in General | Comments Off on TTMMHTM: Guardian getting enabled by design,interview,open hack day,bash magic,and XSS filters
As you probably know, I am spending a lot of time speaking and mentoring at hack days for Yahoo. I go to open hack days, university hack days and even organized my own hackday revolving around accessibility last year.
One of the main questions I get is about technologies to use. People are happy to find content on the web, but getting it and mixing it with other sources is still a bit of an enigma.
Following I will go through a hack I prepared at the Georgia Tech University hack day. I am using PHP to retrieve information of the web, YQL to filter it to what I need and YUI to do the CSS layout and add extra functionality.
The main ingredient of a good hack – the idea
I give a lot of presentations and every time I do people ask me where I get the pictures I use from. The answer is Flickr and some other resources on the internet. The next question is how much time I spend finding them and that made me think about building a small tool to make this easier for me.
The next thing I could have done is deep-dive into the Flick API to get photos that I am allowed to use. Instead I am happy to say that using YQL gives you a wonderful shortcut to do this without brooding over documentation for hours on end.
Using YQL I can get photos from flickr with the right license and easily display them. The YQL statement to search photos with the correct license is the following:
select id from flickr.photos.search(10) where text=’donkey’ and license=4
You can try the flickr YQL query here and you’ll see that the result (once you’ve chosen JSON as the output format) is a JSON object with photo results:
The problem with this is that the user name is not provided anywhere, just their Flickr ID. As I wanted to get the user name, too, I needed to nest a YQL query for that:
select farm,id,secret,server,owner.username,owner.nsid from flickr.photos.info where photo_id in (select id from flickr.photos.search(10) where text='donkey' and license=4)
The next step was getting the data from the other resources I am normally tapping into: Fail blog and I can has cheezburger. As neither of them have an API I need to scrape the HTML data of the page. Luckily this is also possible with YQL, all you need to do is select the data from html and give it an XPATH. I found the XPATH by analysing the page source in Firebug:
This gave me the following YQL statement to get images from both blogs. You can list several sources as an array inside the in() statement:
select src from html where url in (‘http://icanhascheezburger.com/?s=donkey’,’http://failblog.org/?s=donkey’) and xpath=”//div[@class=’entry’]/div/div/p/img”
The result of this query is again a JSON object with the src values of photos matching this search:
The next thing I wanted to do was writing a small script to get the data and give it back to me as HTML. I could have used the JSON output in JavaScript directly but wanted to be independent of scripting. The script (or API if you will) takes a search term, filters it and executes both of the YQL statements above before returning a list of HTML items with photos in them. You can try it out for yourself: search for the term donkey or search for the term donkey and give it back as a JavaScript call
I use cURL to get the data as my server has external pulling of data via PHP disabled. This should work for most servers, actually.
I call cURL to retrieve the data from the flickr yql query, do a json_decode and loop over the results. Notice the rather annoying way of having to assemble the flickr url and image source. I found this by clicking around flickr and checking the src attribute of images rendered on the page. The images with the “ico” class should tell me where the photo was from.
Retrieving the blog data works the same way, all I had to do extra was check for which blog the resulting image came from.
$out.= ‘‘;
if($_GET[‘js’]=’yes’){
$out.= ‘’})’;
}
echo $out;
I close the list and – if JavaScript was desired – the JavaScript object and function call.
} else {
echo ($_GET[‘js’]!==’yes’) ?
‘
Invalid search term.
’ :
‘seed({html:”Invalid search Term!”})’;
}
}
?>
If there was an invalid term entered I return an error message.
Setting up the display
Next I went to the YUI grids builder and created a shell for my hack. Using the generated code, I added a form, my yql api, an extra stylesheet for some colouring and two IDs for easy access for my JavaScript:
HTML PUBLIC “-//W3C//DTD HTML 4.01//EN”
“http://www.w3.org/TR/html4/strict.dtd”>
Slide Fodder – find CC licensed photos and funpics for your slides
Slide Fodder
Slide Fodder by Christian Heilmann, hacked live at Georgia Tech University Hack day using YUI and YQL.
The last thing I wanted to add was a “basket” functionality which would allow me to do several searches and then copy and paste all the photos in one go once I am happy with the result. For this I’d either have to do a persistent storage somewhere (DB or cookies) or use JavaScript. I opted for the latter.
The JavaScript uses YUI and is no rocket science whatsoever:
function seed(o){ YAHOO.util.Dom.get(‘content’).innerHTML = o.html;
}
YAHOO.util.Event.on(‘f’,’submit’,function(e){
var s = document.createElement(‘script’);
s.src = ‘yql.php?js=yes&s=’+ YAHOO.util.Dom.get(’s’).value;
document.getElementsByTagName(‘head’)[0].appendChild(s); YAHOO.util.Dom.get(‘content’).innerHTML = ‘‘;
YAHOO.util.Event.preventDefault(e);
});
YAHOO.util.Event.on(‘content’,’click’,function(e){
var t = YAHOO.util.Event.getTarget(e);
if(t.nodeName.toLowerCase()===’img’){
var str = ‘
YAHOO.util.Event.preventDefault(e);
}); YAHOO.util.Event.on(‘basket’,’click’,function(e){
var t = YAHOO.util.Event.getTarget(e);
if(t.nodeName.toLowerCase()===’a’){
t.parentNode.parentNode.removeChild(t.parentNode);
}
YAHOO.util.Event.preventDefault(e);
});
Again, let’s check it bit by bit:
function seed(o){ YAHOO.util.Dom.get(‘content’).innerHTML = o.html;
}
This is the method called by the “API” when JavaScript was desired as the output format. All it does is change the HTML content of the DIV with the id “content” to the one returned by the “API”.
YAHOO.util.Event.on(‘f’,’submit’,function(e){
var s = document.createElement(‘script’);
s.src = ‘yql.php?js=yes&s=’+ YAHOO.util.Dom.get(’s’).value;
document.getElementsByTagName(‘head’)[0].appendChild(s); YAHOO.util.Dom.get(‘content’).innerHTML = ‘
‘src=”http://tweeteffect.com/ajax-loader.gif”’+
‘style=”display:block;margin:2em auto”>‘; YAHOO.util.Event.preventDefault(e);
});
When the form (the element with th ID “f”) is submitted, I create a new script element,give it the right src attribute pointing to the API and getting the search term and append it to the head of the document. I add a loading image to the content section and stop the browser from submitting the form.
YAHOO.util.Event.on(‘content’,’click’,function(e){
var t = YAHOO.util.Event.getTarget(e);
if(t.nodeName.toLowerCase()===’img’){
var str = ‘
I am using Event Delegation to check when a user has clicked on an image in the content section and create a new DIV with the image to add to the basket. When the image was from flickr (I am checking the src attribute) I also add the url of the image source and the user name to use in my slides later on. I add an “x” link to remove the image from the basket and again stop the browser from doing its default behaviour.
YAHOO.util.Event.on(‘basket’,’click’,function(e){
var t = YAHOO.util.Event.getTarget(e);
if(t.nodeName.toLowerCase()===’a’){
t.parentNode.parentNode.removeChild(t.parentNode);
}
YAHOO.util.Event.preventDefault(e);
});
In the basket I remove the DIV when the user clicks on the “x” link.
That’s it
This concludes the hack. It works, it helps me get photo material faster and it took me about half an hour to build all in all. Yes, it could be improved in terms of accessibility, but this is enough for me and my idea was to show how to quickly use YQL and YUI with a few lines of PHP to deliver something that does a job :)