On aforementioned systems you can simply go to the terminal and do your first cURL thing, load a web site and see the source. To do this, simply enter
And hit enter – you will get the source of icant.co.uk (that is the rendered source, like a browser would get it – not the PHP source code of course):
If you want the code in a file you can add a > filename.html at the end:
curl “http://icant.co.uk” > myicantcouk.html
( The speed will vary of course – this is the Yahoo UK pipe :) )
That is basically what cURL does – it allows you to do any HTTP request from the command line. This includes simple things like loading a document, but also allows for clever stuff like submitting forms, setting cookies, authenticating over HTTP, uploading files, faking the referer and user agent set the content type and following redirects. In short, anything you can do with a browser.
Now, where this becomes really cool is when you use it inside another language that you use to build web sites. PHP is my weapon of choice for a few reasons:
It comes with almost every web hosting package
The latter is also where the problem is. As a lot of people write terribly shoddy PHP the web is full of insecure web sites. This is why a lot of hosters disallow some of the useful things PHP comes with. For example you can load and display a file from the web with readfile():
You will find, however, that a lot of file hosters will not allow you to read files from other servers with readfile(), or fopen() or include(). Mine for example:
And this is where cURL comes in:
// define the URL to load
$url = ‘http://project64.c64.org/misc/assembler.txt’;
// start cURL
$ch = curl_init();
// tell cURL what the URL is
curl_setopt($ch, CURLOPT_URL, $url);
// tell cURL that you want the data back from that URL
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
// run cURL
$output = curl_exec($ch);
// end the cURL call (this also cleans up memory so it is
// display the output
As you can see the options is where things get interesting and the ones you can set are legion.
So, instead of just including or loading a file, you can now alter the output in any way you want. Say you want for example to get some Twitter stuff without using the API. This will get the profile badge from my Twitter homepage:
Why is all that necessary? I can do that with jQuery and Ajax!
I’m right now in Atlanta, Georgia and Georgia Tech for University Hack Day. Yesterday night I kicked off the one week event as my colleagues from California were delayed because of the snow situation in the US.
The presentation covers the history of hack in Yahoo, what makes a good and interesting hack and goes into explaining some of the technologies that people can use.