Dynamic, asynchronous websites made simple with JSONP

I remember the first time I discovered the potential of XMLHttpRequest. It was mind-boggling. I was in heaven with all the stuff I could do. I played like a kid in a candy store until, suddenly, cruelly, I ran headlong into two major problems with XMLHttpRequest:

1. You couldn't get info from anything but the originating server
2. XML stinks.

And then one day, I stumbled upon JSONP. I still remember the day. I had been working with the Google AJAX APIs, and I wanted to know how they got around the two inherent limitations in what I understood to be AJAX. So I dug into their obfuscated code and watched their network traffic for hints. And there it was: they didn't use XMLHttpRequest or XML.

Instead, they used a method called JSONP. (They just called it AJAX because that was the major buzz word at the time.)

JSONP is really a very simple, elegant, and flexible way to communicate dynamic data between client and server. And it's fast. Because it uses JSON rather than XML, the data transfers are smaller and the time it takes for the Javascript engine to parse the data is dramatically shorter. So in comparison to traditional "AJAX," JSONP screams.

Why is it, then, that most people haven't heard of JSONP? And why is it that, whenever I suggest it to people, they stare at me with this blank expression on their face as though I was speaking in some alien language?

Well, the only reason I can come up with is that "AJAX" is the buzzword. It just sounds a whole lot more cool (and clean) than JSONP. And so, since most people have heard only of AJAX, it actually is as though I'm speaking in some alien language when I suggest JSONP.

So let's fix both of these problems in one shot. From here on out, we'll call JSONP the "Super Spiffy Way To Communicate Data Between Your Client And Your Server" (SSWTCDBYCAYS).

Okay, let's not do that.

Seriously, though. Let's see if we can't figure out this mysterious thing called JSONP quickly and easily, once and for all.

In any dynamic web application, there are two sides of the equation: client and server. The client side is what's going on in the user's web browser. The server side is what's going on in - you guessed it - my web server. For the sake of this discussion, we're going to start on the client side, with the Javascript, since that's where the bulk of the hocus-pocus happens.

On the client side, everyone knows that an HTML page is comprised of a series of elements formed into some semblance of a hierarchical structure. (It doesn't matter if you don't get that. Just pretend and keep reading.) Some of these elements can instruct the browser to fetch additional resources from the server. For instance, <img> tags tell the browser to ask the server for an image file; <link> tags can ask the server for a CSS stylesheet, and <script> tags can order the browser to fetch a Javascript file. That last one right there is the key to JSONP.

You see, the secret to the black magic of JSONP is that you can use Javascript to dynamically construct additional <script> elements to append to your page. And the browser will obediently go and fetch those scripts from the server whenever you do.

So all we have to do is write a little Javascript function that uses DOM methods to build and append a new script element to the page. Something like this:

function do_json(url){
  var el = document.createElement('script');
  el.src = url;
  el.type = 'text/javascript'; // gotta be PC here!
  document.getElementsByTagName('head')[0].appendChild(el);
}

Now, any time you want to fetch information from the server, all you have to do is call do_json(<url_of_the_js_file_i_want>); .

Of course, that doesn't help us a whole lot if the server returns a blank stare. So for the next step, we have to turn to the server, where we write a script or program that will get data from any source you can dream up (e.g., database, XML, thin air, subspace transmission from Starfleet Command) and return it in a JSON format.

JSON, by the way, stands for JavaScript Object Notation. It's essentially a string which represents a Javascript object. Something like this:

{
  "name" : "Jeremy",
  "rank" : "Generally Cool Guy",
  "serialNo" : "1234567890"
}

That is a very simple JSON string representing an object with three properties: name, rank, and serialNo. So the bulk of this step in the JSONP process is getting my server to return data in that sort of format. Of course, exactly how you will do that will depend on your backend and language of choice. So I will let you Google how you might do that.

Once you figure out how to get your server to return JSON, we still have a problem. Simply returning JSON to the browser is like taking a phone message without writing down who it's for. Now, you could set the message on the corner of your desk in the hopes that the right person will happen by, notice it there, and grab it. But the chances of that happening are pretty slim. So our server has to be a little more specific. Namely, it has to encapsulate the JSON string in a Javascript call. So instead of the string we had above, we end up with something like this:

my_callback({
  "name" : "Jeremy",
  "rank" : "Generally Cool Guy",
  "serialNo" : "1234567890"
})

Now, that should look familiar. It's plain Javascript. You're calling the function my_callback and passing it the Javascript object you created above. Simple enough.

The problem now is that my_callback isn't home on the client-side just yet. So let's add it to the client-side Javascript code that we had before.

function my_callback(response){
  // do something!
}

All of a sudden, my_callback is eagerly awaiting its message, and as soon as it gets it, it can take off running.

Now, this is just the tip of the iceberg. Because the actual body of the server's response could be an array, an object, a number of arrays or objects, or any number of other things. And you could use it to rewrite your page, update particular elements, and much, much more. The sky is the limit! And because you're not using the SOP-restricted XMLHttpRequest, you can request data from any server on the web, making it possible to create services on one server which can be utilized from any number of sites. And it's a whole lot faster, to boot!

Once you grasp the concept of JSONP, your mind will just start seeing possibilities for uses. Search engines. Maps applications. Updating content. The list is virtually endless.

Well, folks. That's all the time we have for today. We hope you've enjoyed the broadcast! Please tune in next time as Jeremy writes about taming your very own water buffalo!