Coupling YUI and YQL to Build Dynamic Widgets

By YUI TeamJune 17th, 2009

Yahoo! Developer Network evangelist Jonathan LeBlancAbout the Author: Jonathan LeBlanc (twitter: jcleblanc) works with the partner integration group in the Yahoo! Developer Network as a senior software engineer / technology evangelist. Focusing on partner relationships and training, as well as external developer integration, Jonathan works with and promotes emerging technologies to aid in the adoption and utilization of new social development techniques. As a software engineer, Jonathan works extensively with social interaction development on the web, developing new methods for linking social networks to drive the ideal of an open web. Prior to Yahoo!, Jonathan worked within both the media production and fantasy product development teams at CBSSports.com, where he developed on products such as the Emmy award-winning march madness on demand video player. Born and raised in Canada, Jonathan graduated from Algonquin College of Applied Arts and Technology with a degree in computer science and currently lives in Dublin, CA with his wife.

YUI has a wonderfully rich list of data manipulation and UI utilities in its library, but coming up with equally rich and interesting data is generally your responsibility as an implementer. By the same token, YQL has rich data fetching and interaction abilities, but it lacks YUI’s visualization tools. Taking the visualization and control features of YUI and coupling them with the raw data pipe of YQL, we can build incredibly rich, interactive widgets without the need to create any of the visualizations or data storage mechanisms ourselves.

I’ll give you a tour of two widgets built with YUI and YQL in this post. One displays publicly available content and the other builds upon that original widget with a server-side PHP component to display private user data by authenticating with OAuth.

Accessing Public Data

Using the YUI Get Utility, we are able to make requests to the public YQL query URL without having the pain of dealing with the same-origin policy issue that would normally prevent us from making XHR-based requests to a server that is not on the same domain as the originating request. Below is a small code sample which showcases how you would call YQL using the YUI Get Utility:

<script type="text/javascript">
var getYQLData = function(query){
    //prepare the URL for the YUI GET request:
    var yqlPublicQueryURL = "http://query.yahooapis.com/v1/yql?";
    var sURL = yqlPublicQueryURL + "q=" + query
        + "&format=json&callback=yqlWidget.getYQLDataCallback";
        
    //make GET request to YQL with provided query
    var transactionObj = YAHOO.util.Get.script(sURL, {
        onSuccess : onYQLReqSuccess,
        onFailure : onYQLReqFailure,
        scope     : this
    });

    return transactionObj;
}
</script>

The query URL (sURL) is composed of several different parts to make up the request. The yqlPublicQueryURL variable contains the public URI for making requests to YQL. There are also three query parameters that we are passing along to this URI. The q parameter is the YQL query that we use in our request (e.g. select * from flickr.photos.search where text="YDN"), format is the format we want returned (JSON or XML) from the request, and if we want to wrap the JSON returned data in a function (a la JSONP) we can define a callback function using the callback parameter. When this function runs, the Get Utility will make a request to query.yahooapis.com with the specified query. On success, a debug message will be displayed in the onSuccess callback for the Get Utility then the callback defined in the YQL query will be called to parse the returned JSON results.

Setting up this widget on your own site or blog is as simple as downloading the code from the js-yql-display folder on github at http://github.com/jonleblanc/yql-utilities/tree/master and instantiating the widget like this:

<!-- widget file include -->
<script type="text/javascript" src="yql_js_widget.js"></script>

<!-- widget styles -->
<style>
div.imgCnt{ border: 1px solid rgb(96, 96, 96); 
    margin: 5px 5px 5px 0pt; float: left;
	background-color: rgb(241, 241, 241); width:100px;
	height:140px; }
div.imgCnt img{ border:0; margin:5px; }
div.imgCnt div.imgTitle{ padding: 5px; font-size: 11px;
    text-align:center; }
</style>

<script type="text/javascript">
var config = {'debug' : true};
var format = '<div class="imgCnt" align="center">'
    + '<a href="http://www.flickr.com/photos/{owner}/{id}/"'
    + 'target="_blank"><img src="http://farm3.static.'
    + 'flickr.com/{server}/{id}_{secret}.jpg?v=0"'
    + 'width="80" height="80" /></a>'
    + '<div class="imgTitle">'
    + '<a href="http://www.flickr.com/photos/{owner}/'
    + '{id}/">{title}</a>'
    + '</div></div>';
var yqlQuery =
    'select * from flickr.photos.search where text="YDN"';
var insertEl = 'widgetContainer';
yqlWidget.push(yqlQuery, config, format, insertEl);
</script>

Any number of public YQL queries can be made using the above example — all documentation for how the configuration script works can be found on github. If we run the above code we can easily display our most recent flickr photos:

flickr photo display

So, why all the fuss over using YQL and YUI together like this? Well, for me these utilities bring me quite close to a traditional MVC (Model View Controller) type of design pattern. YQL is a wonderful data aggregation and manipulation tool, but at the end of the day it’s just data. We insert the controller functionality using YUI utilities such as the Get Utility or Connection Manager, then we can build our presentation layer on top of that using widgets like DataTable and Charts.

private update stream

Accessing Private User Data

Let’s explore the pairing of YUI and YQL a little further by taking the foundation built in the public JavaScript query widget and attaching a server-side component to authenticate and store an oAuth session. Using the Y!OS PHP SDK, we are able to extend the JavaScript widget to display the update stream and personal badge details of the user who originally authenticated the widget. Since we’re using the PHP SDK on the same domain as our widget to make our authenticated calls to YQL, we no longer need to worry about the same-origin policy issues. This means that we can exchange the use of the YUI Get Utility with the YUI Connection Manager. The benefit to using the Connection Manager here is that we can use a standard ajax request and the event handlers of the utility instead of a callback within the YQL query. Our new request function would look something like this:

<script type="text/javascript">
var getYQLData = function(query){
    //prepare URL & post data for YUI Connection Manager POST:
    var sURL = "private_data_fetch.php";
    var postData = "q=" + query;

    //define Connection Manager event callbacks
    var callback = {
        success:parseYQLResults,
        failure:onYQLReqFailure
    };
		
    //make POST request to YQL with provided query
    var transactionObj = YAHOO.util.Connect.asyncRequest('POST',
	    sURL, callback, postData);
		
    return transactionObj;
}
</script>

The Connection Manager is used to make AJAX requests to the PHP SDK (whose references are housed in private_data_fetch.php within the above code) with the query as a POST parameter. The SDK in turn makes an authenticated request for the user’s personal data and dumps out a JSON string as the return of the request. Then all you need to do is parse the JSON string using the YUI JSON Utility; after you’ve called YAHOO.util.JSON.parse() on the transaction results, your data can be treated in the same manner as the data in the first example.

Since YQL generates results when the request is called, there is no need to house this data for yourself. YQL can pull data from any available source so you can build widgets such as these to display dynamic results with each refresh.

Both of the widgets showcased in this post are freely available for download and contribution on github at
http://github.com/jonleblanc/yql-utilities/tree/master. These widgets are:

  • js-yql-display: JavaScript widget to display public YQL data
  • php-yql-display: JavaScript / PHP widget to display private YQL data

14 Comments

  1. [...] Not sure what to do? Well, they have some examples using earthquake data. Or you could check out their guidance on building widgets. [...]

  2. You made me think about developing a subclass of DataSource to handle YQL requests. You can see it at work in this example together with the DataTable widget.

    Thanks for the idea.

  3. That’s a great example – you beat me to it :). I was taking a look into using YQL as a data source for different YUI components and this is a perfect example of that. The additional benefit is that you don’t have to maintain the data source in a database.

    - Jon

  4. Thanks. Anyway, it shows a minor weakness in the response from YQL: it does not provide any meta-data about the results.

    I only guess that the first record in query.results is a good template for all the rest, but that might not be true, there is really no reason to assume it to be so.

    Furthermore, it doesn’t give datatype information, I should be able to parse dates, numbers and booleans into their native JavaScript types, but I can’t unless the developer tells me, which is a pity because the information is probably there anyway. I could ask for a DESC, but doing two requests would duplicate the latency time, a pity, really.

  5. Jon,

    Does YUI plan to support local-functions/ functions created using closures as jsonp callbacks in the future ?

    The only way I see jsonp supported is through global functions.

    Vipul

  6. Perhaps not necessary to mention, but YUI’s JSON util lives under YAHOO.lang, not YAHOO.util.

    Common mistake. I’ve made it myself a few times.

  7. Hi Luke, you’re absolutely right – it looks like I copy / pasted too quickly when writing that part of the post. I verified that in the checked in code within github it is correctly referenced under lang so the widget won’t need an update and should be usable.

    Thanks for pointing that out.

    - Jon

  8. Vipul – to be honest I’m not certain, the YUI team would probably be better to answer that than I.

  9. [...] Jonathan LeBlanc on YUI 3: YUIBlog contributor Jonathan LeBlanc, a technical evangelist on the Yahoo Developer Network team, was in South Carolina [...]

  10. [...] For a more in depth explanation of what was built out with this tool take a look at my post on the YUI blog at http://www.yuiblog.com/blog/2009/06/17/yui-and-yql/ [...]

  11. Hi Jon,

    I did some search but failed to locate any python based lib/code/example to access private data. Do you know if such “thing” exist?

    Thanks.

  12. Antonio,

    There is a Yahoo! Open Strategy Python SDK available which should help you access private Yahoo! user profile data. That SDK is located at http://github.com/yahoo/yos-social-python/tree

    - Jon

  13. Hello,
    I’m tested “Accessing Public Data” and have a question. When generate a query and have reterned data all is ok, but what we do if no data to return or the “Public Server” is not work at the moment. And if we have a some “push” and one “render” function to make access to some difrent servers, if middle push is not return data, the all next “push” is stop work! How to prevent this. Thanks!

  14. [...] YQLDataSource for YUI 2: “Thanks to Jonathan LeBlanc’s article on the YUI Blog I thought that it would be cool to have a YQLDataSource to make it easier to use YQL data in YUI 2 [...]