Blog: Archive for March, 2009

Building Sideline: Lessons in YUI + Adobe AIR

About the Author: Chad Auld is a Front-end Engineer working with the Yahoo! Buzz Marketing team. A long-time open-source contributor, he recently helped start the MiaCMS project, a next-generation fork of Mambo built using YUI. In this article, he walks us through the process of developing a desktop application with YUI on the Adobe Air platform.

Screenshot of Sideline

Ever wonder what people are saying right now about your company, brand, service, product, etc? Sideline, inspired by a recent internal hack project at Yahoo!, goes beyond the standard customer survey process to let you listen in real-time to people talking about your products and then use that feedback to enhance your service or help users with their problems.

Briefly stated, the goals of our project were to

  • Create a desktop application that allows for the creation, grouping, and auto-execution of advanced search queries against Twitter
  • Leverage existing skill-sets and tools
  • Target the Windows, Mac OS X, and Linux operating systems and minimize the amount of platform specific code that must be written
  • Open source the code so that others can learn from, contribute to, and/or extend the product as they see fit

Our team of front-end engineers are experts in JavaScript, CSS, HTML, and PHP but didn’t have a great deal of experience developing desktop applications. So the question became, how to maximize our existing skill-sets for desktop development? The answer for us was to utilize the Adobe AIR platform, which “lets developers use proven web technologies to build rich Internet applications that run outside the browser on multiple operating systems”. Since AIR supports HTML/JavaScript development (in addition to Flex and Flash), we could build our application on traditional web technologies, on top of YUI, and have it run on the three main desktop operating systems.

YUI Grids in AIR

Sideline contains an extensive implementation of the YUI Library. It should hopefully serve as a great example for other developers interested in experimenting with YUI and Adobe AIR. The application’s layout is constructed using YUI Grids and even makes use of the recently added ARIA Landmark Roles. Grids worked very well in the AIR environment and made redesigns that occurred mid-development easy to implement with minimal code changes. Just like in the standard browser environment, YUI Grids can serve as a great foundation for an AIR application even if the developer decides against using the rest of the JavaScript library and opted for another framework instead.

YUI Components in AIR

In addition to Grids, Sideline also utilizes the Dom, Event, Drag and Drop, JSON, Selector, Container, Button, Menu, Slider, and TabView components. I am happy to report that all the YUI components performed extremely well in the AIR environment and required no modifications. Sideline does implement a fairly customized design and thus some customized skinning of the YUI components was required, but no core modifications. Most AIR applications tend to have a rich desktop application feel to them. For this level of customization, the YUI skinning article is a great reference to get started.

Beyond the Browser

A major enhancement of the Adobe AIR platform over the traditional web environment is access to a local SQLite database and the user’s file system. Local database access is becoming more available in traditional web environments through technology such as Gears and HTML 5 client side storage, but for now these solutions are not ubiquitous. For those interested in AIR development, Sideline has tackled many of the common tasks that a typical AIR application might require, e.g., fetching external data, handling application updates, interacting with the local database, working with the local filesystem, launching native browser windows, displaying desktop notifications, etc. It should prove to be a useful reference in that respect.

Tips for AIR Development

  1. Know your environment. AIR uses the WebKit open source browser engine under the hood. Traditional web development is aimed at making an application or site work across as many browsers/operating systems as possible. Which browsers to support typically comes down to a cost versus usage factor. However, coding for a single rendering engine reduces the need to prepare for and test against the slue of potential combinations in the market. That being said, it still makes sense to develop in a cross-browser manner where possible since there may come a time when the application needs to find its way back into a more traditional browser environment. Using a framework like YUI will make that process relatively painless. It is simple to see the browsers and platforms currently supported by YUI via the Graded Browser Support chart. Developers should be fairly safe to take some basic shortcuts when building AIR application (using -webkit-border-radius makes rounded corners a breeze), but use them sparingly and document them so they are easy to spot later.
  2. During the development of a complex application in any environment a solid set of debugging tools is a must-have. Adobe provides some useful tools for debugging AIR out of the box. Developers should investigate the AIR Debug Launcher (ADL), the HTML Introspector, and the HTML Source Viewer. In addition to the bundled tools, Aptana Studio with its Adobe AIR Plugin proved to be an indispensable asset. The Aptana plugin provides assistance with creation of an AIR project, importing of common JavaScript frameworks, debugging, packaging/exporting, and digitally signing the application.
  3. Don’t forget the performance techniques we’ve learned from the standard browser environment (i.e., optimize your images, minify and combine the application’s CSS and JavaScript files, and for heavy event-based applications like Sideline, take advantage of event delegation techniques). AIR applications run on the desktop and so there is a bit more leniency with performance than in the typical browser environment, but remember just like the browser itself, the AIR container also consumes a chunk of the system’s memory even before the application’s custom code kicks in.

The Road Ahead

The beta version of Sideline can be installed at http://sideline.yahoo.com. The code is open source under the terms of the BSD license and hosted on GitHub. We welcome contributions, feedback, and/or suggestions. Also, in the spirit of keeping things as open as possible and supporting emerging technology we will likely port Sideline to Titanium in the near future. Some initial work has already been done on the port and will continue over the coming weeks. It is also quite possible that Sideline will end up implementing a JavaScript ORM such as JazzRecord to ease database interactions across platforms. If anyone has additional tips for supporting multiple platforms we’d love to hear them.

Now go forth and fork it!

By Chad AuldMarch 31st, 2009

Implementation Focus: DocLanding

DocLanding: Online, on-demand document management

Todd Fishback is the President of DocLanding, a web-based document management solution. Todd joins us on YUIBlog to discuss his team’s choice of YUI utilities and widgets within the DocLanding user interface. You can learn more about DocLanding from its presentation at Fall 2008 Demo Conference.

Tell us a little bit about DocLanding — what are the central problems you solve for your users?

DocLanding is an on-demand document management solution that delivers enterprise class document management functionality for a fraction of the costs of most enterprise solutions. The software can be delivered through our Software as a Service (SaaS) offering or as an in-house system. Our clients are primarily in the financial services and healthcare arenas.

Common issues we solve for our customers include providing a web-based centralized repository for distributed workforces, on-demand web-based scanning for low paper volume offices, and desktop batch-based scanning in high paper volume offices. Other issues we address include secure document sharing and collaboration, document editing/annotations, version control, document commenting, and document watermarking. Our unique approach to separately controlled but linked document repositories allows users to access disparate repositories with one common login.

What were the particular user interface challenges presented by your product’s design?

DocLanding: Document preview UI.

We learned from some of our earlier work that you simply cannot underestimate the importance of user-friendly design. Creating a website is fairly easy, but creating a true web application that has to meet the needs of businesspeople is real work. Our product attempts to take document management from strictly the domain of the large enterprise and make it available to any small business. Electronic document management at its core is not a simple task. The goal is to organize and control access to massive numbers of files in addition to making them fully searchable. Because of this, the user interface is actually where the majority of our development time has traditionally been spent.

We’ve found that you will save time and money on support issues when you make your site straightforward and easy to use. Part of that is relaxing the specifications needed to run the site. We got ours pared down to just about any modern browser with JavaScript and Flash. The core site design we came up with presented its own challenges with its very specific use of the screen real estate. We found our users were better able to make full use of the application when we ourselves paid attention to colors, iconography and proximity of the controls to their function. We think we’re on the right track because our feedback page has returned more requests for additional features than for help requests.

You chose YUI to help power your site. What led you to that decision?

DocLanding: On-demand document management

The simple answer is consistency and speed. We needed a framework that would enable us to meet the design specifications of our product. More specifically, we had ambitious design goals like maintaining a one screen view and minimizing or eliminating full page postbacks. In addition, we wanted our required elements to look and function identically in as many different browsers as we could manage. There are enough consistency issues between browsers and their rendering methodologies to contend with already, so any framework we chose needed to minimize the amount of browser-specific coding we’d have to do. After experimenting with a variety of different toolkits, YUI came out quite clearly on top. There was a bit of a learning curve to all the products, but YUI’s had the best payoff.

The base framework does not require a plug-in, it plays well with .NET, and the scripts are light, tight and solid. Once we got the hang of the framework, we found it enlightening to compare our older traditional interface pages to the YUI versions. In every case, adjusting our UI methodology returned huge gains in performance and consistency with lighter downloads to our clients.

DocLanding: Mult-file uploads using YUI Uploader.

What YUI components are you using most heavily in your app?

We’re actually using quite a lot of the components. The most beneficial ones have been those that allow us to do as much with and on one screen as possible, so the TreeView, Menu, SimpleDialog and Layout Manager have been extremely useful. In truth we’re using nearly all the controls, but we especially appreciate the Uploader Control‘s ability to handle multiple file selection. We’ve been looking for a solution to that problem for some time and YUI’s has been the most elegant we’ve encountered so far. We make good use of the JSON Utility and Connection Manager to greatly minimize the size and number of requests to the server we make, which keeps our footprint down and more importantly keeps our users working, not waiting.

What’s next for DocLanding? What are the challenges you’re working to address in your upcoming releases?

We’re constantly working to improve the feature set of our product. Our users have asked for features to better integrate the editing of their documents with the main application so we’ll make time for that. We’re also working on better accommodating large file uploads. Otherwise, we have several ideas on the table and we’re weighing which ones would be most beneficial for our users. A version of the site optimized for mobile phones and netbooks is in the design stages already, as well as tools to import structured folders from the desktop directly into DocLanding. Experimentally, we’re toying with the idea of only storing the metadata at the website and pulling content directly from networked client machines running our software. Ultimately, the needs of our users will dictate in what direction we move next.

By Eric MiragliaMarch 30th, 2009

Building a Fast People-Finder for Flickr with YUI AutoComplete

At Flickr, we recently added a new people-selector widget to a few of our pages; this feature is based on the YUI AutoComplete Control. The people-selector widget allows our members to select individuals from their contact list, which can contain upwards of 20,000 entries. Due to the large amount of data involved, traditional techniques for fetching and parsing the data were not feasible, mostly due to extremely slow parse times. In this post, we’ll take a look at some of the different data formats we tried and at the AutoComplete configuration we found to be most performant.

First, here’s a video recap of what we were trying to accomplish; the new interaction with the people-finder widget is depicted on the right:

Fetching and Parsing: XHR and Custom Data

The biggest challenge was finding a data format that would be fast to download, fast to parse, and — above all else — secure. We first tried XML and Ajax, but XML parsing proved to be much to slow — in fact, we found that this approach could bring down the browser on larger data sets. Next we tried a combination of JSON and Ajax; this was significantly faster, but it still took more than 80 seconds to parse our largest data set (an array containing roughly 10,700 objects, each with several properties).

In the end, we found two transport/parse techniques that turned out to be extremely fast:

  1. Fetching JSON (wrapped in a callback function) using dynamically generated script tags;
  2. parsing a custom data format (a control-character delimited list) using split(), fetched with Ajax (using YUI Connection Manager).

In the end, we went with the custom format. Formatting our JSON so that it could be executed by a dynamic script tag was a less secure approach and not a performance win. Using XHR gave us a more secure and still very performant solution.

User Interaction: YUI AutoComplete

Once we had a way to get the data into JavaScript quickly, the next challenge was to create a way for the user to quickly search through the list of contacts. To accomplish this, we turned to YUI’s AutoComplete Control. It met our needs exactly: extremely fast and very configurable. To use it with our custom data, we created a function to use as the AutoComplete instance’s DataSource; every keypress in the widget triggers this function and passes in the search string. Within this function, we loop through all of the member’s contacts and try to match the query on four different fields. We used regular expressions to do the string matching.

Even for large sets of contacts, we found this technique to be extremely efficient. Here is the basic version of what we did:

function searchContacts(query) {

       var matches = [],
           queryRegEx = new RegExp(query, 'i'), // query should be
                                                // checked before 
                                                // using in a regex.
           contact;

        for (var n = 0, len = contacts.length; n < len; n++) {

               contact = contacts[n];

               if (contact.username.search(queryRegEx) !== -1 ||
                   contact.realname.search(queryRegEx) !== -1 ||
                   contact.emailAddress.search(queryRegEx) !== -1 ||
                   contact.alias.search(queryRegEx) !== -1) {
                       matches.push(contact);
               }
       }

       return matches;
}

Once we had the data connected to the widget, we made one change to the default AutoComplete configuration: We set the queryDelay parameter to 0 (the default value is 200ms). This means that there will be no delay between a key press and a search being initiated. There are downsides to this (the AutoComplete display tends to flicker a bit if you type a few characters in quick succession), but we found it to be the single biggest improvement we made, more important even than optimizations to our search function. While a queryDelay of 200ms or more might be more appropriate for XHR or other remote DataSources, we found that our regex-based DataSource with local data was up to the task of searching on every keystroke. With AutoComplete, we got free caching added to the mix so that any given search would only have to be done once.

More details on all of these techniques, including full details on the different data formats and extensive profiling data for each one, can be found on the code.flickr blog.

By Ross HarmesMarch 26th, 2009

In the Wild for March 25, 2009

News and notes from the YUI community in the past few weeks. Let us know in the comments what we missed, and we’ll get it next time:

By YUI TeamMarch 25th, 2009

Georgiann Puckett: YUI/ASTRA Program Manager (AdaLovelaceDay09)

Dav Glass and George Puckett of the YUI Team

[Note: This post is part of the YUI team's participation in Ada Lovelace Day, a celebration of female technologists around the world.]

Georgiann Puckett (better known as "George") serves as the program manager for YUI and affiliated projects (including the ASTRA library). Program management of complex technical programs comprising multiple projects is one of the most demanding jobs in a software company, and George is ideally suited to the challenge. She brings to the table a quick intelligence, the patience and discipline to manage large streams of data, and a deeply rooted understanding of the processes by which successful software programs are sustained. Her background serves here well — as a C/C++ engineering veteran, she can empathize directly with the experiences of engineers with whom she works.

YUI releases go out with hundreds of changes, many of which are suggested or contributed by developers around the world. Since joining the team two years ago, George has revolutionized the way all of that information is handled. That’s led to better forecasting, better communication, and better quality across the board.

George has also provided commendable leadership for the YUI team in supporting major internal projects at Yahoo. When we designate an internal project as a "big bet," something critical to the future of the company, we team up with the project’s frontend engineering team and make sure that we’re doing everything we can to support them. George manages these relationships, ensuring that our collaborators get timely, well-documented builds and that their priorities are accurately reflected in our release plans. Having the ability to understand the diverse projects’ needs and to faciliate our successful collaborations is no small challenge, and George has done the heavy lifting necessary to ensure that YUI and ASTRA engineers are providing the right support at the right time throughout Yahoo.

Speaking of heaving lifting…. George is well known at Yahoo as an exceptional technologist and a tireless advocate for YUI, but she’s also well known to those who frequent Yahoo’s employee gym. You’ll find George there four or five nights a week working to better her own world-record form on the free weights.

George’s work and her general commitment to excellence has certainly inspired all of us who work with her over the past few years. I asked George who had inspired her and sent her down the path toward a career in technology.

What was your first experience with computers?

I was intent on entering a pre-med track in college and I had an AP Calculus course my senior year as part of the college prep curriculum. As luck would have it, the teacher got a grant for two Apple computers as part of a trial to teach programming at the high school level. Not only did we get it – we got competitive at it trying to do the most robust features with the least amount of code. The first digital electronics course in college where i got to program circuits on a breadboard using assembly language sealed the deal.

Did you have any female technologist role models that influenced you?

There are two women I’ve worked with that I’ve been impressed by and learned a great deal from. Darragh Muldoon, co-founder of Cricket Software, hired me out of college into by far the the most amazing startup adventure of my career. She was was not a technologist per se, but I learned a great deal from her with respect to her people skills in leading technical folks, building teams, and growing a company. The other woman I look up to and learned from was Sheila Brady, who rose through the ranks to director-level in Apple’s system software division. She definitely knew how to drive a release, in many cases leading teams composed mostly of male engineers. She exhibited a level of confidence, competence, and aggressiveness that could be appreciated by any engineer — male or female.

By Eric MiragliaMarch 24th, 2009

Jenny Han Donnelly: YUI Engineer (AdaLovelaceDay09)

Jenny Han Donnelly, Sr. Engineer, Yahoo; author of YUI DataTable, DataSource and AutoComplete

[Note: This post is part of the YUI team's participation in Ada Lovelace Day, a celebration of female technologists around the world.]

Jenny Han Donnelly is the author of three YUI components:

  1. The DataTable Control: YUI’s DataTable is one of our signature UI widgets, providing a powerful menu of interactive options for tabular data.
  2. The AutoComplete Control: AutoComplete provides typeahead, suggest, filtration and combo-box functionality to any text input area.
  3. The DataSource Utility: Shared by DataTable, AutoComplete and the Charts Control, DataSource serves as a conduit between widgets and potential sources of data — including server-side data, JavaScript arrays, and DOM structures like HTML tables.

Jenny’s work inspires us in part because of the technical challenges she takes on — try getting fixed headers with xy scrolling to work in IE6 using a semantically sound base table sometime, if you have any doubts. Jenny has taken on some of the most complex HCI challenges anywhere in YUI and engineered them to suit virtually any environment. DataSource enables other YUI components to work with anything from flat files to JSON and XML to JavaScript arrays and DOM structures. We’ve heard from thousands of people on the YUI forums using all of these features and more in ecclectic and novel ways.

We’re also inspired by the organizational leadership Jenny has shown in her time at Yahoo. Currently, she’s the lead editor of YUIBlog, bringing technical voices from throughout Yahoo to these pages to share their insights. She has also organized our annual frontend engineering summit at Yahoo, bringing hundreds of Yahoo engineers from around the world together in a rich weeklong technical conference. She’s taught weeklong YUI courses to engineers in the USA, Korea and Japan, and she’s been an integral member of the hack day group at Yahoo that’s such an important part of our engineering culture.

Whether she’s coding, writing, teaching or leading — all of which are aspects of the modern technologist’s job description — Jenny sets a high bar with her intelligence, dedication, imagination and wit. Ada would be proud.

[photo of Jenny used by kind permission of Stephen Woods]

By Eric MiragliaMarch 24th, 2009

Survey: When is an Accordion not an Accordion?

example of an accordionI’m looking for feedback from people who have designed or built an interface using an “accordion” module (or are considering doing so). You see, I’ve been working on a design pattern for accordion modules, and I’d like to throw out a handful of open questions to the community via this brief survey. I’ll be listening elsewhere as well, on twitter (@mediajunkie) and on mailing lists where web designers and developers hang out.

(I realize this is not a scientific survey. I’m just interested in engaging the wider community in a discussion instead of trying to impose my view or Yahoo!’s view on the community as authoritative.)

Everywhere I go lately, it seems that interaction designers and web developers are talking about accordion widgets and debating about what makes an accordion an accordion. Not everyone working in this field has heard the term (some may simply refer to “stacked panels” or “collapsible panels”) but most get the gist fairly easily. Ironically, none of the UI elements described as accordions share the actual behavior of a real-world accordion (the musical instrument): namely, that stretching an accordion opens all the folds evenly.

Accordions have been an on-and-off topic of discussion on the main IxDA mailing list; we discussed them in our Pattern Library workshop in Vancouver earlier this month, and there’s been an ongoing discussion about accordions on our internal designer mailing list here at Yahoo!.

So I sat down with some folks from the YUI team (and Marco, the maker of an experimental YUI accordion widget) a little while ago to sort through a draft of an accordion pattern that might help inform the development of an official YUI component.

Broadly speaking, most people agree on what we’re talking about when we talk about an accordion interface element. Everyone agrees that accordions are used to compress content into a limited space and that they consist of panels that can collapse or expand. Beyond this, there are a number of subtle nuances that not everyone agrees on.

One trend I’ve noticed is that front-end developers tend be agnostic about how the accordion should work, viewing it as really just a variant on a tree widget. Designers tend to be more prescriptive, saying that to be an accordion it must behave in thus and such a way (but not all designers agree on what these rules are).

In the end, the YUI folks will produce code that can be made to do just about anything. We aren’t going to try to impose our own taste or preferences in design through the functionality of the code itself. However, we will use the associated pattern to make suggestions and recommendations drawn from the experience of the entire design community, and we will probably lobby for default behaviors that match what most people expect.

So, if you’ve got a few minutes and an opinion, please visit the survey and let me know what you think!

I’ll close the survey on April 30.

By Christian CrumlishMarch 23rd, 2009