My app was just on TV

CNN just did a piece on the influx of people into DC for the inauguration.  Several of the tools the CATT Lab has developed are going to be used to help manage the traffic problems that are guaranteed to crop up as a result.


It’s a quick spot, but around the one minute mark you’ll see some of those tools, including my baby, the Incident Timeline.  The Timeline lays out every action that’s been taken to clear a traffic accident scene along a temporal axis so traffic operators can quickly discern what’s going on at the location.  Without the Timeline, the operators would be stuck looking at a list of timestamped descriptions, and that’s not really ideal when quick decision making is key.

Also featured are the lab’s travel time prediction tool and a Google Earth-like application we like to call the Virtual Helicopter.

Posted in Uncategorized | Comments closed

The ComparisonMatrix component… or Playboy Playmates and the Economy

One of the challenges of information visualization is to present data in a way that allows the interesting patterns to rise to above the noise.  The ComparisonMatrix allows users to quickly assess where these patterns are so they can make an informed decision about where to start exploring.  To do this, the ComparisonMatrix breaks a multidimensional data set into all of the possible pairs of attributes and evaluates the relationships between those pairs.  The relationships are rendered in a grid where the cell at the intersection of the column labeled “X” and the row labeled “Y” shows you the relationship between X and Y.  The cells are colored to provide a quick visual cue as to which relationships are most interesting.  This is similar to the rank-by-feature prism found in HCE.

How about an example?

In 2004 Terry F. Pettijohn and Brian J. Jungeberg looked at the relationships between the physical features of Playboy’s playmates of the year and the economy.  I grabbed the csv for the playmate data off of Flowing Data, added a few fields to it, and dropped it into a ComparisonMatrix (view source enabled).  Pink represents a positive correlation while yellow represents negative.  The deeper the color, the stronger the relationship.  You can click on a cell in the matrix to bring up a scatterplot of the data set based on the two selected attributes.

The strongest relationship in the playmate data set is actually between GDP and year, so that makes it somewhat difficult to make any assertions about trends in the playmates’ physical features and the economy while not making the same assertion about those physical features and time.  (The authors of the paper actually used something called the “General Hard Times Measure” to evaluate the economy, and it looks like GDP isn’t really equivalent).  Despite this problem, the ComparisonMatrix still provides some interesting information about the playmates.

  • The playmate’s heights, weights, and waists are all correlated.  These women have roughly the same figures, just scaled up and out differently.
  • The playmates’ bust to waist ratios have decreased with time while their waist to hip ratios have increased.
  • BMI, though being a function of height and weight, is more strongly correlated with hip size than height.
  • There is no relationship between the age at which these women claimed their titles and their hip sizes.  That’s right, if you have nice hips, it might not be too late too be a playmate.

Using the ComparisonMatrix

Like all my stuff, the ComparisonMatrix is under the MIT license, so you can use and adapt it in whatever way you like.  The code is documented pretty thoroughly, and the source of the playmate example is pretty useful for getting started.

In terms of usage, all you need to do is supply the ComparisonMatrix with the data set you want to analyze (the dataProvider property) and the array of attributes it should consider (the fields property).  Given those two properties, the ComparisonMatrix will be able to render your data, though there are a bunch of other options available.  Take a look at the documentation in the code for details.

The default comparison function is the correlation coefficent.  As a result, the default behavior is to expect comparison values from -1 to 1.  Since the ComparisonMatrix allows you to plug in your own comparison function, you’ll have to provide your own color and alpha functions if you want to use a different scale.

I made an effort to make the renderers (the actual cells in the grid) as flexible as possible so you can write your own without rewriting the ComparisonMatrix itself.  To do this, write a class that implements the IComparisonRenderer interface and set the comparisonRenderer property to be a factory for that class.

The next step

Statisticians and information visualization researchers have developed a slew of measures for determining interesting relationships between attributes in a data set.  Even though the ComparisonMatrix allows you to plug in whatever comparison function you want, I’ll probably build these in at some point for easy access.  As I make updates to this component, I’ll be committing them to my repository.  As a result, the example here will not always be the most up to date.  For the most up to date code, check here.

I have lofty dreams for this thing.  More later.

UPDATE (January 21, 2009)

I wasn’t kidding.  The ComparisonMatrix has changed a bit.  If you want the best matrix you can get, grab it from the repo, not the source of the swf.

Posted in Flex, Information Visualization | Comments closed

How to check if you’re running a debug swf

There was a brief thread on flexcoders recently about how to check if an swf was built for debugging or for release.  With this check you can build all sorts of debugging diagnostics into your application and then compile an unaffected release build without changing any code.

There is no built in way to perform this check, so putting it modestly, the solution is a bit of a hack.  In a debug swf, the stack trace contains line number information that is absent in a release swf.  To check if you’re in debug mode, all you have to do is throw an error, catch it, and check the stack trace for the square brackets that surround the line numbers.

Here is a little class I wrote that will perform this check.

package com.michaelvandaniker.capabilities
{
    public class SWFCapabilities
    {
        private static var hasDeterminedDebugStatus:Boolean = false;
 
        public static function get isDebug():Boolean
        {
            if(!hasDeterminedDebugStatus)
            {
                try
                {
                    throw new Error();
                }
                catch(e:Error)
                {
                    var stackTrace:String = e.getStackTrace();
                    _isDebug = stackTrace != null && stackTrace.indexOf("[") != -1;
                    hasDeterminedDebugStatus = true;
                    return _isDebug;
                }
            }
            return _isDebug;
        }
        private static var _isDebug:Boolean;
    }
}

I’ve tested this class in the debug and release versions of Flash Player 10.0.12 and 9.0.124.

Posted in Uncategorized | Tagged | Comments closed

Presenting at DC’s World Usability Day Event

Today is World Usability Day, and this year’s theme is transportation. I’ll be presenting the work I’ve been doing at the CATT Lab at DC’s local Usability Day event. The title of my poster is Real-time and Historic Incident Visualization Using Timelines, and it describes an application that allows users to understand everything that has been done to manage an accident scene by presenting all of the events along a timeline.

There are seven other people from the CATT Lab presenting their work as well. The majority of that work is in the realm of information visualization. If that’s your thing, and you’re in DC today, stop by the American Institute of Architects and see what we’ve been up to.

Happy World Usability Day, everyone.

Posted in Uncategorized | Comments closed

Adding hints to Flex 3 MXML tags

ColdFusion tags can be annotated with the hint attribute, effectively creating a surrogate for single line comments. This is something I’ve wanted to be able to do in MXML for a while, and I’m glad to see the Gumbo specs include this feature in the form of private attributes.  If you can’t wait for Flex 4, you can apply this monkey patch to add hints in your Flex 3 MXML.

For those new to monkey patching, the technique allows you to rewrite classes from the framework or from swcs by defining your own class with the same fully-qualified name. Through the magic of inheritance, any changes to a class will cascade down to its subclasses. To create hints we just need to add a do-nothing public setter method called “hint” somewhere where it will be inherited by the entire SDK.  If we could change the source for Object we’d be done, but we can’t so we’re going to have to patch a couple of files.  Practically every file in the SDK has the line

include "../core/Version.as";

so that’s our best starting point. Create a directory structure for the mx.core package in your main source directory and copy the SDK’s Version.as into that directory. Add the line

public function set hint(value:String):void {}

to the copy. This isn’t going to change anything immediately because the compiled classes are still using the old Version.as, but this step makes the rest of the patch completely painless. To get the change to go through we have to grab a copy of each of the classes we want to add hints to and place them along side the patched Version.as.

Most MXML tags describe components, so grab FlexSprite.as from the SDK source and place it in the same directory as the new Version.as. Without editing FlexSprite.as you can now say

<mx:Panel hint="The compiler is cool with this." />

Sadly, FlexSprite isn’t the base class for everything that can be declared in MXML, so there are a couple of other classes you’ll need to grab if you want hints for them too. A few good candidates for this patch are mx.styles.CSSSytleDeclaration, mx.effects.Effect, and the entire mx.states package. I’m not going to try and patch everything, so if you use this technique and the compiler complains about the absence of the hint property, just grab the source for whatever class needs patching and put it in the patched mx package.

Posted in Flex | Tagged , | Comments closed

Parallel Coordinates Plot in Flex

I spent the past few weekends writing a parallel coordinates plot.   For the uninitiated, parallel coordinates plots are used to show the relationships between items in a multidimensional dataset.  Three dimensional data points can be difficult enough to visualize on a two dimensional screen, and when you throw even more dimensions into the problem, things just get nasty.  Parallel coordinate plots mitigate this problem by rendering each of the n axes parallel to each other and each data point as a series of line segments corresponding to where the point would appear in n-dimensional space.  So the point (3, 10, 7, 6) would be rendered like this:

Here’s a little example application using the period table of elements as a data source.  If you’re interested in the details of the implementation you can check out the source.

When I started the project I was concerned that using the charting classes as a base would bulk up the code and ultimately hinder performance, so while the MXML that defines the plot is reminiscent of the charting components, the base is actually UIComponent.  I’ve left plenty of room for extension in the classes that make up the plot.  It’s licensed under MIT, so if you want a different renderer for each axis, you got it.  If you want to draw loops instead of straight lines between the axes, go for it.  I’m going to add the project to my repository, so if you’re interested in any updates that I make to the code, you can check there.

If you need variety, two of the other members of the Fervor team have started open source projects for their own flavors of PCP.  (Swapping the acronym “PCP” in for “parallel coordinates plot” makes sentences more interesting, don’t you think?)  PCPlot belongs to Dasha and contains the code more or less as it appeared in Fervor.  FlexPC is a project Krist started with a couple of developers he met through a flexcoders post about PCP, but that project is still in the planning stage.

Posted in Flex, Information Visualization | Comments closed

Memory leak caused by setTimeout

I have a Cairngorm-based application that polls a server for data every few seconds.  To do this I have a command that initiates the server request, and in the result handler I set a timeout.  When that timeout is up, I dispatch the event that is responsible for kicking off the command in the first place.  It looks something like this:

public class PollServerCommand
{
   public function execute(event:CairngormEvent):void
   {
      // Create and call delegate 
   }
 
   public function result(data:Object):void
   {
      // Process data...
      setTimeout(2000,poll);
   }
 
   private function poll():void
   {
      var pollEvent:PollEvent = new PollEvent();
      pollEvent.dispatch();
   }
}

This results in lots of commands being created, but in theory the garbage collector should be cleaning up the old ones every so often.  Well, that’s not really the case.  While profiling my app, I noticed that every polling command I created stayed in memory.  The solution?  Swapping out the calls to setTimeout for instances of the Timer class like so…

public function result(data:Object):void
{
   var pollTimer:Timer = new Timer(2000,1);
   pollTimer.addEventListener(TimerEvent.TIMER,poll);
   pollTimer.start();
}
 
private function poll(event:TimerEvent):void
{
   var pollEvent:PollEvent = new PollEvent();
   pollEvent.dispatch();
}

I’ve always known that the Timer class was preferred over setTimeout, but now I know why. Can anyone from the Flash Player Team shed some light on why setTimeout is keeping these objects alive?

Posted in Flex | Tagged , | Comments closed

HeatMap added to Google Code Repository

I’ve started a Google Code repository Github repository for all of my little experiments.  At the moment it just has the HeatMap component, but — baring any unforeseen circumstances — it will soon flourish.  You can check it out here.

Posted in Flex | Tagged , | Comments closed

HeatMapping the Spread of WalMart

I recently came across a post on FlowingData with a visualization showing the growth of the WalMart chain over the years.  It’s very well done, but all those icons really clutter up the map.  The fact that they’re semi-transparent makes it possible to distinguish the dense areas from the sparse ones, but it’s difficult to tell those intermediate shades of green apart from one another.  It sure would be nice if the color varied across the map instead… heat maps to the rescue!

I’ve put together my own animation of the WalMart dataset using ModestMaps and the HeatMap component.  The visualization is most impressive when you’ve zoomed in far enough to see all of the major roadways converging on the large cities.  From there you can watch as WalMart takes over the urban areas, turning them red hot while leaving the less populated areas (relatively) cold and blue.

Unfortunately my dataset is somewhat incomplete.  The version I grabbed from Thomas Holmes has addresses for each store, but the geocoding service I used failed to convert about 100 of those addresses to latitude and longitude values.  If anyone can point me towards the complete dataset, I would love to add those missing stores to the visualization.

Before I close I want to give a nod to Universal Mind.  Last week they released SpatialKey, a heavy-duty temporal-spatial visualization tool that really shows off the strengths of heat mapping.  They’ve taken a crack at the WalMart data set as well, and it’s impressive to say the least.  Congratulations on your launch, UM.

Icons by famfamfam.com

Posted in Flex, Information Visualization | Tagged , , | Comments closed

How to make heat maps in Flex

Last semester I worked with Darya Filippova, Joonghoon Lee, Andreea Olea, and Krist Wongsuphasawat on a project for Ben Shneiderman’s Information Visualization course.  We took a subset of the accident data collected by the CATT Lab and built a tool to help figure out why certain stretches of road have a higher number of accidents than others.  We developed a number of visualizations to achieve this goal.  My major contribution was the heat map.

I’ve only seen a few Flex apps that use heat maps to show clusters of data, and I’ve never come across any source code.  Corunet has a blog post detailing their server-side heat map algorithm, and I ported it to Actionscript.  I’ve continued to play around with the code since the semester ended, making it more efficient, extensible, etc., and I think it’s about time it saw the light of day.

The application below traces your mouse movements, storing the x and y positions in an ArrayCollection which is fed to the heat map.  The size of the collection is capped at 1000 points because I didn’t want to blow up anyone’s browser, but any machine that can actually run a web browser shouldn’t run into any problems with that many points.

You can view the source here.

The algorithm isn't all that complex, and the functions built into the BitmapData class take care of a lot of the dirty work.  Basically, a circle with a gradient fill is drawn for each point in the collection.  The gradient ranges from black on the edge to blue in the center.  Bluer regions represent areas that have had more "influence" than others.  The exact shade of blue used in the center is dependent upon the maximum number of coincident points.  As this number increases, the relative influence each point has decreases, preventing the entire heat map from being rendered as a supersaturated mass.  The BitmapData's threshold method is used to convert the black-blue image to a fully colored heat map.

The HeatMap class is driven by an ArrayCollection of Objects, and the Objects are mapped to the screen coordinates based on their x and y values if they exist. Because not everything has an x and y value, the HeatMap has a transformationFunction property which is used to project objects to screen coordinates. Also, a weightFunction property is available which allows weights to be assigned on a per-object basis. The usage of these to functions is similar to that of the dataFunction property in the charting classes.

That's all there is to it.  The source is pretty well documented, so you can get the nitty gritty details there.

Update (August 25, 2008)

I've added the HeatMap component to my Google Code repository, so the source shown here isn't the most up-to-date.  Rather than continually update this post when I make a change, I'll let the most current revision speak for itself.

The HeatMap, and all of my future opensource work will be licensed under the MIT license.

Posted in Flex | Tagged , | Comments closed