AI Scheduling Visualization

For this project, I was hired onto an existing AI research project helmed by Dr. Edmund Durfee to help with data visualization.

Dr. Durfee's research is about the coordination of schedules between multiple individuals, a famously challenging problem in AI. From a mathematical perspective, the problem is NP-complete, which means that there is no straightforward way to optimize the schedules of multiple people to make everyone as satisfied as possible, so research continues to attempt to find new methods of finding solutions that are close enough to suffice.

For this project, the software was designed to create multiagent schedules that prioritized flexibility over all else. The project was intended to be used by families with disabled children, who often must adjust to unexpected medical emergencies. Additionally, some cognitive disabilities can make it challenging for children and teens to make and execute plans or balance current desires with future consequences.

The scheduling software designed by Dr. Durfee and his team of graduate students calculates all the possible configurations of the day's schedule, and as individuals "lock in" their choices about how they spend their time, reflects how that impacts future possibilities. If someone presses the snooze button on their alarm clock, the rest of their morning routine becomes constrained, and optional activities such as a morning jog may become impossible.

I was given the task of creating a data visualization in Java for this utility. This was a challenging brief! Not only did this visualization need to show a full day's schedule, it also needed to show the full range of possibilities for how a day could be ordered in a way that would be intuitive to a new user.

The data for me to render only existed in a complex XML data structure, so the first step for creating a visualization was to program a utility to access this data structure smoothly. I programmed custom code to access and request schedule updates from the research codebase, as well as a utility for end users to specify the constraints on their own schedules without needing to write custom computer code.

Once these basic utilities were complete, it was time to conceptualize possible designs for visualization. I eventually settled on a box and whisker plot paradigm in which the whiskers would show the times a task could be completed and the box would show the amount of time the task would actually take. As users made decisions that constrain the rest of their day (and the schedules of their family members), these charts would shrink and change color to represent how deterministic the schedule has become. Once a task is entirely locked in (for example, a doctor's appointment from 2-3 pm) it would be represented in red, but a more freeform task (such as an evening shower) would be represented in cooler colors like blue.

Then, I began work on the visualization software. Because I was instructed not to import additional packages to the codebase, I coded the UI entirely in base Java.

My visualization code made it possible for users to view a complex data structure that represented all possible futures in a day's schedule. Instead of presenting candidate schedules or branching paths, the final version of my visualization software showed the possibilities in a single screen that was designed to be accessible to our audience of cognitively disabled teenagers. We empowered our users to audition possible decisions they might make and see the the effects of their choices propagate forward.

My visualization code and design is still in active use more than three years later by researchers and users alike!