Augmented-Reality Sandbox Turns Dirt Into a UI


WE’VE SEEN HOW kids take to touchscreens. To them, our unfathomably sophisticated smartphones and tablets are about as hard to figure out as a bucket full of blocks. It’s a little bit eerie, watching a youngster prod his or her way around an iPad; it makes you wonder if future generations will ever have a chance to foster any of the skepticism we feel towards cyborg enhancements and brain implants and other unsavory flavors of invasive, singularity-style computing. Hopefully they’ll grow into it.

But while all their deft, chocolate-fingered swiping might make a future of baby’s first tablet seem inevitable, a project out of UC Davis offers a glimpse of an entirely different type of high-tech educational engagement–one that doesn’t involve youngsters pressing their faces up against screens for hours on end. It’s a sandbox that lets kids mold its contents into miniature mountains, lakes and rivers–and then, with a little high-tech magic, brings that terrain to life before their eyes.

The project offers an irresistible combination: the timeless tactile joy of sand in hand plus a dollop of whiz-bang tech to top it off. A Kinect camera mounted above the sandbox tracks the physical activity below. As visitors young and old go about their terraforming, a projector throws a dynamic topographic map on top of it all, updating contour lines and elevation colors in real time. Then, the fun part: a virtual rainstorm, also supplied by the projector, sends a torrent of blue water cascading down the peaks, showing runoff and watershed on the landscape created moments before.

Inspired by a YouTube video from a group of Czech researchers, researchers at UC Davis’ W. M. Keck Center for Active Visualization in the Earth Sciences (KeckCAVES) started work on their augmented-reality sandbox in early 2012, as part of an National Science Foundation-funded program focused on water system education. Last summer, the video below, showing off their prototype, garnered over a million views on YouTube. Now, a pair of museums on either coast of the country are home to augmented sandboxes of their own–one at ECHO Lake Aquarium and Science Center in Vermont; one at the Tahoe Environmental Research Center at UC Davis–with a third being installed at the Lawrence Hall of Science in Berkeley this month.

The exhibits are presented with minimal instruction. Play, curiosity, and self-driven learning are all encouraged. “The power of the sandbox is that it’s trivial to make your own terrain completely from scratch and in moments, without any training,” explains Oliver Kreylos, one of the lead UC Davis researchers on the project. “There’s just no better way to teach how topographic contour lines work, or how water flows over a landscape, than building whatever terrain you can imagine, and then seeing the contours and the water react in real time to any changes you make.”

At ECHO Lake Aquarium and Science Center in Vermont, some 43,000 visitors have played with the interactive installation since it was installed in May. “We have watched our guests at every age dig in and explore with their hands, ask lots of questions, test ideas, challenge their friends and family, wonder about how it works, and express curiosity and joy,” says Julie Silverman, the museum’s Director of New.

She says the exhibit often fosters collaborations between strangers, prompting groups to build mega-mountains with all the sand or to see if they can flood the entire landscape to create an island chain. And according to Silverman, it’s the rare exhibit with replay value; people actually come back later in their visit for more time playing in the sand.

ECHO’s sandbox is one of three official versions of the sandbox in the wild, but Kreylos, the UC Davis researcher, says he knows of at least six others around the world, all built using the software and schematics his team has made available on their site. One of these was built by a high school robotics team in Ithaca, New York, another by a science center in Sao Paolo. There’s one in an elementary school in Australia and another, Kreylos says, somewhere in Mongolia.

Touchscreens are better than mouses, but surely dynamic environments are better yet.
The point is that far from a complex, custom-built device, KeckCAVES’ design is relatively simple for others to put together themselves. It doesn’t take much more than a Kinect, a projector, a spare computer, and some of their code. “It appears the installation is quite straightforward,” Kreylos says, “and I’m guessing there are a good number out there I don’t know about. I’m currently helping two groups doing final calibration for theirs, and another group is out buying equipment right now.”

The applications for the AR sandbox, Kreylos admits, are fairly limited. Sand can’t teach you much about math, or history. Indeed, much of Kreylos’ work involves developing immersive, intuitive virtual interfaces–next-gen tools that could prove useful for all sorts of disciplines. You can watch him demo one of his latest projects involving an Oculus Rift headset and two Razer Hydra joysticks here.