While the venerable computer mouse pre-dates many of its users, the simplistic two-dimensional pointing device still remains the de facto method for translating input from users to computers. In the decades following the mouse, joysticks, foot pedals, trackpads, gyroscopic input devices, trackballs, styluses and other unique peripherals have positioned themselves as possible alternatives.
Admittedly though, many of those devices have been relegated to the realms of special applications and extreme ergometry. In truth, only touchscreens and trackpads have proven themselves as alternatives with mass appeal, but even those popular devices aren't properly suited to replace the inexpensive mouse tethered to desktops everywhere.
The Leap Motion controller is another such attempt and an interesting one at that. The company touts the device as a new paradigm in human input devices, allowing users to wave, swipe and poke their way through a digital world otherwise meant for keyboards, mice and touchscreens.
So, what exactly is the Leap Motion controller and how does it work?
The miniscule gadget barely measures larger than a USB flash drive, however inside the Leap you'll find two cameras and three infrared LEDs. Using these components, the device is capable of tracking hand and finger movements in all three spatial dimensions. We've seen similar technology applied elsewhere (e.g. Microsoft Kinect), so the Leap Motion may seem a little less fascinating than it would otherwise. Nevertheless, making use of motion-sensing technology to interact with your computer is at least refreshing, if not magical.
It's hard not to draw parallels to existing technologies like Kinect, but it's worth noting the Leap Motion performs with remarkable precision. The controller is able to detect the smallest of movements (1/100th of a millimeter, according to the literature) with little perceivable delay. Despite this impressive accuracy though, the device is certainly not without its flaws. More on that in a bit...
First Impressions and Setup
The Leap Motion controller ships in a small, sturdy box with a premium feel about it. Inside you will find the controller base unit and two USB 3.0 cables with varying lengths -- one 6 feet long and the other 2 feet. Leap Motion has done a good job of putting together something that feels high-end in spite of its sub-$80 price tag.
For the unit we received, a paper manual was absent. However, owners just starting out will want to make certain their base is placed directly in front of them with the shiny side up and the green LED facing toward themselves. For set up instructions, users are instructed to visit leapmotion.com/setup, which is essentially a link to the latest controller software.
After installing the software, users are presented with a demonstration to get them acquainted with the device. It isn't a tutorial necessarily; rather, it's a tool that serves to "wow" you -- some spacey new age music and a way-cool visualization of what the Leap Motion controller "sees" as you wave your hands about. Although it doesn't do much to show you how to use the device, it's here that you begin realize (or fantasize) the Leap Motion's potential.
Ideally, the Leap Motion base sits in front of your keyboard. Despite this suggestion though, I found placing it in that location produced unwanted input while typing and futzing about at my desk. As a result, I placed the controller behind my keyboard to remedy this issue. Of course, reaching out further is a little more fatiguing during extended periods of gesturing, but the location was not entirely awkward or unpleasant for casual use.
I was asked to calibrate the device by pointing toward a shiny surface and moving it around. As the device is moved and tilted, you "paint the screen" until you achieve a good calibration score. In all honesty, I was left a bit mystified. "Am I doing this right?" I asked myself while moving the device around in front of a mirror (Leap Motion says any reflective surface will do, but mirrors are ideal), Interestingly, I couldn't actually calibrate it using a mirror. Following that failure, I was ready to turn in my nerd badge. Instead though, I pointed it at my matte 27-inch display and it worked like a charm. Go figure. To be fair though, calibration seemed adequate out of the box.
One More App-driven Ecosystem
To start using the device outside of the demo, users will need to visit Leap Motion's Airspace app store. Because the gadget provides three-dimensional input, applications have to be designed with it in mind to put the Leap Motion to good use. It should be noted that using the device as a straight-up mouse replacement is doable too, but users be warned: it's truly frustrating.
Airspace is Leap Motion's attractive dashboard for managing apps and it's available for both PC and Mac. It doesn't offer much in terms of features or configuration, but it looks slick and does what it purports. Although installing apps was generally a browse-point-click affair, I found the need to separately install "dependencies" experience-jarring.
Installing "Cut the Rope", for example, issued a warning that Microsoft .NET 4.0 and XNA Framework 4.0 are required. Airspace brings up some links where you can click to download and install each of the packages. Making this particularly annoying though, is I already had these frameworks installed on my system. There may be technical or licensing reasons for XNA and .NET not being included, but I'd like to see some sort of dependency detection and package bundling in the future for a true one-click install.
The Leap Motion Control Panel offers a reasonable selection of fine-tuning options, including interaction height adjustment and tracking precision/speed. I found "automatic" height (~22cm) and "balanced" were the best settings for me.
Right clicking on the system tray icon lets you bring up the Leap Motion Visualizer so you can see some nifty wireframe hand action whenever you wish. You can also pause and resume tracking.
Controlling the Leap Motion
Utilizing the controller seems pretty intuitive. The tiny Leap Motion base has a rather wide field of view directly above it. When you place your hand within its pyramidal view, a circle (or circles for multi-touch) appear on the screen. This lets you know where your hand and fingers before you interact with anything on your screen. If you want to push or control something on your monitor, you extend your hand and or fingers forward until you reach an invisible barrier -- this unseen "wall" is the threshold you must cross before the Leap Motion will register your jabs and waves into clicks and swipes.
In theory, the Leap Motion lets you rotate, pinch, zoom and move about within three whole dimensions. With that in mind, I thought Google Earth would be the ideal playground to test out the device, particularly since Google added Leap Motion support months ago.
After launching the app, before me stood our beautiful planet Earth -- I needed only to reach out and grab it. So I did... and so I failed. The slightest twitch of my finger sent the globe spinning frantically, a move which likely jettisoned all of its virtual inhabitants into the inky blackness of space.
I kept at it though. As I did, I discovered some helpful gestures -- make a fist to stop the spinning, flatten your hand to slow the spinning and so on. I even started looking up gestures online. Ultimately though, at no point did I feel like I mastered Google Earth with the Leap Motion. It's this experience which epitomizes my relationship with the device: it's incredibly neat, but flawed in this respect. The input/output isn't always what you intended and that lack of absolutely predictability occasionally breeds frustration.
If we forget a moment about the device's three-dimensional spatial awareness though, Cut the Rope is actually a good example of what works on the device. The reason it works is because the controls are simple and there isn't a lot to interact with -- you just need to swipe really fast across some cartoonish ropes. "Pressing" a virtual button in any app can be a bit tricky because you need to push forward with a certain level of perfection, lest you run the risk of not following through to your target. Meanwhile, as you are focused like a laser beam on the perfect button touch, you also need to be mindful of the invisible barrier -- the wall that separates movements from actions. Once you cross this imaginary barrier, your finger begins to "click" and swipe at things. Without a real perceivable difference though, determining where that barrier is takes a bit of practice.
Needless to say, pressing buttons indeed becomes easier, but there's no denying how difficult it is starting out. Recognizing this problem, some software titles (e.g. Cut The Rope, Frog Dissection, Corel Freestyle) incorporate an automatic selection feature that does the pushing for you if you hover over a button for a few seconds. I appreciate this and apps that incorporate this definitely get a +1 in my book.
Speaking of Frog Dissection, I was actually really disappointed at the lack of interactivity in this app. It may seem macabre (and naive), but I was really hoping to "peel" the skin back and dig into a frog cadavre with my bare hands. As you've gathered by now though, it didn't exactly work out that way; the experience felt hands-off and lacking. It's a fine educational tool to learn about frog parts, to be sure, but it was a recommended pick and I don't feel it properly showcases what's cool about the Leap Motion controller. It could have been done more easily with a mouse or touchscreen and I don't feel like I would have a missed anything.
Boom Ball though is a recommended pick that did a very good job showcasing the Leap Motion's features though. It's essentially a three-dimensional version of Breakout, adding depth (literally) to a classic game. It's not exactly Skyrim, but it plays almost like tennis -- a player tosses their ball forward, it breaks some stuff and bounces back. When it does, tilting your hand also tilts the angle of your paddle, which in turn can be used to coax the ball toward a desired location.
I was very interested in Corel Painter Freestyle, but I found its user input far too sensitive and shaky. I'm unsure why, but I gave up on it because it was nearly unusable to me. Freestyle supposedly makes use of the Leap Motion's depth to give the illusion of pressure, in addition to the usual X and Y that computer mice and most capacitive touchscreens provide.
Using Leap Motion for Productivity... or Not
As mentioned earlier, applications need to be designed to take advantage of the controller's unique three-dimensional input. For software that isn't though, there is always the "Touchless" app which runs in the background and allows you to trade in your mouse for hand and finger gestures. The Leap Motion essentially turns your PC into a touchscreen after all, so why not?
At first, I found Touchless an incredibly neat but entirely futile exercise. I hamfisted my way through Windows 8's Metro interface in what was a somewhat frustrating experience. I managed to accidentally do all sorts of things I didn't mean to do. Although my initial attempt to navigate Windows with the Leap Motion was difficult, my apparent clumsiness did manage to invoke some laughs from those around me. Let's just we're still pretty far off from manipulating our computers a la Minority Report.
After giving it the old college try and revisiting Touchless a few times, I did manage to improve enough that using a PC is possible. The key is to take it slow and be very deliberate. However, I admit it didn't take long before this became more trouble than it was worth. I'll definitely be sticking with a mouse and keyboard for desktop and Metro use.
Apparently, third-party utilities exist which can supplement your Leap Motion experience. Although I didn't try such programs for the PC, BetterTouchTool for Mac looks intriguing. I also found that some people have also written their own programs to do some really cool things, like this individual tearing it up in Borderlands 2, although admittedly after "many hours of practice".
I admit, the Leap Motion controller is something I want to fall in love with. It's just so cool. However, my experiences with the device indicate that at least for now it's more of a neat conceptual "toy" than a practical device you'll be reaching for every day. Your mileage may vary, of course.
Bear with me, but one of the controller's greatest failings seem to be its inability to deliver both 100 percent accuracy and simultaneously read your mind. It's a very sensitive device and it really does pick up the smallest of movements. I'm truly impressed by its precision. What it doesn't do though, is correct my not-so-perfect 87-degree finger jab or smooth out my heavily caffeinated, shaky hands. Actually, it does -- but not always. Sometimes it would mistake one of my knuckles as pointing; other times it would lose track of a finger because another one of my digits were vertically overlapping. Its this unpredictable but occasional input which adds frustration to the experience.
For the most part, the accuracy is there. To make the Leap Motion truly awesome though, it needs to better understand what you're trying to do. A good example is trying to "press" a software button in an app. I hold my hand really steady and concentrate, I push forward as straight as I can and sometimes I can't help but miss the target. I'm not a robot and I'm not always willing to carefully and deliberately move like one. Unfortunately for the Leap Motion, I found myself remembering the times I failed more often than I succeeded because its easier to remember faults. It's impressive, but not perfect.
Then there's also the whole, "What should I do with this thing?!" aspect of the Leap controller. That very conundrum has also been raised by tablets, mice and even the home PC itself (if you travel far enough back in time). I believe more novel uses for the controller are bound to pop up, especially given the endless creativity of software and hardware makers. For now though, there are quite a few apps which treat the Leap Motion like a touchscreen. Unfortunately, in many cases, this leaves you wondering why you'd use a Leap Motion controller in the first place.
As it turns out, "thinking" in three dimensions is a bit of a chore too. As intuitive as the Leap Motion may be and seem, there does exist a kind of hand-vision-mind learning curve to surmount. One does become acclimated to it given a long enough period of time though -- I'm sure my experiences would only get better with time, although I don't suspect there's much room for any sort of huge improvement.
After several days, my experiences with the Leap Motion controller were peppered with juxtaposing moments of enjoyment, amazement and frustration. With the right app, like Cut the Rope, Boom Ball, Fruit Ninja or even Google Earth (if you can figure it out), the device delivers brilliant fun. Sure, a mouse might be more accurate for some apps, but when it comes to casual gaming, the added challenge isn't really unwelcome. With the wrong app though, like trying to navigate your desktop with your fingers via Touchless or finger painting in Corel's hypersensitive Freestyle, the opposite was often true.
With that said though, the apps which make the most out of the Leap Motion controller appear to be games. The growing library of tablet-inspired games available through Airspace are generally forgiving enough and make good use of the device's features to produce an enjoyably unique experience. With all said and done, the Leap Motion controller is good for play, but bad for work.
Pros: Impressive tracking capabilities for a first-gen device. Small footprint. Not expensive. For now, one neat toy to play around with.
Cons: Depending on the app, making things work can be frustrating. Better suited for simple gesture games. Poor performance on productivity scenarios.