This is a presentation of all the key gestures I’ve collected based on recordings of 4/6 user tests that I conducted for extracting people’s natural instincts of interacting when using gestures.
User 1
“OPEN THE CURTAINS”
This gesture was intended to signal the initiation of an interface. This user used this gesture in attempt to “start” when the screen implied to “hover over”. He uses it because he wants to proceed to the next page. For this example, I tried multiple opening interfaces to see if the user would react differently, when the hand-waving gif was not present on the interface yet, the user opted for this gesture to initiate the “start”.
“THE WAVE”
The wave gesture was what the designed interface is trying to convey. The idea is to make the computer vision notice the presence of a person, by the person initiating a wave, to say “hello”. This gesture was used when the hand-waving gif was present.
“THE NUMBER POKE”
The user’s first instinct of navigating a multi-optional menu was to present a number using his fingers, signaling he wanted either the first, second, third, or forth option to be selected.
“THE PALM CURSOR/PAN AND GRAB”
The user proceeded to change his instinct gesture of the number poke when navigating the menu into a pan and grab gesture when noticing that the cursor is present and can be controlled by hovering, grabbing=selecting.
“GRAB AND PAN”
The grab and then pan gesture when the user was navigating a catalogue where he tries to “flip” or “scroll” to the next page.
“THE NUMBER POKE TO GRAB”
The user used the number poking gesture again when wanting to select an item in a page with multiple selections. When selecting, he used the grabbing gesture, which appeared smooth because he simply had to shut his hand into a fist from pointing a “five”.
“THE PALM CURSOR TO GRAB”
The user implemented the panning with palm gesture again to navigate a menu with multiple selectable buttons. He didn’t use the number poke for this one (why the inconsistency?). Again, grabbing to signify selecting an object.
User 2
“VIRTUAL TABLET”
This user utilized their finger to “virtually” scroll and tap the screen when wanting to select an item from multiple options.
“THE WAVE”
The wave gesture was what the designed interface is trying to convey. The idea is to make the computer vision notice the presence of a person, by the person initiating a wave, to say “hello”. This gesture was performed by this user as intended by the interface, where a hand-waving gif as well as text instructions for how to proceed were presented.
“ZOOMING AND MINIMIZING”
When presented a map, the user had the instinct to zoom in to be able to view the information presented clearly. she proceeded open up a fist to signal zooming in, and vice versa for panning back out.
“THE POKE”
When selecting an item, this user opted use her fingers to to “tap” in the air to select an item while her hand/fingers are not too close in proximity to the screen.
“NEXT PAGE PLEASE”
When presented the start screen of the SAS infotainment interface, despite my attempts of putting the hand-waving gif up front this user still opted for a quick swiping hang gesture to move on to the next page.
User 3
“THE PALM CURSOR”
The user, when presented a menu with multiple options that open up modals, used their palm as a virtual cursor to slowly pan between the different options available.
“DON’T PUSH THE RED BUTTON”
When selecting an option, the user uses a “pushing” gesture to signify selection. Imagine having a big control panel, and pressing a big button the size of your palm.
“THE WAVE”
The wave gesture was what the designed interface is trying to convey. It tells you to wave to start scanning. The idea is to make the computer vision notice the presence of a person, by the person initiating a wave, to say “hello”. The wave was the first instinct for the user to perform upon seeing the text “wave to begin scanning on screen”, there was no hand-waving gif in this example, but the user performed the waving gesture unlike some other users that didn’t until they saw the gif.
“PAN AND PRESS”
The user maneuvers their palm acting as a cursor to the press-able button and does the push button gesture again to make his selection.
“THE NUMBER POKE”
When presented with a more complex interface like this with multiple text options, the user presented numbers with his fingers to convey which option he wanted, despite there being no numbers beside the options.
“THE DOUBLE NUMBER POKE”
A follow-up of the number poke. Here you can see that the user was trying to input two options, three and five, by gesturing number three and five with his fingers. I call it the double number poke.
User 4
“THE WIDE SWIPE”
The user explained this gesture as “like turning a page in a book”, the user performed this gesture when the instructions to “wave” at the screen was not present.
“THE GRUMPY OLD MAN”
When then presented with text instructions on the screen to “wave to begin scanning’, the user stated that he would probably still do the “wide swipe” gesture three times, get annoyed, read the instructions on the screen again, then proceed to do the wave.
“THE INTIMATE POKE”
According to the user, when presented with a “button”, he would of course do the poking gesture, with finger close in proximity (as opposed to previous user that did the poke and poked from afar).
“THE WAVE”
When presented with instructions to “wave” at the screen for this example, the user went directly for the wave because the text instructions were more “prominent” on the screen as opposed to how it was in the previous example.
“THE SCROLL-UP”
When asked how this user would scroll up a rather complex text-based interface, the user performed this scroll up gesture. Whole hand is involved not just finger.
“THE SMALL SWIPE”
When presented with multiple options in a menu for locating a store in a mall on an info kiosk interface, he would opt for doing small swiping gestures, kind of like dragging an item from the left-hand menu and bringing it up onto the map.
“THE THREE-FINGERED ZOOM”
When presented a map, the user had the instinct to zoom in to be able to view the information presented clearly. she proceeded open up a fist to signal zooming in, and vice versa for panning back out. Compared it to “like on the iPhone”.
“THE SWIPE UP TO OPEN”
When presented with a graphic menu where you can open to present new windows, this user wanted to swipe up to open. Maybe it’s because he noticed the undertext of each element slide upwards that triggered that response, but we cannot be sure.
Journal with me! 