Mobile UX Design Series: Designing for Touch (Part 2 of 4) – Josh Clark

July 27, 2019 posted by

welcome everyone we're going to get going like eat oh I like to welcome you to part two of our four-part series on designing for touch part of our user experience track with the New York Technology Council or nonprofit into the association here in the New York greater new york area first I want to thank our sponsors sounds trite but if it worked for them we wouldn't be here tonight Frankfurt Kernan giving us the space tonight here's one of our sponsors and with that let me introduce Charles Mora who organizes this user experience track and Charles thanks Eric ok thanks everyone for coming tonight this is the second in our mobile series and just a couple of points that before we start off we're looking for a larger home and if any of you work for large corporations or have contacts with venues that work with nonprofits would be great to hear from you and contact us through e mail that would probably be the best better are as Eric said our last two sessions are now up Paul thurman session the two ago and then the session with Jason farming from last time both of those went up this afternoon we will send you the links to both of these sessions with our follow-up survey which you'll get tomorrow morning around nine o'clock incidentally we read all of the feedback all the surveys and we take all of that response very seriously ok if you want to follow us on Twitter of course we set this up a couple of months ago and we're have substantial following they're like everyone else we have facebook as well just a couple of the events that are coming up only in the ux track coming up part 3 opportunities in mobile and complex business models mr. Reese from google who show up last time is coming back we've roped him in and david fine from time warner will also be here both great speakers and it should be a very interesting program and then part 4 we have specialists and simulation ux design tools and we've identified a couple of speakers there but haven't announced them yet ok so for tonight's speaker josh Clark he's designer a designer that specializes in mobile strategy and user experience Josh's business global Moxie is a high regarded consulting firm here in the city and has done some very innovative work in the category of mobile and responsive sites he's the author of three books most specifically tap worthy just published recently I guess is now in four other languages or something like that and he worked as a management consultant for monitor I don't know thinking no monitor but it's a very large firm sort of on the level of Mackenzie and he created a number of years ago a relatively famous program called couch to 5k which was responsible for bringing a significant number of people to formal exercise more interesting I think is the fact that he was a producer for PBS wgbh in boston and he interviewed among others a Margaret Thatcher Gorbachev and Nancy Reagan but it not necessarily in that order and he's a graduate of Harvard so Josh welcome thank you thanks everyone we swing you guys some of my slides here so yeah this is a Ottesen has been a mobile UX track and I'm really pleased to be part of the series my principal sort of area of practice is both in terms of strategy of how do you use all these different devices but in particular to a particular favorite topic of mine is just to think about one of the new interactions that these devices propose today of course I'm talking about touch I'm super excited about the possibilities of touch interfaces that we've already begun to see but also still the things that we've yet to really properly explore as a UX practice and I believe it that touch forces are should force really fundamental differences where does the fundamental changes in the way that we approach user interfaces both as as consumers people who use these things but also and really more importantly perhaps as designers I think that touch is going to help us to sweep away decades of buttons and menus and folders and all this administrative debris that we've accumulated over the last three decades of the graphic user interface so you get rid of this thing you know the cursor in the mouth these prosthetics that we've been using appointed stuff for the last 30 years all that remains is you and the device or better yet you in the content that's the illusion that we have the opportunity to create you know all user interfaces are illusion right just a thin layer of magic that we stretch out over the stern of ones and zeros to try to make what's happening understandable an exciting thing about touch so we can create this illusion of direct interaction the illusion that there is no illusion right which is sort of an exciting shift it cuts through complexity to give the impression that you're working directly with the content manipulating information as if it were a physical object kiss you know buttons and controls and all the things that we've been using and graphic user interfaces do add complexity to layer of abstraction and visual processing that we have to do to understand how to work the thing you know the problem that we always confront as designers the more features that we want to put into software the more controls that we need right so you get this clutter of toolbars and visual complexity I want the great things about touch and gesture is that you can add features without adding that visual complexity that comes with trade-offs right how do you understand that the feature the features and functionality that aren't visually advertised and we'll talk about that in a little bit but it one important piece is that some of those traditional controls just don't work as well can you put them into a touch interface and it's particularly true I think it's a touch interfaces get larger if you consider the iPad firms for example same looking at email on my iPad maybe it's email from thinkgeek by the way if you're not doing all of your shopping at thinkgeek com I think you're probably not hooked up right and to stay in check it out so get this email right but I want to go back to my sent messages the ipad mail apt like so many ipad apps uses this so called you i split view to have this back button that reveals you know the sort of the sibling content at the same level of information so i reach up and pack at this little button and I'm tapping at all these little tiny controls and specific areas this big expansive touch screen but my interaction is just limited to this little tiny area probably more sensitive to this than most people but friends I want to share something with you and that is that then yes hates the back button but loves keynote transitions and the same way we'll see that one again if its law comes into play here many of you are probably familiar with Fitz law but quickly it's it's this this principle that emerged out of studies that were done in the middle of the last century sort of measure how long it takes to move a physical object with some kind of a pointer think about a puck in a shuffleboard stick to move it to a target the emerging principle is pretty obvious not surprising now the smaller the target and the further away it is the harder it is to hit it's what makes golf such a miserable sport it's long this came to be in the 80s applied to visual interfaces and so sort of began to inform where you should put mouse targets and buttons and how large they should be and what proximity they should have and again it's just the idea is that the further away it is and the smaller it is the harder it is to hit the more concentration it takes turns out of words for touchscreens too even though it's not a pointer like a mouse or like a shuffleboard puck but your actual finger it maps pretty well the mathematics of how hard it is to hit targets maps pretty well especially as these screens get larger because if you think about iOS if you have these same buttons same size buttons on the phone versus the iPad easy to hit on the phone it's a sweep of your thumb on the iPad you're lifting this whole meat pointer up and over into this sort of top corner and he's actual physical effort let along the concentration to actually hit these things you know our sort of guiding principle as designers ought to be this let me be as lazy as I can right don't make me work for it certainly don't actually require physical effort to do something it's something to understand is that when you're dealing with these physical devices there's now physical ergonomics to consider here and it's a sort of you don't want to make me work too hard just to sort of push it those buttons if you even need buttons at all but can you at least provide an alternative so iOS for example to return he back in iOS 5 instead of making me hit that inbox club they introduce this thing to sort of swype out from anywhere on the screen and then swipe it back in so I can just do resting my hands here at the side just get a quick flip no arm motion required and I'm starting to use the entire screen as a control right a course gesture instead of fine tune pecking and that sort of the opportunity that this introduces what else might we do but one thing that you could do that they don't do we could start to think about things like well five finger touch could take me directly to my sent mail are there problems with this right one is discoverability how am I supposed to know that a five finger touch will do anything because if you don't advertise it's just an easter egg right you discover it by accident or someone tells you about the surprise another thing is accessibility everybody has five fingers right said New Orleans a few months ago my cab driver had hooked welcome to New Orleans obviously this guy's going to have trouble with any touchscreen interface right but there but even for sort of less severe disabilities it's pretty simple things of mobility not everyone's going to have the dexterity or the ability to use more complex gestures like this so can't be the only way to do things especially when you think about these sort of shortcuts for doing things and moving through your application your interface for to quickly do you really think about these as a keyboard shortcuts of touch you know that these are things that are often sort of understood to be advanced features but not the only way to do things what makes some shortcuts it's this time-saving aspect that you slap it the whole screen instead of concentrating and fine-tune pecking at this slapping the screen instead of sort of fine-tuned control let's talk a little bit about this reader it's an iPad app to read google reader or at least for the next month or whatever it is where we still have google reader it's okay i am i'm looking at one of my newsfeed is an article that I'm reading here's some unread articles in this in the speed that I have and if I want to go back to the group of feeds that I've accumulated here I've got a back button so there it's visible but also i've got a gesture of possibility here which i can just pinch it this thing it closes that out it takes me to this group i can pinch again it shows my list of groups this is a sort of an emerging pattern that we're seeing called semantic zoom we're used to be seeing zooming in and out on sort of a very physical metaphor right maps and photos we're starting to see an emergent more of a semantic metaphor of zooming in and out of information levels expanding exploding a group or collapsing a group to go to the group above it but one of the things that sort of putting that sort of metaphorical notion aside what this means is that I could just be sitting here I can pitch anywhere on the screen in the corner wherever my hands are convenient I don't have to reach up and hit that button so again there's this economic convenience to this as well as somewhat elegance medication metaphor again there's a there's a discoverability issue and what good will get to that throw it toward the end how do you manage that stuff and still there's a back button here but you know do you even need that Twitter before they wrecked their iPad app and put in something completely laying add this app for their for the ipad thing and what's interesting about this is they specifically tried to kill the back button I think it's good success what Twitter does if they sort of put with this version of the app that was really encourage you to explore very deeply through the little rabbit hole of one single tweet right so you tap on the tweet and I will show you the tweet in the URL there tap on the avatar get the profile to see more tweets there cap another and so on then you sort of go back you just swipe back right so sort of you know in two things are happening here one thing that's very important is that when you're designing a touch interface you want to give physicality to your information objects right so in this case it's saying well let's take every slice of your history of Twitter and give it physicality make it a card the card the page very sort of basic metaphor of all touchscreen interfaces of course even from the web you know we've got this very page interface of course you know if you've got a browser well you've got to have a back button right but when you think about the basic metaphor of flipping through pages and it's the last time you turn to page in the real world using a back button bring some metaphor right so it puts this layer of abstraction and into that really basic physical metaphor and here's just a cards just like just swipe through the cards to get back through and again you're using these very broad gestures is to slap at the entire screen rather than pecking away at one but this is such a big thing that I want to mention here is that especially these screens get larger you guys now we're shipping 18 or 20 inch tablets I don't know this like at least five people who bought one probably but you know they've got these things now we're getting bigger and we're doing that you want to think about how can I manipulate these with course gestures instead of again we're tapping it buttons talking about this zone in a real sort of issue of physical comfort of ergonomics and that's important as I hope I made it clear there's also a more fundamental conceptual issue here is that buttons are a hack I don't necessarily mean that in a bad way and I think the buttons are an inspired hack both in the virtual world that many of us work in but also in the physical world you know a light switch over here to turn on a light over here it's not intuitive it has to be learned every time I check into my hotel it takes me 15 minutes to figure out how to turn that light on in the corner it's a whole different thing I know there's a light switch I understand the basic thing but I have to figure out the connection myself the wow it's a lot better and make me go into a dark room at the ladder and screwing the light bulb directly right that's this inspired hack that we came up with a hundred years ago to affect primary objects from a distance and we otherwise couldn't and that's why we came up with all these controls for the desktop to was a convenient metaphor to try to express what was happening underneath know that we had we put in this necessary layer of abstraction but understand that it is a layer of abstraction it's a workaround it's a hack it was brilliant because we needed some way to get around it when we have touch we have the opportunity again to create this illusion that we're working on this direct object itself the information you can create that physicality that we may not need buttons to do it so a point here is not that buttons are bad or evil but our since there a workaround and in some cases you may not need that work around anymore maybe worth thinking again about how should I rethink these settled patterns that I know from the desktop from mouse and keyboard to think about it for mouse let's look at some of those ways that you can kind of think differently about things windows 8 search has its religion it's a touch based OS as i'm sure you all know design from the bottom up and for touch first they haven't cracked the code yet i would say we're sort of great UX for a desktop operating system it's based on touch but a lot of great ideas they're doing some necessary work to figure out how should this work and it takes some time to get it right so I think their first version search shows and it's not a complete success I would say it by any means but while they haven't quite cracked it they do show a lot of innovative thinking about how do you have to think differently to make this work login for example login you don't do that username and password where it's great right it's terrible you know at Yahoo ten percent of their login attempts and in request for password ten percent right so as a result you can't ever remember your password we do you know the most popular password 12345 number two five four three two one yeah read password i think is number three so we have these really insecure passwords or we can or or they're so secure we could we can't even log n broken system right so you've probably seen this in some ads ads or maybe you've tried it yourself in windows 8 they have this alternative it was like Bob we've got to touch operating system oh how can that what do we think differently now about that for doing login they just let you choose a picture and then some personal meaning to yourself and you choose a pattern go on there so your login might be the circle dad connect your sisters punch mom in the face you logged in you know it turns out this is orders of magnitude more secure from a brute force security perspective then username and password and it's more personally memorable to how cancer a direct interaction allow you to think differently about what you would might consider to be sort of a settled problem another example I think it's kind of interesting adobe proto it's for folks like us export information architects people we want to make wireframes and what they do here is they use basically gestures let you sketch out your wire frames using all the established notation familiar patterns of working on paper to do this and so if i want to draw a little heading here there you go maybe a little menu bar underneath great probably use an idiot video over here and make an image to a couple of columns of text underneath boom tackle button explore it is html5 you're done right now look I don't is this is this better than doing it on vizio or OmniGraffle on the desktop using all that sort of traditional buttons and controls and information panels I don't know this is definitely the right way to do it on a touchscreen though right where's otherwise you know you're opening all these panels and trying to sort of keep it all straight not at all tuned for these fat fingers but doing this sort of lifting these patterns are so familiar from the paper world and putting them directly onto the ipad or this case it's also runs on android great interesting model do it so that takes a familiar physical model sketching and applies it kind of as is to this new digital format i think something though that's particularly interesting those is when you start to say well what about more native digital objects so we're not so going to try to pretend that this is paper now we're going to say all right well how do you manipulate digital objects as if they were physical mean you probably familiar with clear to do lat app that came up for iOS that's almost what a year ago now take a look at how it works now I want to point out first that clears a crummy to-do list app right unless you're like 12 years old and you're like three things to do all week there's no way it's been like manage everything you doing up but as sort of a prototype of how to manipulate a list with touch it sounds like you're the fundamental building blocks of a list a list item and in several lists in ways that you can kind of zoom in and out of of groups of lists it's a great example of that you know how do you insert all this site make room for it how do you cross one out or just cross it out you know in part of the thing too that I think is is interesting about this is no buttons to from the keyboard which you guys is a great place for buttons let's keep buttons on the keyboard we haven't sort of cracked that one yet either it's a for speech which is something that I'll talk about in a little bit too but one of the things that's also interesting is that sort of as you watch this guide use it particularly at the end it seemed less like he was using a tool and more like he was using an instrument right especially they even add those sort of chimes as you cross out more than one item at once sort of runs the scale talking with Phil Roo that does the developer who made this and that was by design if we wanted to sort of give this sense of playing an instrument piece when you think about all of you guys with keyboards or those of you who are musicians you don't even think about that interaction not doing that visual processing anymore you're actually sort of going with intense fluidly transformed to action right without even sort of thinking about it talk about that in a little bit that importance of muscle memory which is a difficult thing to pull off on these glass screens where there's no feedback right but there's still sort of the thing I've once you get used to gestures of one two or three finger gestures for example Mac users use gestures on your trackpad going to talk about youth wow she was a fingers you're not thinking about it anymore right so it just becomes part of how you use things that's where we need to get all of us of the community and also of users as well as designers one gets our one more example of ways if you sort of think differently about this stuff touch-up is is another iPad app and as its name suggests oh hi Charlotte as its name suggests the way that touch up photos right basically you paint on filters you know using your finger as a brush very simplest example is just putting a color on right so if we do that here and we use use the finger to sort of brush on that's great what if you want to change the brush stops solve problem right two ways to do it really have a little slider or maybe you you have a button to bring up a palette you choose your brush trouble is you've already got a brush it doesn't change size so somehow i change it so that the imprint that my finger makes on the screen doesn't match my finger size uncertainty right I don't know what's going to happen when I touch the screen anymore so touch-up doesn't let you change the brush size instead you change the canvas size by stretching it out and your finger remains the same on the screen super elegant and obvious when you see it but turns on its head this convention that we're so used to as being the obvious settled solution for the desktop so throughout there's just this thing of every time you encounter a problem you're going to solve that done that for 20 years on the desktop I think we'll wait is that really the way to do it for direct interaction doing a date range is it really to calendar Pickers or can I just stretch the date range now what happens when you can add physicality to your data objects think about it I could put this on a table as a flat thing is malleable and stretchable how would I manipulate that information give me physicality to the data objects the same time as I've been working with touch instead of the last five or six years really since since the iphone came out in particular also realize that sometimes the best touch interface is no touch at all did you know that it sort of sucks sometimes right I'm just tap stuff out and your autocorrect is screwing you up and it's just sometimes these things are not great and again it for the spine to own interactions things like typing that we get fouled up much better at these more coarse interactions right so one of the interesting things is to understand that touch is just one of an emerging set of interactions it kind of got here first before the rest if you look at what's happening now with the emergence of speech Apple still call Siri beta and it feels like it right you wouldn't want to run a nuclear power plant with Syria out there or connect you know again part of its its charm if you if you play this connect on an xbox is that it's an exam it's not like you're snowboarding God you know it's like part of the game is figuring out how to game the kinect but you can see how good it's becoming things like facial recognition camera vision all these ways that we interact as humans touch sound speech facial recognition and natural gesture the machines are beginning to understand and I was up to us as designers to figure out how do we put all those things together and frankly the combinations are especially exciting we tend to think about things up oh this is a touch interface oh this is a keyboard and mouse interface this is speech interface the truth is is that we're not going to have the luxury to keep thinking that way that's what Windows 8 is trying to do and that's why it's so hard they're tackling this problem ahead of us which is how do you create an interface not only that can go out to any output sort of a problem that anybody's working on the web is sort of thinking months ago with all the screen size headaches but also in the input windows 8 designed to work with keyboard mouse touch stylus speech natural gesture Wow hard problem but if these combinations again are especially exciting I think especially a speech and gesture put together which is gonna be sort of necessary you have to talk to the device to let it know you're about to wave to it or wave to it to let it know that you're not to talk to it guys when you put speech and gesture together you know what you get yeah right it spells you guys that's what we're working with that and you know it and I mean that you really can create the sense of magic this is my friend we're all calm he looks a little tired here because he's just been out all night at a hackathon her always does things right but if you did a hackathon and calm on a boat you can see his wine glasses back here so he's he's doing all right so he did this hat it came up with no I want to point out you did this overnight half drunk on wine boat is it using these consumer products that many of us already have in our house he had bought an iPhone and connect a monitor and a match it so this stuff is just looking it over night got this if you think about it it's actually while it seems incredibly magical what he did it's sort of simple you taught to connect this gesture take a screenshot one touch to the phone move the screen shot over here so he can do the same reason look I've got it right here right I got the rivers right here anytime interaction is actually super simple the magic of it is combining gesture touch and speech right this stuff is again you know this is the same sort of sci-fi future this is all right now we have one thing i want to show you here is leap motion which is this little gadget that sits in front of a computer ception release next month already in the hands of 12,000 developers basically a incredibly sophisticated connect web centimeter of sensitivities you see here the attack fingertip motion or our pencil motion this is something that again the device itself is going to be around 100 bucks you put it in front of your computer and boom your app is Minority Report right also asus is the what I'm sort of the big laptop makers kind of commodity laptop makers has committed to putting these things into their laptops this year this is the kind of stuff it's like wow this is shipping it's like real stuff it's like we're gonna have these things in here Microsoft is already experimenting with building connecting two displays and into laptops so it's sort of this kind of gestural I can supplement not a replacement for using a keyboard and mouse but a secondary combination something we're going to have to prepare for one of the exciting things is is that this means that we don't actually interact with the screen itself necessarily so again sometimes the best touch interface is no touch for the first time you guys Sasha's am right whoa it listen to the song you recognize the song that's a great that's a great touch screen app it doesn't use the touchscreen I'll show you another example something similar a table drum is an app that was put together from some friends of mine and sweetie it's a sort of they called it augmented audio the thinking of an augmented reality is I'm is often sort of visual in this case it uses the microphone it's basic onboard sensor on every smartphone to listen in to the environment and create a whole new kind of experience it is a drum machine at tons of these things in the app stores little things you pick out some drums need to tap on I'll write them on the touch screen you can do that with this but the really interesting thing is the way that it uses the microphone to listen to you tapping on the table nearby and then it you've got the drum set the logic is happening here but it's really just a speaker as far as the interactions and say look doesn't know the glass sound let's teach it would choose this symbol put in the training mode all right ready the weights done no it's so interesting to sensors basic onboard sensors are in every one of your phones to move the interaction all of the screen and into the environment around these are have a much more sort of natural experience we're starting to design not only for touch but for sensors right and sensors that are that come for free and all these consumer devices that all of us have now things get even wackier when you start to think about actual sort of custom sensors and how your phone your tablet or other devices can interact with these custom sensors here's one sort of literally Mickey Mouse example that the Walt Disney research group came out with late last year the colorful tanika's interacted us mechanicus interactive is a new interactive plant technology oh yes it requires no plant instrumentation a simple electrode placed in the soil turns any plant into an expressive multi-touch gesture sensitive controller plant you on you guys you know you've been waiting the ready oh yes that feels good a little more of that place well you know I have no idea what they're using this for I think maybe the carcass maybe hetero they're gonna do something the parts that are oh but there Boyd is like wow you can turn anything into a sensor anything now can be something that you use to interact because it's so trivially inexpensive to put a sensor and an internet connection on anything that which means that the physical world is getting more digital just as you know the digital world is getting more physical because of these things one lasts or wacky example and we'll get back to touch in Switzerland naturally farmers are trialing these cows sensors so these things when we put the sensor in the cow and detects when the cow is in heat and then it text the farmer sending texts when you're in heat I have friends who do that stare right of course because it's Switzerland the text can be an Italian or French or German you guys how love knows no language boundaries based in Switzerland so we get the internet of things we've got the Internet of cows now we can get put sensors anywhere and talk two things the physical world in ways that we haven't thought of that so that the cows and this thing can actually act on our behalf all right so one of the interesting things about all this stuff is well now we're getting really far afield from our traditional desktop graphic interfaces right we're again we're so used to having things clearly labeled in buttons but you know this is a much more amorphous kind of thing how do I know that I can gesture at my TV screen to take a screenshot and in touch my phone these things aren't advertised what I want to do just sort of close out is to talk about how do we teach this stuff how do we make it more discoverable first you guys doing okay you look good all right so that's that's the challenge here is how do you find out about these gestures unlabeled invisible things right and typically we rely on visual clues or past experience which is why things like the pinch gesture for maps and proposed often has to be taught quickly but then we get it because it feels physical similarly it'll work for the desktop too so that new users for example to iOS Maps app will quickly figure out the pic and double tap on the map to zoom not because they would ever do that to a physical map because they would know it already from the desktop google maps double-click zooms in what they will never figure out it's a two finger single touch will zoom out has no correlation to anything right so for these abstract gestures we need to help people understand them but you'll understand that with a little bit of help people will learn to work your interface side I'm seeing even if the gestures are are invisible so we do this all the time if not using invisible interfaces and at least you know interfaces that are in the dark all right anybody still wake up to this sound one of these things usually one person on your heart attack to my sort of plan alarm clock sound all right well they use your phone and you said that's not me that's something else let me use your phone or one of these things you know it's like wow you can turn this off figure out in the dark in your sleep right look about invisible controlled unconscious use of these things right you wake up two hours later and you realize you turned it off I'd you know while you're asleep so you know this is something that again sort of gets to this muscle memory thing that I was talking about something you know so well it becomes literally unconscious use know and it sort of goes to again as I mention all of you with keyboards it's a study that was done years ago seventies or eighties of professional typists which I guess all of us are now uh asking them you know write down the order of the keys on the keyboard most one couldn't do it these are people who could type I don't know what you wanted words a minute or whatever a lot of words Minotaur and yet they had no idea what was on the keys underneath their hands but because they had gone from visual processing to this muscle memory this thing's became instinct tool being able to use this stuff super rapidly and that's where we need to be able to get people in terms of using at least basic gestures on these screens the trouble with these examples that are right is it with the clock get to learn where the snooze button was before we could turn it off right with the keys we need to do hunting pack before we could do that touch typing it's so much faster so nearly everything we now has to be taught or learner observed you can't just jump straight to it before we get to helping people find invisible gestures want to talk a little about about the importance of visual cues this is the OCD chef cutting board but did you know if you are someone who really wants your alamat or julienne cuts to be just so right this is what you use you haven't you haven't quite got it dead so just muscle memory where you can just do it your eyes hat you need this right is a cheat sheet we have tons of these things in our lives they could be your own personal system of reference or post it notes that you surround Roger around your desk or it could be a more often systems that are imposed on us their social systems of reminders and cues that aren't in our own control either as consumers or as designers great Don Norman in his book looking with complexity talks about salt and pepper shakers it's this one its salt nobody means itself you've got one salt pepper whoops pepper there's no idea yeah so God make the point that doesn't matter which one is actually correct all that matters is what the guy who filled them thinks which is why when you go into a restaurant you see this you don't just dump it off right just you know that not everybody gets that you know you understand that this is a social convention that is not evenly understood so you test it right uncertainty all you I conventions are social conventions some of them are better understood than others right now with gestures or in bed we're in that uneven state where we're getting emerging conventions but not everyone knows what they mean yet what does a two-finger swipe me what changes according to application because it's not fixed yet we're still trying to figure out figure this out typically though what we do to try to give confidence to people who use any interface is to you know use labels do you get something like this which helps if your if your language you know pepper and salt starts with P and s pretty good pretty good start you won't be able to confuse if this is upside down you're like I don't know what D is are you getting there you know it's better than this what this yeah right salt and pepper you want it just you grab its content it's the label here right not any sort of visual processing I understand it more instinctively this is still a social convention because you'd be surprised if this was sugar them but it's still something that's much clearer because you're letting now the content speak for it who needs a control we need to label when you have to content itself so one of the great things about touch interfaces is they really encourage and allow direct interaction with the content itself you do this obviously with with mouse and keyboard things too but this case you're actually touching the thing you're saying I want that and it sort of does something that triggers a little transistor in our heads so don't touch a button touch the content you want to work with photo galleries are great about there's really dense interface and yet it's almost all content which photo do you want to see this one here you go this this one of the things exciting about this what are you saying haunted doesn't control and thinking about that as being a primary interface is that you know if you go back to what Marshall McLuhan said years ago the medium is the message well now to a place where the message can be the media the information can be the interface content can be in control use your own alliteration here if you want it 23 letters to go stuff so I think we may finally beginning to a place where we going to do that but that's something that again sort of helps to advertise invite a touch right until the content is front and center what about again there's more abstract actions well a lot of times this is where we start write instructions if you're going to use the popular science magazine app on iPad this is where you have to start screens of every gesture that you can use to read the magazine you know incredibly detailed you know New Yorker does something similar all these apps do a lot of tons of these apps do this where they actually want to use every aspect the application before you've even understood what it can do for you from a basic level it's not just premature it actually seemed makes it seem more complicated to use may all i want to do is read a magazine how hard is it do i have to read all these instructions koi been used to be the design director at New York Times digital sort of books and fun at this you know what if we did this for actual print magazines this is what you get like pages stack on the left side display your reading history and over here pages on the right that's your unread content because I don't know if you know about this save feature that magazines happen but look yes this is the most elegant interface that we've yet to come up with as human beings and look how complicated it is when we do this to it so this is something that I think is becoming a convention unfortunately for a lot of touch screen apps its overlay everything you can do and I don't think that's helping people very well norwegian TV i but you didn't see that coming have his comedy sketch many years ago called the medieval help desk what's going on here we can come in the middle is that the book has just been invented and scroll readers everywhere are in a panic right this monk is called the helpdesk and for a little assistance he thinks he's got it i'll read out read this out loud here because you can't see it in the back he thinks he's got it and so he sends upon his ways again I get it now it takes a while I wouldn't use two Scrolls to get to these books all right so everything's all right get one last thing you know what let's let's run through it before you go so I open it like this and then how do you say you turn the page okay turn the page back and forth and when I finished I just closed it sorry see ya yeah all right bye but can't believe it it's like this again I can't open it you want the wrong side up what do you need the wrong side little bleep from the other side so it matters which way you open it yeah open it from this side oh I see all right look it read the manual manual yeah comes with this guy for users here it is that's it problem yeah look even if the guy could figure out how to open it and read it you know he would there's nobody recent all of us in this room the tools we use every day we have only in complete exercise exporters Annette you complain about every user / I with these instructions but none of us do it no why because it seems like it's a diversion from what we're trying to get done to get work done or play a game or whatever it is we're doing we want to get right to it that's my monopoly is such a miserable long game didn't let us read the instruction make up our own rules that's forever you read the instructions you play by the rules it's really fast it's like a really mean-spirited awful game but it is quickly at phase right so nobody reads the manual nobody read the instructions so since we can't get people to rebook people like to watch TV right let's maybe watching make him watch out screencaps maybe with this guy so al gore wrote a book called our choice it was adapted into really wonderful ebook lots of great thoughtful interactions by a company called push pop press which is bought by Facebook so we'll never hear from them again but most of it really self-explanatory but out of no chances so when you launch the app you have to watch this video people say outdoors a little wooden I personally don't get it you decide this app it divides text and images as well as video interactive infographic interactive infographics audio commentary all to help bring this important message Columbus you can browse through the different chapters by swiping through the visual table of contents or browse through a chapter by scrolling through the pages at the bottom to start reading use two fingers just pop up a joke to go back to the table of contents just inch the page then you'll find images movies and interactive info interactive infographics basic surroundings you can pick anything up using two fingers and pop it up cap the globe in the corner to see a photos location on an interactive map interactive intention in turn of the page you were in movies work the same way zombie madness unfold who reveal the other half some photos have audio commentary industrial agriculture is a hand icon marks are choices interactive info interactive infographics use your finger to explore the day I hope that you will enjoy this new breed meet you out he's really suck the joy out of the first two minutes but wow what great interaction really direct interactions right on the content itself very few sort of traditional buttons or controls most of its really pretty obvious few tricky things right I'm folding out the pictures with two fingers and so even the most elegant interfaces and some of the edges are going to need some help some instruction but you don't necessarily have to do it with a manual or the video of the guy who invented the internet to be able to do it I think it's worth pointing out first so that nature itself doesn't have instructions it doesn't mean that nature is an easy interface all of us in this room spent the first two or three years of our life as complete morons covered in bruises and scratches trying to figure out this really difficult interface and being surprised by a constantly but out of the time we sort of get better at it you guys look like you're doing pretty well with this interface that we call a major which means that all of us as well as all of our users and all of our customers have this incredible base of knowledge and experience to draw on for how the physical world works that we can then apply to our these digital interfaces that have the illusion of physicality you know Apple emphasizes this a lot in a two-man interface guidelines make it realistic and its so-called skeuomorphism which is something that becomes somewhat aesthetically I would say controversial in the design community make it look like a regular device and then people will understand how to use it and you'll also get this emotional connotation comes from whatever you think of it as an aesthetic approach it's a sound teaching approach right and so the problem is that it's more than just sort of making it look pretty they can't look like a real object in Apple's own applications sure to show some of the difficulty with this calendar app for iPad lovely date book and one of the great things about the design is because it's this page book you know how to use it right you can just sort of swype to turn the page huh now for the first 18 months of the app nothing happened took an 18 months that's how you turn the page this desktop style interaction you guys that you know so you have to hit that to make a go your interfaces you know gives people hints about how to use it and this was a lot right this is misdirection actually much worse for the contacts a person this which remains this way to this time are you going to turn this page you start deleting data eggs wow you know your interlace metaphor again suggests how you use the app this is dangerous misdirection if you're going to go this way you got to embrace that metaphor if you can't make it act like the thing don't try to make it look like the peg you know it's got if it looks like a book make it act like a book otherwise you know yours don't make me sort of hit these buttons an alien you get things like magazine apps they're basically just sort of glorified PDFs so very true to the original experience is linear swipe swipe swipe thing a lot of times a table of contents is hard to find in these things right you're almost reduced to this really a linear swiping experience easy to use except that it takes away the great advantage of the digital medium which is random access to content I'm gonna go straight to this thing good interesting so there there as you embrace these kind of realistic interfaces if you go that route don't forget to enable what digital media can do best which is to unshackle us from these linear reading experiences interesting example of a way to get around this is from The Sydney Morning Herald newspaper app works like you'd expect you can sort of browse it like a newspaper sort of very familiar of physical experience and this these uh page indicators show your progress through that something a little unusual is it can actually touch those indicators to show all the headlines on the page and as you sort of swipe across you can quickly scan everything in the entire edition and then jump straight to that issue so it sort of has the best of both worlds in a way so lots of great opportunities especially with tablets like this which suggests things of the desktop of bones of physical experiences of traditional software of the web trick is that we can't mix the metaphors of all those things you have to be very certain cautious about how we do it and in fact I would say that we need to model our interfaces to help people learn the same way that they learn in the real world right from our earliest days we rely on physical hints about how something works and then verify it through the feedback that the thing gives to us you watch out toddlers using iPad you guys have seen toddlers use an iPad right at least on YouTube and cats are using iPad cats and toddlers are figuring out how to use this thing your customers can too in fact I would say you know it's amazing how quickly toddlers can figure this out zipping and swiping and pinching through through these interfaces because they're using it the way that they could come to understand the physical world and oddly right sort start to understand the physical world where they learn it on the digital room they might not get your multi-level menu navigation but neither will your adult users so that's okay but if you if you will get any physical metaphor so it's really worth asking would my five-year-old niece understand how to use this control necessarily what the app is for but the control itself let's figure this out I'm serious about this get a four-year-old or a five-year-old and make her your beta tester because they're better at this than we are so haven't been poisoned by 30 years of desktop interface interactions now thinking about that from a pure and naive point of view to follow the tablets they're better at this along those lines and to wrap this up here this is a couple of things here versus your homework play more video games game designers are at the lead as far as I'm concerned of all interaction design for teaching and interface mini-games you know you don't even know what your object is when you start let alone what your capabilities or what your obstacles are going to be in the way you find out is not to read the manual or to watch a video the game shows you draws you through gives you q's gives you challenges shows your new information at the moments that you need it let's look quickly the way they do it and I'll wrap this up where we can go to some questions among other things games use these three techniques to teach an interface coaching leveling up and power-ups every modern theory of learning emphasizes the importance of active participation active discovery supplemented by coaching and mentoring so let's start there with coaching simple demonstrations right prompts to tell you what to do this is the game riding along with the player the application or that website riding along with the user we learn by doing right we learn best in the moment tell me people how to do something not nearly as effective as coaching them through it especially again when we're thinking about these physical interactions teaching an instrument tennis serve how to throw a football all these things are these physical actions that's something you can read about so that someone has to show you and then you practice practice practice demonstration and practice so that's what you see in a lot of games and I've had game called dead space very first screen of the game teaching you how to move little text that explains it you probably don't even need that there's this overlap and it just waits nothing's gonna happen it's all right now we're going to come in and blast you away whatever happens in games like this instead of just waits and once you do it you move the guy around the overlay goes away you did it your first interaction is a success and that by the way I don't need to coach you anymore so many times in applications we keep coaching people after they've already learned it pay attention at a learning layer to your application so to see where people have gone so that you can give them appropriate information so these simple temporary overlays and call them coach Mark's often on the web or Thanks so here in Google Drive showing you about a new feature a lot of websites do this yours can too some of you might be squirming a little bit because this might seem a little bit I don't know like a certain interbase block block named cookie was never helpful right always a distraction pop it the most inconvenient times the clippy concept is the execution right and especially the in main content you know only offered to do the dopiest things you write a letter letter done right though this assistant feature like you see in gmail or facebook or so many games can be really helpful to give information at appropriate times it can be subtle and respectful and just a little tweak a little animation composes the USA Today app from right after the app store open back in two thousand eight and they have the snap thing where you can navigate the sections of the newspaper and it was this dial yeah this was a commission at facebook used at the times i navigate they found that people didn't see it or didn't recognize that it was an interactive and if not sections were missing so they added this little thing where every time you visited this front page it would zip in from the right it moves maybe i can move it to instantly got much better pick up on it thing is that can be really annoying if it keeps happening every time you visit your favorite news app read and so basically they would say it's one we go thanks for the phones only one user on it basically for the most part once you've used it yourself wait don't show the animation anymore the users figured it out just pay attention to what the person is used and done and sort of change your interface appropriately you got to provide visual cues for these hidden things you can't just count on people to find the next a little bit about this is is leveling up an important component of coaching is that you don't teach everything all at once we learn best by getting it in doses so you ease players into a game you use your users into an application introducing one element at a time encouraging people to master that before you teach them the next step let me block access to more advanced features so you don't necessarily overwhelm them with education about them all at once learn it all a little bit at a time we're most motivated to learn a new skill at the moment that we discovered that we need it right like I don't know when you're about to get your butt kicked by a guy with a giant sword right so this is infinity blade another ipad game incredibly complex battle system but they make it easy by breaking it down and teaching you one step at a time and they do it by freezing the action right here and incredibly give me moments and say I don't know you want to learn to block yeah I really am motivated shows you you do it and again your first interactions success do it over and over again but I'm not a Dodge yes and so it stops the action shows you how to do it and makes you doing my demonstration and practice os10 did this by the way when they first introduced under of kids remember this in ores 10 lion it changed the way these scroll so turn gravity upside down here to swipe or move your mouse in a whole different direction to move big deal so when you did the installation they told you that that it just happened and is it here scroll in this little window that's how you got to the continue button make you do it demonstrate and practice right that's it alright so the last thing I'll show and then I'll shut up and let you go to ask some questions if you like is power-ups and power-ups are sort of like the keyboard shortcuts of video games the same way that the gestures are the keyboard shortcuts of touch think about the concept about evolving from beginner to expert expertise is where the fancy moves coming the power-ups so you know in video games like you this you're the super mushrooms and Super Mario Brothers it gave me some kind of superpower the turbo boost your game giving you a shortcut or some other advantage but it's also a reward right it's an achievement it's a marker there's a thrill to bat a lot of loose talk about gamification of here's some lousy badge for going to the dry cleaner more than anyone else has an empty meaning right what's the real thrill of a game getting better at it wow I'm good at this right if you think about that this isn't just for games this is in our own work say you know whatever your use visio or illustrator and then I got every day somebody comes over looks every show me are you doing it that way did you know about this right satisfaction of learning something new of sort of going to the next level of mastery that's something that your application can do and that you can also use those teaching moments to deliver that so last example Twitter about a year ago change their iphone and android app completely push some really popular features down a level so direct messages in accounts which used to be available from anywhere in the app but then they put them here in the me tab they knew that that would upset some users who really use those features a lot so they added some gestures to make it easy to get to those so if starting at this me tab where these things are located you can swipe this way to get to account switching slight the back get to direct messages up here like that not advertised anywhere no instructions yet find it yourself by accident or have somebody tell you about an easter egg right this could have been a powerup moment there's utility and having people do it learn the slow way reinforces the mental model of where this stuff lives in the application direct messages cops searching there under the me tab but after the tenth time that you hit that direct messages thing this is what I wish they did show you how to do it and then make you do it demonstration and practice right observe how people are using the application when they get to a little level give them a power this is this is combining coaching leveling up and power up all and once and there's a satisfaction who uses the Twitter app who knew about this whose sight to know about it right power up there you go the fact that we need to do this though shows that we're still in the salt and pepper stage of developing these gesture conventions which means you guys it's really important to be generous generous with our users and explaining how you use this stuff but also living with each other what's working for you guys how can we as community arrive at standards that actually make sense so having meetups like this where we can talk about it it's important but also reaching out to your fellow developers and designers hey you I see you did this interesting thing how is that working at we need to talk about this stuff one of the really exciting things about this is that we are part of creating this new way of working with information this doesn't come along a lot you can come off along very often centuries of time go before we have new ways to manipulate or share information you guys were in the middle of it all most exciting times in the history of Technology possibly in the history of culture so you guys embrace that well make something amazing for serious thanks a lot so genius actually that just seems quite genius what percent of apps out there and would you say nearly every game and then nearly nobody else you know I mean very few kinds of things and even some of the really forward-looking things you look at clear which again I think is a great prototype for how do you manipulated things by touch and they basically kind of Pumped it a little bit on that they put some instructions kind of us to do is sort of to try any things which sort of makes sense this is because it thanks so much more time really that's that much more time and develop i think i think this air is that effort we don't yet have frameworks for doing this games do right games have to sort of these engines and these frameworks built in but you can kind of do some of these things if partly you know a lot of it is frankly that as always education tends to fall upon us build the bank first and then we can tell people how to use it and think in general you see this pattern over and over again that in the first years of any new technology we're just trying to figure out how to make it work you know and then the polish comes in then we can sort of figure out how to how to do things and i think that that here was very much seeing that but you know i think seriously you know i'm not a huge gamer myself but it benefits so much every time i do play games and like wow right every enterprise software financial management application should do what this game is doing because it makes it more pleasurable easier to figure out how to use contextual education yes I know bro you know the question is that the w3c and other standards bodies many industries have standards bodies right to sort of say this is how things should work and the question is do we have anything like that for for gestures and for interfaces like this you know we don't really and in one case I'm not sure that I think we will eventually meet some stuff like that at least sort of suggest some broad patterns from some sort of central kind of thing that we can all use as a resource trouble of standards bodies is that they are graded in abatement know so standards bodies are great at saying innovation has happened and we believe this is the way to do it not always the best ways that they choose but that's that's typically it so standards bodies and generally broad communities often aren't where the innovation happens and for better or worse that's often in private playgrounds so you see a lot of innovation in this stuff happening right now with players like Apple for example some that early stuff I was saying about how device is interacting with each other and some of the far-out stuff it's already arriving now you're seeing that stuff happen in private playgrounds because that's typically where innovation happens before it can become sort of standardized there are risks to that we were sort of talking about this before there are intellectual property risks if you know that companies are very patent happy these days and there's a real danger to companies patenting gestures like Twitter is as has patented the pull down to refresh gesture or typic they're actually to pull down and let go gesture so Apple when they introduce that into their mail app just did pull down sort of sidestep that patent you know this is dangerous when we're sort of trying to figure out what's the way that all of us will communicate with machines and touch screens now the idea that they can be that they will be patented or owned by one company potentially litigated is not a good time at a huge fan of the patent system when it comes to software and interface design and Joan else ever happy thoughts about business award it's private yeah how to make a lot of navigation menus revengeance Barry yeah yeah well I wouldn't I would I wouldn't say that any of this is an enemy of agile process particularly right if this is more priorities of the organization and unfortunately the some notion for the people who fund and create enterprise software and suffer supposed to be used by their employees that people are supposed to be taking care of it or taking care of them just sort of say you know what you don't need a good user experience this is this can be miserable user experience and I don't think that that makes good sense but I think that one of the exciting things right now is that through the history of computing it's always been Enterprise pushing out into the consumer world so we had these business machines you know from IBM and the early days pushing out from the corporate world into the consumer market and now we're seeing the reverse where the consumer market is now entering the enterprise where it's it's our experiences with phones that we're bringing up our phones into the office and in our tablets and so I think that we have an opportunity to start to see some typically consumer experiences start to hit enterprise software to as that becomes an expectation so I think that there is a place and ought to be a place in the new price software for this stuff because everybody deserves a great user experience and I think that when it comes to touch screens that these kinds of gestures are more efficient easier and frankly these are more pleasurable and intuitive to use and so it does take the thing is they read this period but we have to think about this stuff a lot more it's hard to get you know big screen interfaces into the small screen is to think about them differently after think about the interest interactions for touch and you know naturally internal projects often don't get the financial or schedule love that but they probably deserve so keep fighting the good fight is all I all I can suggest you're doing the right thing yeah multicultural world Yeah right language you're right don't you okay in some in some places yeah that's right what sort of like minds yeah I mean it's great question it you know I think that for a lot of this stuff because it's such basic physical interaction it's sort of like how would you manipulate this then it has less of the cultural overtones that maybe you're talking about right now and I'm really sort of saying wow you know what does a full finger full hand press do but any sort of shortcuts I'm not sure that at least some of the basic building block work that we need to do some of the cultural risks that you're talking about are really at the forefront but I think that some of the you know before some of this more physical metaphor came and we were looking at touchscreen is particularly stylist experiences that was much more of sort of drawing figures and letters and things like that sort of made something happen and now that we've moved too much more of a physical interface rather than sort of a semantic interface a lot of those problems go by the wayside and not as concerned about things yeah he's going to you too to be able to turn off these gestures low because my teeth of the provincial yeah blow something up yeah I got away yep now I got to get back to it and how do you turn this feature off that is I mean yes we need to think creatively some people yeah yeah but so the question is you know how do you if we're doing everything with touch you know sometimes you're going to like blow the thing up by touching you know how can we sort of turn it off or make it less sensitive to this so that you know we don't run into that if something happens with this actually with Apple I guess iOS 5 I think they introduce some gestures to be able to work at the operating system level and so a four or five fingers swipe moves you among applications three or four finger swipe down to get to your you know last applications trouble is is that they did this in the canvas itself taking these great gestures or five finger touch gestures and using them for the operating system when you look at a lot of other touch based operating systems including windows a webos blackberry they use the edges we use the frame so you sort of have to drag from the frame sort of trigger these operating system things which is interesting because it sort of like oh it leaves the main canvas for the application but also it uses the metaphor of the frame of the device itself that matches this sort of metaphorical frame of the operating system for the for the application so i think that there are sort of some things in terms of this of how we think about competition with applications and operating systems that have to be careful about i think you're also right there are sort of things that something need to make it easy to be sort of like actually don't want you to be so sensitive right now it says to me all the time but but also that's the opportunity I think combining some of these inputs right you know that you can sort of give it a touch and in tell it to do something and it sort of you know open its so I think there are some things that as we start to combine the inputs and start designing for sensors as well as for touch that there's some opportunities that feel like there's somebody got in the back our top user can you ever envision something like a piano being improved on with touch instead of instrument using such ever approved that paradigm it's interesting right so the question is you know we've got these great physical objects that are doing sort of doing all this stuff can something as sort of you know week in a sense is a glass screen that looks like a piano ever replace that and you know and I think that that we we're always in a rush right sort of improve on the physical with the digital I think that a lot of us who are book lovers you know we'd read that even as we start reading our kindles you know recently go this is what a shame if this is happening to the book we do lose something right I mean it's something of the residents and the experience of playing a physical instrument like a like a piano or guitar or something can't ever be the same it's using a digital device so there are some things that are matters of convenience it's like well at least we don't have to carry all those books around with us we've got this sort of very light convenience that many of us are saying well that's that's an acceptable alternative and you know for some musicians the ability to sort of be able to compose on something without actually carrying a piano around is it convenience but i doubt that most of those musicians would prefer to replace the piano entirely with it so i think that what I'm suggesting is not that the digital should replace the physical but really that the physical should inform the way that we design digital interfaces so at the same time you know as I mentioned earlier the digital world has been a more physical by the fact that we can actually carry these interfaces around with us and manipulate them with sort of folk physicality the physical world is also becoming more digital so what's interesting is well what happens when I can talk to that physical piano from any device that there becomes manipulation with that physical world what are the opportunity is there where we can have real player pianos we're at a distance example sort of riffing yeah so most of what we're talking about is in the realm native apps yeah and so much of what probably many of us still do is in the realm of the browser yeah where do you see the browser do you see it evolving to this more of the capabilities or do you think that there's really a paradigm shift yeah it so the question is well isn't all this stuff sort of the domain of native applications what about the web you know where does the web sit with all this let's get me crucially important no matter what to knit all these things together there's a lot of controversy on those that do we do native apps or do we do web apps how do we do this important considerations for the folks who build these things have to pay for them maintain them users don't care you know as long as the thing works the really important thing is that all of these things are converging in the cloud alright so the web is going to remain central first in terms the interface question that you're giving at the device api's will come to the browser no I mean that will happen and I would say a little bit shame on the browser makers for not making it happen faster because you do see some technologies like PhoneGap which are sort of like that would essentially like you build your own custom browser with all the features that the browser should have had in the first place it will happen there are some issues to get that you know which is just sort of like things like well should the browser have access to your contacts the way that you know the way that native apps can you know what does it mean that the web browser can easily get at the camera but many Android apps can get it to camp and the Android browsers can't get at the Cameron app to record photo and video so you can create a camera app in the web now for many Android so this is happening the other thing that I would point out though is that you can be a web partisan without necessarily being a browser partisan and we're seeing the web start to infuse all kinds of places showing up embedded in in ebooks showing up embedded in native apps that windows 8 itself you can write entirely native applications using HTML CSS and JavaScript compiles to JavaScript right down to native code and so the point is is that web technologies are going everywhere and that it's really more about the web going into applications more than application going into the web and so you know I think that that the web and web technologies are going to be as strong as ever but I think that the idea that as soon as we can sort of release ourselves from thinking that everything has to happen in a browser give us the web so much more power and I think it will have access to all these different sensors and abilities right now web technologies are really crummy about sort of dealing with touch really bad touch the poor really primitive touch the poor it's really difficult to code gestures that aren't a touch or swipe which is good but let's start there but you know doing something like though good luck with that three finger twist you know it's really difficult to program so it's got to improve but we're seeing things like in windows 8 where they added a bunch of custom JavaScript to get at things like a rotate or a pinch so you can have those sort of Dom level access to say Oh is there a pinch on this element in the same way that we have a tap or click on something that will come too so the web is lagging native apps and again you see this innovation happens often in proprietary systems before it goes to sort of more general standard systems and the web is always a story of that when we're always waiting right it happens eventually look at html5 finally yeah how are we doing more any other questions yeah recently saw like a gel into the screen part of this since there's some haptic feedback in the screen itself yeah that seems like a sort of going in all right squishy screens there you go no i mean i think that's right i mean one of the things that makes it difficult to use an ipad for example for typing is you don't get anything back right and so they are as you say working on some of these things these sort of things that can raise lenders things that can give you some some physical cues that will make this stuff easier to use and you know but i think also you know sort of go backs is also some of these ideas of where we can put sensors and just about anything and we're able to manipulate or get information from the world itself or we may not even need to manipulate a screen directly anymore talk to a lot of medical device manufacturers where they're really doing some interesting things medical device manufacturers are great at creating sensors terrible and creating user interfaces right so the guide like you're gonna have to build these computers to sort of like run or make sense the sensors but what's really interesting is now that you're like well win second everybody's got a great computer with a great user interface in their pocket now what if we just create a sensor that can talk to these things so for example you're seeing really amazing stuff is this company called Clodius it's basically made a pill that tells when it they can tell you when it's been taken so it's basically this little sensor that's about the size of a grain of sand has the same stuff that they would have in a vitamin magnesium copper is when it hits your stomach acid it turns into a battery strong enough to send out a little signal that you know people who are taking us have where this little patch that has a Bluetooth transmitter relays the signal here to your tablet and boom to your doctor Wow right other things you know there's a medical device manufacturers talking to that come up with this sensor to help people who have very advanced pulmonary illness really sick hearts in this case you know it's good one doing well at all anyway but you can tell when they're about to have a turn for the worse by changing their blood pressure this little sensor or something that goes into the body embed it into an artery near the heart and the trouble is you know detects the blood pressure thing but how do you how do you get the information from it it's in your body right how do you do it you touch your skin you just touch it a tablet it downloads the information through your skin tricorder stuff right I mean it's just like sit where it's just like we're getting to this thing now where you know I don't have to type or tap or say anything up into this thing it just knows from my body what's happening some of us can get a little creepy right but I mean it's but this is the nature of where we are now this is not coming down the road this is not this will be possible five years now from now this is inexpensive and happening now a lot of this stuff is happening with the sensors that you guys already have and the phones you're carrying in your pocket now this is like basic consumer technology now and so I would say that the user interface field just hasn't quite caught up with it yet of what can you do with all this stuff it's just now available you know when you see sort of things that toys and games lead the way and experimenting with this stuff things like that drum app right it's a little toy but wait a second this is like figure out how to move the interaction off of the screen just using the microphone to the table around them that's the stuff that I think is really exciting sort of where I'm turning my attention to now that sort of more basic mobile practices sort of becoming absorbed into into companies into regular practice and of my sort of thinking somebody who tries to help people look ahead is that's it because that's what we're going to be all dealing with in the next Oh in theory a new book is in the works yeah my editor thinks a new book is in the works working on yeah so we'll see everyone we hope in a couple of weeks I guess it's a month for a number three thank you very much for coming good I don't wanna thank you

1 Comment

One Reply to “Mobile UX Design Series: Designing for Touch (Part 2 of 4) – Josh Clark”

  1. Elisabeth Marina says:

    Create My Free App is a unique platform that have many mobile application features to promote your business free of cost from

Leave a Comment

Your email address will not be published. Required fields are marked *