A Practice-Led Account of the Conceptual Evolution of UX Knowledge

July 24, 2019 posted by

okay good morning my name is Colin grant I'm presenting this paper on behalf of the ux pedagogy and practice lab at Purdue University and the lead author yubico you both completed this work while he was a postdoc with me at Purdue and now he's an assistant professor at Florida State unfortunately was unable to make it so the conference this year in this talk I have a few goals but overall we're going to frame UX designers online communication as a valuable means and resource for recognizing and characterizing UX knowledge as it evolves over time with a lot of potential uptakes both for UX research and UX education so as I think has already been framed up pretty well this morning HCI and more specifically in this context UX is an emerging transdisciplinary context and you know we might actually dispute either one of those terms but I'm going to give you a little bit of evidence to maybe character give you a sense of why we characterize things that way so according to Google Trends search interest in both user experience design has been growing steadily over the past 14 years you can see quite an uptake as this name has taken hold within industry even if it really reflects some flavors of HCI that had been in industry for quite some time many corporations are discussing ways of adopting UX designed not only into their products but also into their over organizational practices and services and while UX design looks promising as a career in occupation there are still numerous challenges to be addressed on the organizational and individual level we've also seen rapid adoption and expansion of program offerings I'm just listing a few from the United States specifically in user experience design and increasingly at the undergraduate level many universities are building new UX or ixd focused programs but there are a number of challenges in creating these programs and sustaining them especially in parallel with each other different host disciplines depending on where these programs are arising from computer science from psychology from information science from design all bring with them different domain knowledge different epistemological and ontological framings of their work and a different conceptual vocabulary and this is increasingly a concern as you start to see all these disciplines funneling down into the same types of industry positions there's also a little consensus over what a body of disciplinary knowledge might be practitioners are constantly taking a new knowledge and methods from related disciplines as they do their work and so as educators I would argue we need to form at least some consensus over like what is the larger conceptual body of knowledge that we should be teaching students and how should we sort of contribute to this larger discourse specifically around user experience design and I'm not suggesting here that we all become the same but that we start to identify that we do in fact have a shared vocabulary in some sense there's also this persistent gap between research and practice and this is not true only in HCI of course but we certainly feel it pretty substantially in this space because of this rapid evolution on the industry side and a different kind of evolution that's been happening on the on the research side so simply put what academics do and understand might not be the same as industry practices even if there is some shared vocabulary and so what happens is when undergraduates or graduates leave our educational context they may adapt well to industry practices but they may not depending on how well things are aligned or not so this project fits into a larger research initiative that I've been working with for several years now and you buzz really been instrumental in some of this early work as well and looking at how do design practitioners develop and maintain their occupation occupational competency for some of these individuals they've never had formal design training or HCI training at all for others they have received that training but by the time they graduated the industry had already moved on and so there's this constant need to resituated to update your store of knowledge and repertoire both on the methodological front and on sort of the practical artifact from we're also looking at what can we do is design educators to further inform students professional development so thinking of this not as something that happens only in the classroom but that were a series of lifelong learning practices in motion that are going to carry those students through the rest of their professional careers so really understanding what those lifelong learning practices are becomes quite important to our work in formal educational practices there has been quite a bit of prior work on what it might mean to define or demarcate UX as well most of this has been around the survey and interview sort of space primarily surveys and so a lot of these a lot of this work has gone out and tried to figure out what are people doing in relation to sort of structures that are already being put in place from an academic perspective and while this is really useful most of our work over the last few years at Purdue has been focused on doing practice led work trying to figure out what are practitioners saying on their own terms and how then can we relate those discourses back to the academic community and productive ways so we've done this in a few different ways and I'll go through these briefly so one is just exchanging experiences both the good the bad and the ugly I suppose about what it means to do UX practice work a lot of these debates do come around conceptual vocabulary and so I'm just going to give some little snippets from each of these studies in this case there's this ongoing ux/ui debate and what that looks like in practice and how one can actually talk to other organizational partners to make sure that your your work and your capabilities are being projected and the most reasonable reasonable light we found a lot of discussions of this type in online spaces around what the definition of the field is and how those definitions can be related back to other organizational partners we've also looked at how critique practices are quite prevalent in online spaces some have studied this from a crowdsourcing perspective but we found a lot of sort of more mundane examples of people sharing sharing their work at multiple stages of representation so instead of just the work like the dribble style work where you're saying final artifacts we're also saying work at the early problem framing stage at various user research stages and different wireframe stages that may not be well addressed by some of these other creative communities so in this case we had on a subreddit asking why their registration went down after they tested in a new social login button which very long conversation ensued on the psychology of this visual stimuli and the the kinds of behaviors that you might anticipate from users and finally we've looked at professional disclosure and really trying to describe these communities not just as a repository of knowledge but also a means of actually leveraging one's own expose even in pseudonym anonymous communities like Reddit we found great value in people projecting their background and why they might want to get a new X or why they might have some expertise in the field even if they didn't necessarily graduate with a degree in the field so much of this work over the past couple of years has really been focused on these informal conversations on reddit in particular the slash are slash user experience subreddit but what we realized as we started to look at these conversations in more depth is that there were a lot of different kinds of patterns of conversation and deliberation that were occurring this was from this morning and there are also a lot of there's a lack of structure because that's sort of inherent to Reddit and inherent to subreddits in general and so we wanted to also look at more structured forms of knowledge building so started to look at the Stack Exchange for user experience and so Stack Exchange for those of you that might not know is a social question-and-answer community and this is one of 170 different QA communities within Stack Exchange and this one is specifically focused on UX design this set of communities also includes Stack Overflow which if you're CS person you probably spend about half of your life googling on Stack Overflow the site includes built-in reputational mechanisms to encourage high quality questions answers and edits of existing questions and answers and so thus to some extent in this study we've relied upon the community curation that occurs and the roles that are built into the ICT itself to validate the relevance and quality of the overall corpus that we were able to capture since we've already sort of established that UX designers are using social media to discuss their design practices and share knowledge we decided to extract that that knowledge from this social media site in order to inform innovation and pedagogy and also our understandings of practice so used a mixed-methods approach here using natural language processing and qualitative content analysis to collect and analyze nine years of qat communication to answer to specific research questions first question was what are the concepts and knowledge categories that we could use to characterize this body of UX knowledge and how is that knowledge that has been shared on the site evolved over time so here's what we did we collected about a little over 20,000 question-and-answer sets from 2008 to 2017 and that represents the the complete corpus that was available to us in 2017 20 2008 was the first time a question was answered question asked and answered then we used a part of speech tagger to extract all the unigram nouns there were 53,000 I suspect there a lot of spelling errors in that as well and we didn't really try to rectify that at this stage wasn't that important for the type of analysis we were doing and at this point we identified nouns within that as candidates for UX concept and formed by psycholinguistic work that finds that nouns are to be more basic than verbs in human language learning and this had a long tail as you might expect a maximum of 90 thousand occurrences of the unigram each unigram an average of 37 and a standard deviation of 557 after that two of the researchers manually coded through the unigram x' and identified 600 for UX concepts that we felt had unambiguous meanings the criteria we used to delimit a UX concept was that a word must have an unambiguous meaning that is relevant in some way to UX design for words that for which the two coders disagreed we went to the actual QA communication that involved that word to check to see if the word was used with only one meaning and so we were trying to find word words that were not generic but that were somehow specific to design practices or UX practices more generally and lastly we used a qualitative analysis approach to establish categories and interrelationships among the words that we were able to capture and the unit of analysis here was at the concept level and coding was performed in an inductive manner moving towards axial code so we started to develop structure in the data this is essentially what we've come up with it's in the paper so I'm not going to belabor each point here this is the body of knowledge UX knowledge that we were able to generate through this method and I'll talk about the limitations of this at the end so if you're forming questions in your mind like I don't believe you we'll talk about that we divided the body of knowledge into six major categories in each major kept major category in several secondary categories I'm just going to walk through these a very high-level very briefly the conceptual category includes UX concepts that we felt to be abstract and with two secondary categories one theory and one intermediate level knowledge and we drew on hook and logan's notion of intermediate level knowledge here to sort of play this out in a little bit greater depth you can see Norman gets his on key keyword we also identified six specified knowledge branches within UX which seem to be especially salient salient a lot of these probably seem quite obvious some of our reviewers pointed out some things that they expected to be here that weren't well we didn't see some of the structures that people would anticipate seeing in this area so that might be an area for further work we this was a bit of a mix so we didn't find any clear distinction between UI design and interaction design concepts from from a noun level and we're just looking at you know grams here not not by grams or engrams and so really here we categorize these as perceptual features of a design product and then we talked about elements and patterns as relationships within their the designer arif oriented category described words that were just specific enough to be represented design activity rather than a technological activity more generally and then tech basic were general technology words that weren't specific to design but obviously have relevance to doing UX work and finally this category of beyond the cubicle was a pretty messy set of terms but they often describe this context around doing design work both in the office and in society more broadly that shaped designers work in important ways so this categorization strategy that the word level allows us to look at UX knowledge primarily from linguistic perspective and this is different from prior work that's looked at question the topic of questions primarily at the content level and so the typology of question topics that we've looked at here suggests that UX designers conscious knowledge sort of involves this casual nuance often messy sort of set of structures that reflect larger social and cultural trends in relation to UX and relation to disciplinary norms more broadly because we have this UX vocabulary we also able to look at it a little bit over the past 10 years to see what other trends might be salient and we feel like this could be actually quite helpful for educators and researchers to start to look at and track these trends over time and well it's not possible to look through every temporal change I would like to just highlight a few things here so for instance basic unambiguous terms and design basic tech basic and perceptual perceptive perceptible features have occupied a substantial portion of UX practitioners language this isn't really surprising user and design have really been set at the center of what it means to do UX work since the beginning and the usage frequency of interaction patterns was actually quite a bit lower than the interface element because it's a little bit more abstract designer oriented terms were generally stable but user research experienced a slight increase and so we see this as part of a growing consensus in UX that it's not just about doing interface work it's also about engaging with research activities surrounding that work and we're also noting here that theory and intermediate level knowledge are quite low if you look at the percentages and this reflects the persistence of the research practice gap that I mentioned earlier so we've used social media data and data analytics to define an D market UX knowledge however there are lots of limitations to this approach and we readily acknowledge the limitations to the approach because some things just might not be talked about they could very easily be marginalized or rendered invisible and that's a big risk here there's also differing sort of conceptual baggage that comes from different epistemological spaces and we weren't able to reconcile and fold the overlap among all those spaces so fortunately we've published the data set of the you know grams that we identified along with the frequency counts so if you want to do a different analysis on us we would welcome you to do so the materials all there and we want to continue engaging with this work as well looking to the bigram an Engram level and also look at this as a way of actually mapping learning priorities for emerging disciplines more broadly not just things like user experience design but also emerging areas like data science and with that thank you I'd be happy to answer any questions thank you so much and also thanks for sharing the material that's really useful for the community hi hi Colin I'm Elizabeth Bui Sigma consulting solutions and recently of Northumbria University so I I'm gonna question your use of Stack Exchange I realized that it's been around a long time but so have other media and and the question essentially is based on the the fact that I don't know anybody who uses it I've been in UX practice for quite a long time and or I don't know anybody who uses it a lot and so I'm wondering what the level of expertise is of people who use that and if it's really representative I realize it's very large and it's been going on a long time but but how much can you extrapolate from this the the people I know they in interaction design association and in the use of uxpa and there's a mailing list for usability and and the information architecture institute and and i am just not sure that Stack Exchange is really representative I think it's a very fair criticism and so I have taken the multiple approaches angle so Stack Exchange is certainly one area that we're investigating we've done a lot of work on reddit where we've also found some pretty strong uptake we've also looked at Twitter and some other sites and we're at we actually article into review right now looking at and can herring discourses among multiple channels I think we need to be investigating a lot of them and not just one of them so I think that's that's perfect yeah and in a lot of cases is very difficult to know because their student on those communities and unless they disclose which they often do it's very difficult to know but it's a fair point Thanks hi Colin thank you awesome piece of work I wanted to just understand a little more about acronyms cuz they're often nouns but they're not necessarily in the dictionary but you frame this around UX and it feels like you're your analysis would not have flagged UX as a term or MVP or HCI I'm wondering if you did do you think about that did those get excluded how did they play out we did so I'm gonna have to look back at the data set because I can't answer that question off the cuff I believe I believe that we did capture a class as nouns within the data set but I would have to confirm and what we what we did not do is we did not collapse instances of like user experience versus UX and collapses down into single instances and so I think there's still a good bit of work that could be done to clean the data set and do a little bit of further work so actually I wanted so great talk thank you very much my name is Anya Doshi I wanted to ask about the sort of perceptions of differences between UI and UX that are very common in the industry and you remarked about and then eventually didn't come up in fact the slide is the screenshot that is showed of the UX group on interaction Design Group on slacked Exchange stock-exchange actually I was trying to read the first three questions they were all about UI and not about UX at all so is there something that you can sort of reflect upon particularly with respect to you know how this goes into teaching design service I mean I think this is a challenging area and what we've seen in our prior work in this in this space is that UI conversations tend tend to occur with more frequency because they're a little bit easier to talk about and it'll be easier to show and so there is a more standardized vocabulary around these areas we are seeing the emergence and in particular work around design critique in these spaces has shown that there are a lot of final artifacts but there are lots of artifacts that are messy and in between but they're going to show up in very different ways in these communities so it's not to say that we're completely excluding UI but we have found there that there are multiple discourses in play in these communities and we probably need to actually start to separate those out as well what we found in our work on self-disclosure is that well people are coming from a very wide range of Industry context and I don't know how how simple it's going to be to actually separate out those discourses but I think would be a fun next step [Applause]

No Comments

Leave a Comment

Your email address will not be published. Required fields are marked *