Episode 5: Using in-app UX Analytics to drive decision making

UX Leadership Podcast Intro Photo of Greg Thomas.

In this episode of the UX Leadership by Design podcast hosted, by Mark Baldino of Fuzzy Math, Greg Thomas, Director of User Experience at TaxSlayer, shares how his team captures user data at specific times to make data-driven decisions and presents a strong case to executives, business owners, and product leaders. By using a one-question survey with a one-to-seven scale (Single Ease Question or SEQ), they prioritize fixes in the backlog and give users a voice without adding any overhead to the team. This episode is a valuable resource for UX design leaders seeking insights into capturing customer data and making better decisions.

Topics Covered

  1. Making data-driven decisions in UX design
  2. Benefits of user-centered design for business success
  3. Importance of data capture tools and user feedback in UX design
  4. Understanding and using Single Ease Question (SEQ) Scores to improve UX design
  5. Prioritizing fixes and challenges in UX design
  6. Building and leading UX research teams
  7. Driving design decisions through in-context data capture
  8. Strategies for Prioritizing Fixes in a UX Design Backlog

About Our Guest

Greg is a software industry veteran with an extensive background in researching and creating user-centered omnichannel customer experiences and in managing, advising Product Management and helping set strategy going forward.
Examples of his work can be found on a wide array of platforms including; web, mobile apps, tablet apps, mobile web, point of sale, pin pad, kiosk, voice systems, mobile payment, and internal employee operation systems.

Currently, he is the Director of User Experience Design at TaxSlayer.com where he reports to the CEO and works directly with Senior leadership.  Prior to joining TaxSlayer, he worked for Fortune 500 companies such as Walgreens, Motorola and for other industry leaders such as Peapod, and Legacy.com.

He holds a Master’s Degree in Human-Computer Interaction from DePaul University in Chicago, a Bachelor’s Degree in Marketing from Butler University, and an executive certificate in Product Management from the Haas School of Business at the University of California, Berkeley.

His innovations have led to him being listed as a co-creator of one patent and sole creator of another.

Resources & Links

Single Ease Question (SEQ)

Connected with Greg Thomas on LinkedIn

Connect with Mark on LinkedIn

Fuzzy Math

Transcript

Mark Baldino  0:00  
Hello, and welcome to UX leadership by design, podcast buy in for UX design leaders. I'm your host, Mark Baldino. This podcast is and always will be brought to you by fuzzy math, the user experience design consultancy that brings consumer grade UX to the enterprise. It fuzzy math, we deliver award winning digital product design, and we'd love to partner with internal UX teams to help them grow and scale their impact.

Today, on podcast, we're talking to Greg Thomas. He's the director of user experience at TaxSlayer, which, as the name suggests, is in the tax preparation space. And he's going to walk us through how they drive decisions at TaxSlayer. from a design perspective, and they do that by capturing data at certain very specific times in context or in situ with their users. So after somebody completes an action, they measure the, their the experience of the customer just had, they do that throughout. So they have this really rich picture of the data, but not too much data, as Greg will explain. And that data helps them drive decisions and make their case to executives, leadership, business owners product. So it's really I think, interesting, it's very much just describes how they went about the process of implementing it, and then how it can drive decision. So I think you're all gonna love it. appreciate you listening. Enjoy.

Greg, how are you? 

Greg Thomas  1:30  
I'm good. How are you doing Mark?

Mark Baldino  1:31  
I am doing very well. Thanks for asking. And thank you for joining us on the podcast today. Super excited about some of the content we're going to be chatting with, or chatting about. I like to ask people to give us a little bit of background on you and your role. And I'm always really interested in the shift from being a team member and individual contributor to you know, now you're a director of user experience user experience at at TaxSlayer. Like, what was that transition like, from either individual contributor to too late? 

Greg Thomas  2:06  
Well, a little bit about me. I started off my career professionally promoting monster truck shows and motocross races. Well, unreal, I'm the poster boy, for all those people out there that were doing something else, and then found their way to UX, man. So I've been doing this work for maybe 15 years now I've worked for pea pod and Walgreens Motorola TaxSlayer, a couple startups kind of be all over the place. 

So that's enough about me, the transition from being a contributor to being a leader. It has its challenges, but I there's a ton, you could probably identify and talk about the one that I think you have to get the hang of really crucially, and really quickly because it, it cascades into all your other conversations about design work about working with clients about working with internal people, is that you have to learn to prioritize the, you know, the challenges that you want to have. So, you know, just today, there was a collaboration session on a design that we were working on, that we wanted to AB test what the group wanted to AB test, I really don't think it was a good design. But because I'm having a, you know, a challenge with that group on another topic, I'm gonna let that first one go and work on the other topic, which is more important that so that's, you know, that speaks to the project specific aspect and like what you have to get used to is making those trade offs and quickly making the decision of this one's not worth it. I've got to go on, and I've got to go on, because I have people that are counting on me to do other things. So I can't let myself get to, you know, mired in things that when I was an individual contributor, I was expected to know. 

Mark Baldino  4:04  
Yep, yeah. Why do you think? Why do you think that that's a challenge for some folks, I mean, met the individual contributor level, very tied to your designs, you're putting your heart and soul into them presented to a group of people and maybe they say we don't think this is a great idea. Like how did you start to realize the woman's digital separation from those ideas, bigger picture, like what was that? What's that process? I think it's a challenge for a lot of designers to separate. 

Greg Thomas  4:31  
Yeah, it is a challenge because you spent so much of your career worrying about every little detail, which is really in my opinion is the crux of experience design is catching all the little details over and over and over so that they never get through. So it's it's a challenge the ended I start to really recognize that it got real simple for me because of the way that I set teams up. I was off working on projects with but where they would get to the ICs. And discussing the requirements and the business case and what problem we were trying to solve with stakeholders. So that then when I got to the ICs, I was able to to individual contributors, I was able to let them take the bulk of the work. And then if there was something pertinent to the conversations I had earlier, I would chime in on that aspect. But I trust that you have to trust your people, once you start to be a leader to do what you're you're empowering them and you're paying them to do, you have to to let go of it. And you learn real quick because you're having, you know, maybe 10 conversations a day about stuff that's coming down the pipe, you have no choice, you learn real quick to pivot, your focus, and start, you know, okay, start talking about this thing. We were talking about login and registration before. Now we're going to talk about cart and checkout. Now we're going to talk about search, I need to get just enough so that I can scope it. And so I can have a rough idea of how long it will take. And then I need to get it to the contributors so that they can do what they do, which is look for the rest of the details. 

Mark Baldino  6:04  
Got it. So you talked about separation from from personal connection to maybe what you're working on, you're kind of thinking a little bit bigger picture. How are you doing the mental gymnastics have always shifting priority and importance like for you? What does important mean? And does that align with the rest of your either your team, maybe the leads you're working with maybe even organizationally, like? Do you have your own set of priorities? Are they aligned at a higher level?

Greg Thomas  6:31  
What I typically that's a really good question, what I typically try to do is to think about the things that I want to accomplish from an experience perspective, which are things that are germane to me, I know how I'm going to say those, I know where I want the experience to go, I try to roll those needs into the conversations I have with stakeholders. And and then you know, you work back and forth to make sure that I'm listening and understanding their needs. And then I'm just gonna roll a lot of my needs in and then if I have to prove something to them on an experience standpoint, then we'll talk about it. Otherwise, I'll just roll things in and get it done. And then show them later on and say, you know, we also we fixed the kerning on this, and we changed the h1 and yada yada, I tried to layer those things in so that it's it's less of a less of a big deal less of a light, you know, I'll say showing off for lack of a better term or term, it's, it's less of UX saying, look at the great things we did. It's more like, here's the great thing we did that meets your needs. And here's something we included, that meets the user's needs, have to talk to people on their language, you have to be able to speak the language of business to product managers, you have to be able to speak the language of marketing, to marketers, you know, on and on, you have to be able to speak the language of health care when you're talking to pharmacists, you know, so they don't care that I'm concerned about the cognitive load on the user, when they hit this specific page. That's not what they're going to come to you and say, what they're going to come and say is I'm not getting people past this page helped me figure out why. I'll say okay, we'll fix these things from a design perspective. And and then you will do these other things, you know, that help you out? And then I'll come back and talk about those. Does that make sense? 

Mark Baldino  8:21  
Yeah, no, it does make sense. I mean, you, you kind of describe stakeholders as coming with their background opinion needs presenting to you. First goal is to speak their language, I think that's really important for designers who want to be leads to learn that we need to empathize with those stakeholders, we need to recognize the value they bring to the business, but what are their concerns or needs? And so it's it's speaking the language, understanding them, and then meeting those needs. But I think what you said was also kind of adding your human-centered or user-centered approach of like, Hey, we've done A but we also included B.Do you need more than that to make a pitch? Like, are there ways that your business and I don't I'm not saying TaxSlayer in particular, but folks you've worked with, like, outside of using their languages are other stuff that you can bring to the table, to sell an idea to push an idea through to to demonstrate importance, not just to us UX folks, but to a broader, like business audience?

Greg Thomas  9:24  
Yeah, that's, that's another good distinction. When someone brings you work, the you're the onus is on you to speak more of their language, which happens a lot in our world, you know, people bring us business cases that may or may not be true or accurate. And we have to help work with them on those, and that's what they care about. But if there's something that, you know, a stakeholder, a product manager, whomever, if they didn't bring it to my team, but we have something we want to fix with it. We're going to use any number of tools to illustrate why this thing we want to fix is important.

But any of those tools, go back to this idea that we have to be able to tell stories. And we have to tell the story of what the user is going through. So that the, the audience can empathize with why this is important, and why this should be included into the backlog, you know, to be worked on. And if you can layer into that business reasons that will resonate with them and they didn't know existed, then you're gonna get really far, because they're gonna say, wow, you're improving the experience. And you're coming to me with something that the business would enjoy the support of. So tell the story when you're bringing the work otherwise, layer in the things important to you? 

Mark Baldino  10:40  
What's an example of of something that's resonated with business that you've sort of layered into story?

Greg Thomas  10:49  
Oh, gosh. So a lot of times, what we take forms, for example, you can imagine that, you know, Luke Wroblewski would be happy to to know like, you know, he makes an art out of creating forms. But there's different things you can do to forms that will pull different levers, right? So if we have a forum where a person needs to slow down, and you know, absorb the information more, then we'll highlight that we're using that form pattern. And here's why we use it. And explain to them that like this is actually going to be better so that people remember this other thing. 

Yeah, the interesting

Mark Baldino  11:28  
It's such an interesting concept of slowing people down, because the business tools we work on with our clients, speed and efficiency are important accuracies there, but some people even like time to task type, metrics, and KPIs. And ideally, you're trying to slow someone down so that they're more thoughtful, or they remember something from maybe step four to step five is like a super, that's a good really interesting challenge as and we all love Luke W. But like, like, are you able to, like make that case like this is? This is like how I made this better question how you will make that case that like you want somebody to be more patient and thoughtful in their experience. 

Greg Thomas  12:13  
Well, some of the ways that I like to make the case are storyboards, storyboards are, you know, very crucial and easy to make. And you depending upon your audience, you can make them as high fidelity or as low fidelity as you may want. I really liked storyboards Another way is to, you know, start the pitch off using pitch and air quotes. Because, yep, internally, I don't actually I actually have to do a sales pitch, but your pitch, I might start with a video of the user actually encountering the problem.

And then say, Okay, we've seen this, here's, here's the observation, here's the recommendation. Right? So a good example is that in the healthcare space, a lot of health care workers become notification blind. They have so many like sounds and things beeping at them. And phones, ringing cell phones, ringing, yada, yada. So sometimes, we have to give them a notification that they need to read, they need to not ignore this one. So we may lay it out differently. You may use different icons, different color, lots of things that experienced people know that it's a situation like that where you can come back and you can show a video and say the person who right through and they didn't ask that customer, if they needed to consult a pharmacist, we've got to slow them down. So that they remember to ask that question, because if they don't, we're not in compliance with state and federal regulations as a pharmacy.

Mark Baldino  13:39  
Yeah. Interesting. How you mentioned, user sessions and videos and starting off there sounds like that's one way to sort of capture a little voice of the user. What are you all doing? And either in storytelling or general, around like, testing, and data and analytics? What information are you capturing? And how does that inform? How you? I mean, I think pitch is a great word, actually, because you need to, we are salespeople as designers, I'm sorry, but that's just that's just the case, right? We're coming up with our own viewpoint. And a lot of times it feels like data can help. What do you all doing to capture that data? How does it inform your design process decision making process? 

Greg Thomas  14:25  
Yeah. So we have probably all the standard tools that the audience for this probably has, right, like we've got different digital analytics tools that are tracking clicks and views and so on. We have a session replay tool. You know, we have those common ones. One of the ones that I am especially proud of and I also like that we use to help determine the severity and the priority of a problem is the inside of our tax application. And we have all these different places where when a person, when a person finishes that action, we pop up a little survey to them, it's called the SEQ, or simple ease question. And you can read more about it on just walrus, he has lots of articles for those out there listening.

So it's just one question with a, you know, a one to seven scale, and it pops up, it's very easy the person makes their selection and goes on about their business. So we ask that one question at, you know, roughly like 75, or 80 different places in our application to random people. So so so for example, I might get the survey, after I fill out my W2, Mark, you might get the survey, after you complete your Schedule C. I don't get the survey, because I don't even do this, it'll see Brad who don't get the W two. So we have that sampling figured out so that you're not getting, you know, to bother. But then what we do is we take all the W two people, we roll them up, we get an average of their score, we take all the Schedule C people, we roll them up, we get an average of the schedule C score, then we have just a rank ordering that we say okay, here are the 10 Lowest performers, based on this many respondents to this survey, you know, so the lowest average rating was a 1.7. The amo, and we might go up a little bit from there. But then we have that, that listing that priority or that rank ordering, that then we can turn over to business people and say, you know, here is a number where people have actually told us, they don't like this particular thing. And because they told us, that's why that particular thing needs to get, we need to start on it needs to be included in the backlog. And here's why. And it's really interesting.

You know, business people, that people that are stakeholders in our, in our MSP, they really love numbers, if you can give them a number, even a derived number, such as an sau score, they they resonate to it, they can look at a list and say, Ah, that one's worse because it's a 1.7 than this other one. That's a 2.2 that clicks for them. As we're people like us we can we can look at the Schedule C experience when we know it's not pleasant. 

Mark Baldino  17:27  
Yeah, that's, that's amazing. So I just want to take a pause, because you at this, at the beginning of this describe things that kind of everyone has, which I don't think everyone has, I think a lot of designers and leads struggle to have data capture tools playback session. But this sort of integration of SEQ, I've been in the business for 20 some odd years. And very few of my clients or projects, I've worked on platform business application tools, have any sort of in app monitoring, maybe some basic metrics, but like very rudimentary, like usage of features is would be great. But that idea of like capturing an SEQ satisfaction score in in situ in context, I think is very rare. Yeah, it's you reference that your stakeholders are numbers oriented, but a lot of a lot of stakeholders are and a lot of organs like organization. So what was the journey to because I assume this before you were in this role, they didn't have that. And this was part of your introduction, like, what was the process of making the case for we need to track this, we need to track it in this manner. I get that they like the data now. But I think a lot of people are like, I don't even know how to make the request. I don't know how to make the story. They don't want to invest in this. This requires technology and like implementation, like code and stuff, you know, that easy to do. So like, what did that journey looked like to get it up and run it

Greg Thomas  18:56  
Yes. So initially, I think anytime you move into a leadership role, you have to step gingerly and maybe a little bit slowly, personally, that's my style is. So I took my time when I first came here, learned from everyone asked lots of questions. The first thing that I realized was that we struggled to be user centered. You're at TaxSlayer. Brook for a variety of reasons. But one of the reasons is our number of people, we're a small company, we don't have a ton of time to be going out and getting data. One of the other things we struggle with is the seasonality.I can't, I can't go and talk to a person about their taxes in September, because they're not thinking they're not in the same headspace. So what I quickly realized was that we need a tool that runs automatically to give the users a voice that doesn't have any overhead on my team. And it's that simple. It's not going to give so much data that we get stuck in analysis paralysis type situation. And so the the SEQ is something I read about in grad school a long time ago, it's been around for a while. And I just basically was like, well, people kept coming to me during that introduction phase, and saying, you know, this was really bad, isn't it? And I'd say, Well, you know, kind of, but is it worse than this one? And? Well, maybe. So I, then I went back to people is that okay, remember that question? I think I have a way we can answer that. And it's very simple. And here it is.

Mark Baldino  20:29  
And then acceptance, sort of, I'm sure it took a while to sort of integrate that this is all like homegrown code, you're not using a third party platform for,

Greg Thomas  20:39  
 We do use a third party. 

Mark Baldino  20:40  
Oh, you did.

Greg Thomas  20:42  
The SEQ structure and format is very simple, you can find that out on the internet and get, I would just stick to it pedantically. So that, you know you, you don't introduce any any variables. And there's any number of survey tools that you can use to drop the code onto your page, and then have them collect it. And some of them even have replay tools built into them. So you end analytics tools. So you might kill multiple birds with one stone, which is going back to the earlier point of I'll layer and stuff that they don't even know I want, and then tell them about it later on. So So for our SEQ and replay tool, the I have stakeholders that didn't even know that I knew the number of pageviews from that tool. I know how people were using that page from my tool over here. So I can come back to them and say, I'm getting a different number than you're also for something on this page. Oh, and here's the SEQ number. Do you see I'm saying so you layer in things that are that are important to you, and the growth and maturity that a design practice at your company. But then you also find ways to speak about the business needs, and explain to people why is important and show them?

Mark Baldino  21:54  
Yeah, I like that you're I think it's really interesting. You focused on the simplicity, right? We don't need too many. We, we need it, we need signal to noise here, right? We want one thing plus I can pull a play back and tell a pretty comprehensive story. And I have some data to support it. There's also, you know, data can at times say what, what, what certain people want it to say any like, have you run into any drawbacks, or any separation of somebody being like, well, they love this step, because they just learned they're getting $5,000 back from the return they hate this step because it's 50 fields that you have to like, you know, click through like that difference of, of satisfaction score, how do you ensure it's related to the digital experience that they're having with your tool versus their emotional state or well being given that you're dealing with taxes, which is a pretty, frankly, a stressful, stressful thing?

Greg Thomas  22:52  
Yeah, it. So there's some, you know, in our application, there's some parts that we know, are pretty neutral, people don't have a real positive or negative feeling about them, as long as they see their refund amount go up. So a good example is the aforementioned w two, most everybody gets a W2 or a 1099 of some sort. The SEQ will measure their feelings about entering that item. Right. And then they go on about their business. So we've compartmentalize it. So we know that score speaks to just that specific item. And we're lucky in the tax world, I don't have like, I don't have situations where I might have a page of search results. They're all slightly different each time. The W2 is the w two you go through once it works. That's where there's SEQ notion, it will work for some things, it may not work for others, and you've got to think through it. For what works. Did I answer your question? 

Mark Baldino  23:50  
Yeah, you did. I mean, I was just thinking if there's times when the data tells a story, and it's a little bit murky, or it's conflicting, it sounds like you're trying to silo these things and measure it. And that gives you enough comfort that it's telling the true story, I guess you've probably established a baseline, you know, what the W2 was for, you know, 2021 taxes, and you kind of have a baseline maybe for 2022 that you've been working against or something like that. 

Greg Thomas  24:17  
You work against the baseline, and you just try to make things better. And the first, the first time that you roll these things, things like this out and into any type of survey, I think you get pushback from stakeholders, because all they see is a hindrance to their goals. And so you've got to really explain to them the value. 

Mark Baldino  24:41  
Okay, and so, sorry.

Greg Thomas  24:44  
I would say I can't think offhand of times where I would like we didn't have any. So far. We haven't had any shsu scores where they were super low compared to what people would think there's parts of our application that we know aren't that good. But what we do is we that we take that se que score and we combine it with other metrics, such as the number of views, the number of clicks that are happening on a page. And we use that to reinforce and bolster like, okay, not only did they tell us it didn't know, they don't like it or unnecessary perspective, here's the all the things on the page that they're doing, we can see it, we see them moving around this way, you know, layer in the other data, here's the number of views, this particular thing doesn't get many. So I don't think it's that important product manager, but this other one gets a lot, I really think we should work on this one. And what I'm working on my next adventure with this whole thing is I want to take the SEQ score and combine it with like the number of views, number of clicks other other digital metrics and make a composite score, I'm trying to get to the place where I can just show stakeholders, imagine a list of red light, yellow, light, green light, for all the sections of the app, but that stoplight is made up of a composite of other pieces of data, so that they can just look at it and go, ah, red light here, tell me again, what the red light means like, Oh, it's a weighted composite score. So that you know that if this thing says red, don't worry about it, or whatever, you know, however you work, the composite, that's what we're trying to get to. But we're a small, small outfit. So it's going to take some time. 

Mark Baldino  26:29  
Well, for a small out, I'll fit you've got a lot further than huge outfits that that we've worked with. So it's really impressive. And I was going to ask you how you're reporting up. I know for specific features, you kind of have the data, it sounds like you're looking to do some roll up some reports that you would share with sort of senior management that get here's a heads up of where we are in general. And then here's some problem areas, and some areas that are performing well seem super valuable. 

Greg Thomas  26:51  
Yeah. So during season, I don't really do a whole lot now that I've won the battle. And I've gotten this SEQ in, I've sold people on it, and then they've seen the outcome. And we actually use the SEQ values at the board meeting to discuss improvements. During tax season, now, I don't really mention it much, because it is an average, it needs time to run, it's not a metric that is immediately directionable, it's a metric that you will get direction out of, after a period of time, I have the benefit of having a very defined tax season. So we run in a very specific circular kind of way. And so you know, I can silo it in to tax season and keep it simple.

Mark Baldino  27:37  
Awesome. It is really super impressive. If you haven't patted yourself on the back, you show that and your team and, and the broader organization for kind of accepting that sometimes people don't want bad news, right? They don't actually want to be they don't want to get a score that they've invested time and energy, and obviously dollars into the development of a platform and customers aren't, you know, satisfied with it. So it's scary, which may be kind of relates to my last question is kind of wrap up here. Like, why isn't everyone doing this? Why isn't? Why doesn't every two I know a lot of E- commerce tools, right? Websites, marketing websites, lots of funnel analysis, like they're good at data, it's easy to track the dollars. That's that. But like tools that we use every day, whether the consumer facing apps or more, you know, business or in your case, kind of a mix of both? Like why aren't people tracking more data?

Greg Thomas  28:27  
Yeah, well, I actually think that, in both cases, I think people are tracking too much data. And we don't have a method to to, like, you know, to bring all those data sources together quickly. I mean, I, you know, I've got, I've got click, click Data View Data, change on field data, multiple touches, the page crashing, I got all these things. But you know, nobody's doing the work to tie them together into something that makes sense. So So I actually kind of feel like a lot of places have way more data than they know what to do with. And so that makes it a real hindrance. Like they stand it up. And then they're like, Oh, what am I going to do with all this? So I think that's kind of like one problem. I think that another reason why a lot of other people don't do this is that in bigger companies, you might actually have a siloed off group of people that handle research, versus the people that actually do the design work. And those people work together they talk. But it leads to this sort of situation of you stay in your lane, I'll stay in mind we'll work together and advise each other. But, you know, we kind of stay separate. And when that happens, that's where good leadership comes in, is that you know, you need to come in and look across all those spaces and say, Okay, how are we going to improve our ability to, in my case, make decisions that are truly backed in data? 

Mark Baldino  29:53  
Yeah, fantastic. That's a hard thing to do. So again, plug the effort at TaxSlayer and I know under your leadership of the UX squad to get that up and rolling. Thank you very much for your time today. Thanks for sharing your background and your sort of data, data informed decision making on the design side, sort of that journey you've gone through. And I wish you and the TaxSlayer team good luck as you're heading into your busy season, I'm sure.
envelope mail-envelope-closed file_pdf arrow-up chevron-left arrow-left close x linkedin twitter facebook mailbox search
Work with Fuzzy Math
Small teams of passionate people working towards a shared vision.