Season 3, Ep. 7 – Conducting user research, with Fatima Karwandyar, Product Management Leader

(THE FRONTIER THEME PLAYS)

Faith (00:05):

Hi, Fatima. How are you doing?

Fatima (00:07):

I’m good, I’m good. How have you been?

Faith (00:09):

I’ve been good. Just, you know, I’ve been bracing for this end-of-week rainstorm-slash-windstorm that we’ve been promised all week, and it’s like, a beautiful day in Nashville.

Fatima (00:22):

It really is. Yeah, I thought like, oh, it’s gonna get super cold and no, it’s…I know we’ve had some really nice weather. It feels like it’s spring here.

Faith (00:32):

I know. But it’s been nice weather when we’ve been promised terrible weather. So like, I canceled my hiking plans this morning, so I was like, well, the weatherman says we’re gonna get a ton of rain, and it’s gonna be really windy, and meanwhile, it’s like blue skies. It’s just like, absolutely gorgeous. So I’m a little bitter about it <laugh>.

Fatima (00:52):

Well, maybe cut your day short, you know, and get out there at like 3:30, and get your hike on <laugh>.

Faith (00:58):

Yeah, I would, maybe. I think it might have to be like, a city hike, as you will. (Fatima: Yeah.) But I live in East Nashville, so we’ve got, obviously, hills all over the place, so sometimes it does feel like I’m on a hike <laugh> (Fatima: Yeah.) when I’m walking around the neighborhood. Well, I’m so excited to have you on the Frontier. This is just, it’s kind of like my sneaky way of just like, getting to chat with people and learn things from them. And the last time you and I worked together, I just learned so much. So I’m excited to share that with folks, as well. But I guess to get started, Fatima, why don’t you just introduce yourself, kind of share what you’re doing now, kind of what your specialty is, and then we’ll jump right into, hopefully, learning a whole lot about user research and interviews.

Fatima (01:46):

My name is Fatima Karwandyar. I have been in product management, I guess 12-plus years now, and most recently am working with companies around product strategy efforts. So I realized there was a missing gap in, I would say, the front end, you know, beginning of product management, which is really understanding the customer problem, user research. And so I have a passion for that, given that I would say I’m part of the generation that lived without social media and the internet and then grew up in it and using it. And so seeing, kind of, that user experience really lacking for a lot of companies really gave me an opportunity to say, let me help other organizations that may not have that skill set in-house. And so I’ve been leaning into that and working with really, really cool companies and other industries, which I’ve enjoyed.

Faith (02:40):

That’s really cool. I didn’t realize that you were doing, kind of, consulting work mostly full-time at this point. So that’s really exciting. So with consulting, it’s really cool to hear, kind of, your sweet spot, it sounds like, in terms of stage and when you can add the most value to companies. I’m wondering how you gauge potential clients in determining how much value you could provide to them. Like, is there something you’re looking for in particular that’s like, oh, these are…this is a company that I could really have a big impact at?

Fatima (03:14):

I think usually the company’s ready for it. They may not know exactly, like, “Oh, I need user research help,” or “I need product strategy assistance.” But they probably haven’t tapped into, kind of, “Okay, we’ve been doing this for a while; we’ve done a really great job.” There’s been some changes whether the company got acquired or they’ve been in startup mode for a long time and, kind of, really built the product, and they haven’t had a chance to revisit it, and so they’re usually ready for that. I would say that conversation of, you know, “Is it time for us to look at a new product? You know, do we really need to evaluate the user experience?” So I would say those are probably the best clients, because they’re looking for my help, but they just don’t know exactly like, what it is, exactly. So I think I’ve never really knocked on someone’s door and said, “Hey, your, your product’s not delivering a great user experience.” And they’re going, “What? It looks great.” Usually, I think they’re ready for it.

Faith (04:08):

Yeah, that’s interesting, because usually when I think about user research, I think about startups who are building their first product, and they’re trying to validate it in the market. And it sounds like, based on your experience, maybe an equal need to that is folks who are established and are exploring maybe a new product line or maybe the environment has just changed since launch, right? (Fatima: Mm-Hmm <affirmative>.) Like, if you put a product on the market 10 years ago, the industry is completely different at this point. User expectations are totally different. I wanna put a pin in that, because I think that’s important to circle back to you in a second, but I realized we didn’t, kind of, lay the groundwork here of just base-level. What is user research, for someone who’s listening, and they’re like, “I do not do that,” or, “That sounds like something I should do, but I don’t quite know.” Is it like, Googling, you know? Go ahead; let’s clarify that for folks.

Fatima (05:05):

When I had a defined user research, it is truly sitting across from someone that could be your target customer or someone that’s actually a user of your application. And sometimes like, in the case of something new, you may not know if they’re your customer yet, but you’re having that one-on-one dialogue to show them something, get their feedback on something that’s existing, or you’re actually meeting with them to understand the problem space. So that’s how I would define user research. I think the idea of customer feedback comes in different ways. You know, there are surveys, there’s, you know, net promoter scores, there’s all those other means of understanding how people react to your solution in the marketplace. I would say what I do is I try to hone in on that one-on-one dialogue to really, you know, bring out more information from people. ‘Cause I think that’s the thread that you don’t get when you send someone a survey and expect them to answer things, because no one wants to fill out four lines of information. I mean, I think a journalist would be a great user researcher, ‘cause that’s what they’re doing. They’re trying to hone in, they’re digging in asking those questions and, kind of, pulling on that thread. So that’s what I would say user research is.

Faith (06:18):

It’s interesting, ‘cause I feel like we’re in, and we’ve been here for a while, just as like, a tech industry, we’re in this phase of, well, let’s like, growth and A/B test everything, right? And so I feel like the sentiment is, well, why would I spend time and money talking to like, five people who have very different biases, experiences, expectations, when I could just A/B test several thousand or tens of thousands of users and learn about what they want that way? What have you found as the real value you get through these conversations that doesn’t always show up in A/B tests and that kind of thing?

Fatima (07:05):

I think people that get into product really enjoy solving problems, and A/B testing is…it is useful and is purposeful when used at the right time in your process. But I think if you are looking at your data, and you’re not seeing engagement rates, or you’re not seeing sales close as many deals as they want, changing a button or the placement of something, it’s probably not going to move the needle the way that you need it to. Especially, when you think about like, true revenue dollars for your business. And so that’s where user research comes in, is you’re really diagnosing the problem. You’re understanding people’s emotions, and frustrations, and joy and ensuring that whatever you’re delivering is actually solving their customer problem. And I think that’s the value that you get when you sit and talk to five to eight people.

Faith (08:03):

For us, I feel like talking to customers is what gives us the ideas or the hypotheses and then doing our, kind of, large scale “a/b” tests to see like, all right, which problem statement really hits, is how we validate what we learn from user interviews. So yeah, that’s really awesome. I think, I mean, obviously, I’m a marketer, and I focus a lot on growth, and I work closely with our product team as well. And so product and marketing, I feel like the use case for a user interview and the really rich data we get from those are obvious for product and marketing teams. But what are, maybe, some non-obvious ways that other teams can really benefit from hearing directly from the market? And the way that you, you know, conduct those interviews?

Fatima (08:54):

One, I always record my sessions, and I share back the feedback, whether it’s with, you know, the customer or the client, but even the internal stakeholders, you know, when I worked within an organization, it was really valuable for me. Everyone that touches the product, to see that feedback, especially if you’re in a technology world, getting everyone to buy-in to what you’re doing is really valuable. (Faith: Hmm.) And I think, sometimes, technology folks, developers, engineers are really neglected in making sure that they hear those customer conversations, ‘cause the salesperson is probably talking to the C-suite, and talking to the product people, and the customer service people. Well, what about the people that are actually building the solution? You know, when they hear the value of what they’re building, and the impact that it’s providing, everyone now has skin in the game, and they understand the “why” behind what they’re doing.

Fatima (09:48):

So I would say everyone should have exposure to it and understand the “why,” and I think that’s the way to, kind of, include everyone in that process. So, you know, for me, whenever I’ve been, you know, within an organization, really sharing that intel company wide, is really valuable, and people like hearing what customers are saying and what users are saying, especially when you live kind of in a B2B world, ‘cause I think those conversations are very visible. It’s talked about at the highest level, but sometimes that consumer, that in-customer of that software product, is really valuable. And people really, I think, are hungry for hearing that feedback. And so that’s kind of my way of like, trying to include as many people in that process.

Faith (10:36):

Hmm. I hadn’t thought about using it as a tool to generate buy-in, kind of, across the team. So product is, obviously, typically the closest to users and responsible for running user research. And you’re totally right that if I’m sitting, kind of, across on another team, watching what product is building, and being like, “Now why are we doing that? That is like, not even, kind of, related to the last product we put onto the market, or what I’m hearing from folks on Twitter, or whatever.” So using user interviews to generate buy-in is really interesting. When you work with your clients, what team are you usually working the most closely with?

Fatima (11:18):

It really depends. I prefer, I would say, more of the business side of the organization. So, usually, it’s product or like a business leader, so they’ve got some chief in their title, (Faith: <Laugh>.) because they see value in trying to improve the customer experience. And so they’ve got revenue tied to that; they’ve got data. So I think those are the key stakeholders that I work well with. And I tell people, you know, I’ve been in the software industry, but product is very much a business role. You know, if you think about it, you know, we have to ensure that we’re, you know, delivering on metrics. Are we actually helping to support our sales team? Are we growing engagement? And so it makes a lot of sense to really work with those people, ‘cause the product represents the voice of the customer. And so that’s who, usually, I have worked with, you know. And sometimes you’ll get really savvy people in other industries, you know. Technology officers that are like, that have worked in other places, and they’re like, “Yeah, we need this feedback. We don’t have this intel. Can you help us get this intel?”

Faith (12:24):

Yeah. You know, thinking of product as a business-level function is sometimes not obvious, but you’re totally right that that’s probably the best way to run a business that’s, kind of, aligned on all fronts. You touched on, at the beginning, when we were talking about your ideal clients, that usually what you’re looking for is someone who’s established but is really ready for like, a refresh to understand what’s new in the market, or to maybe pressure test a new idea, or get inspiration for a new line of products. So, obviously, like, as a brand evolves, there’s a need for continuous user research. So I’m curious, how often do you think brands should be revisiting user research? Is this something that should be done weekly by somebody full-time? Is this a quarterly or a yearly initiative? What’s your take on that?

Fatima (13:20):

I think you should always, as a product person within an organization, you should always be monitoring, keeping a pulse on your customers in some form or fashion. And so there is a time and place to have user interviews, user research, but hopefully you have other metrics that you’re actually measuring, engaging how customers are feeling about your products. You know, how do customers reach out to you to let you know that, hey, they want a feature or they’re feeling friction? So do you have a direct line to customer service to get that information? So I would say you should always keep a pulse on your customers, and I think sometimes you’ve failed, and I think with everything else that you have going on, you’re so busy with the day-to-day operations of, okay, what’s the roadmap? Are we executing on the roadmap? Are we delivering this on time?

Fatima (14:08):

I’m now on a customer call doing this. And I think always being close to your data and knowing that you have access to that information is really valuable. And I know I’ve been in organizations where we didn’t have a direct line, we didn’t have those metrics, and so I would recommend, you know, that’s the first thing standing up. Like, you need to have intel coming to you so that you can monitor that, and then when you start to see that things don’t look as optimal as you would like, okay, let’s talk to customers, let’s really dig into what it is that’s not like, we’re not seeing the level of engagement that we want. And so I would say it’s a continuous effort. You know, products should never be far away from knowing the sentiment of their customers.

Faith (14:53):

Yeah. You touched on this like, product is never short of tasks to be done <laugh>, right? I think like in any growing company, that’s probably true across teams, especially for product. And so making the case for putting applause on the stuff that has a lot of energy and to-dos around it, to revisit user interviews, which aren’t, they’re not low-effort, right? Like conducting a really great user interview takes a lot of intentional time, and planning, and skill. So I’m curious, for folks listening who maybe are operating on limited bandwidth or resources, what are some scrappy ways that they can get this done? They can still talk to the market and learn from the market without having to pause other things that might be more mission-critical.

Fatima (15:46):

If you’re an established company, and you already have customers, you already have people that you can talk to to get feedback on the software that they’re using. So pull into them, and I would say, you know, set time aside, you know, schedule five, you know, I always say schedule six to eight people just ‘cause you know, sometimes you’ll have really engaged people, and then sometimes people cancel. So I’d say, you know, try to schedule eight people, and then if you can get five to six, great. You also, if you’re a entrepreneur, or you’re someone that has an idea, and you are like, okay, you don’t have the resources to do that, you probably know someone in your network that knows someone, that knows someone that can get you in front of a person that can give you insight.

Fatima (16:40):

So if you have an idea about an app to help you help a consumer learn how to compost better, I’m making this up, well, go talk to someone that, you know, is trying to compost or has no idea about composting, and ask them those questions, and say, “What would you want, as a consumer, to help you understand like, how to begin composting, you know?” (Faith: Mmm.) So, you know, we’ve got recycling here, so how do you move into that world? So, I would say, it’s just a matter of carving that time out, you know, reaching out to people and just start asking questions. And so it’s a pretty low-level effort, and the more that you do, you will get better at it. You’ll learn techniques about, you know, not leading people and letting people, kind of, really lead you to, you know, tapping into those insights and sharing that with you. So I would say it can be a, you know, you don’t have to hire an outside service company to find people for you. And there are, there are ways to do that, but I also think, like, to your point, you probably know people, and I think it’s just a matter of talking to people and saying, “I have this idea,” or, you know, “How do I get in front of this person?” Well, they probably know someone. So, (Faith: Yeah.) and then if you’re an internal organization, reach out to your customers. They will probably be excited to talk to you.

Faith (17:52):

I mean, it sounds like the advice there is don’t overthink it <laugh>, right? Like, don’t let perfect get in the way of good and just do it. Right?

Fatima (18:00):

A hundred percent. Yeah.

Faith (18:02):

Well, as somebody who has worked with you on user research and received feedback that you’ve generated through user research, I know firsthand how overwhelming it can be to be, kind of, given a massive collection of feedback and try to sort through it to determine what’s most important. What do we still need to validate what’s actionable now? And I’m curious if you have a playbook for how you recommend folks do that? Kind of, what to wait higher, what to maybe put on the backlog? How do you help your clients navigate that?

Fatima (18:39):

One, I like to present people everything, because I don’t want, you know, if I’ve spent, you know, eight hours, you know, talking to people, sometimes longer, and you only present 15 minutes worth of information, it makes me question, did you even talk to them? Did you synthesize the information? So I really try to give people everything that I’ve heard (Faith: Mm-hmm <affirmative>.) from their customers or their target end-user. And from that, that’s why there’s a sweet spot between, you know, interviewing six to eight people, is there will be natural themes that arise that will allow you to explore and try to solution for those. And so I try to do that for clients, ‘cause as they’re trying to weave through, they’re like, “What do we focus on?” “Well, actually, these three things, you know, I shared everything with you.” These are just like, keep them in your back pocket.

Fatima (19:30):

If you start to hear more of it, you’ll know that so-and-so, we talked to that person, did have that opinion, maybe we need to dig into that further. And then there’s the question of do we do A/B testing? Maybe let’s get on the phone with them and validate if other customers are interested in this capability or feature. But, usually, I would say themes develop and themes, you know, if you talk to six people, I would say at least three people or more felt the same way about something that they saw, regarding your product. (Faith: Mmm.) And so that I would say would be…I don’t try to prioritize, hey, you know, here’s number one, number two, number three, but here are the themes. You know, three or more people felt that they would like, they either really liked, or they had an issue with. And so I’ve never given someone like, a laundry list of a hundred things, and I think you, you know, I gave you everything, but I would say there were probably some key themes that arose. And it’s an iterative process, so (Faith: Mmm <affirmative>.) tackle those things, test it again, and then try it again. And so I would say that iteration’s a really good thing, to your point, like, don’t let perfection get in your way (Faith: Yeah.) of making an impact.

Faith (20:43):

Yeah. It sounds like the strategy is, try to gravitate toward themes, rather than like, one very, very strong opinion, right? If you get one user who’s like, “This is awesome! I love this,” and everybody else is, kind of, neutral about it, maybe don’t double down on that feature quite yet, because you’ve got one evangelist, and let’s listen to the thematic, kind of, the majority of folks, right?

Fatima (21:08):

Yeah.

Faith (21:10):

Well, Fatima, last question. Inevitably, someone listening is gonna be like, “Fatima has the coolest job ever.” (Fatima: <Laugh>.) “That sounds awesome.” For somebody listening who’s interested in getting into product and specifically user research, how would you advise them to get started?

Fatima (21:27):

One, observe someone doing it. And so if you could find someone on YouTube that’s showing you how they would do a one-on-one interview with an existing website or discovery where there’s not a website, but they’re probing that person to understand, you know, trying to pull up that customer problem, watch them and observe them doing that. And like I said, just practice. A book that, I think, early on in user research careers that people relied on, was Don’t Make Me Think. It (Faith: Mmm <affirmative>.) really provided a, I would say, kind of a nice framework and coursework around, okay, how do you even begin in moving this process? And he started really simple with just website testing and, you know, how do you go through that? So, and there’s techniques out there, you know, like I said, you wanna take more of a journalistic approach.

Fatima (22:18):

You wanna ask questions and really let, you know, I’ll tell you a technique that I use. And so when you put something in front of someone, and they’ll say, “Oh, is this where it’s supposed to go? Or if I click this, will it take me there?” I’ll say, “Well, what do you expect it to do?” Because one, you’re validating, okay, the user, the way that you’ve designed it, the user does expect for it to act that way, and they’re telling you that’s how they would like it to act that way. So that’s, kind of, one technique where you really, you wanna limit your opinions, and as someone that’s conducting the research, one, really sit back and just let them kind of drive the conversation. Obviously, reel it back if it’s going off topic, but I would say, you know, that’s one book that really helped me. There are local resources in your city, where you can take UX courses, where you don’t have to, you know, be enrolled in a school. But, I think, just learn a little bit, do your first one. And I think we, as human beings, naturally, you can ask questions. Everyone can do it. So it’s just a matter of getting in there and just trying.

Faith (23:26):

Yeah. Fake it till you make it, as some say.

Fatima (23:29):

Yeah, yeah.

Faith (23:30):

<Laugh>. Well this has been really helpful, Fatima. If people are listening, and they are interested in getting in touch with you, where can they find you?

Fatima (23:38):

Yeah, I feel like LinkedIn’s a really great resource. I don’t know, we never…[when] you post this, if they see my name, I’m probably the only Fatima Karwandyar in the entire universe. So…

Faith (23:49):

That’s very helpful <laugh>.

Fatima (23:51):

Yeah. So I think you shouldn’t have any trouble finding me, (THE FRONTIER THEME FADES IN) and just send me a DM, and if there’s any way that I can help someone that wants to get into this space, I’m happy to chat.

Faith (24:03):

Awesome. Well Fatima, thank you so much. It’s been so much fun to chat with you and we have to get together in Nashville. I say this every time we chat, but we’ve gotta do it. Thanks for listening to the Frontier podcast, powered by Gun.io. We drop two episodes per week, so if you like this episode, be sure to subscribe on your platform of choice, and come hang out with us again next week, and bring all your internet friends. If you have questions or recommendations, just shoot us a Twitter DM @thefrontierpod, and we’ll see you next week. (MUSIC STOPS) What’s your, (DOORBELL SOUND, DOG BARKS) do you have a kind of rating system? Oh, who’s that?

Fatima (24:48):

Gosh, that’s…I think you’ve probably heard a dog and a delivery in the middle of the day. That’s what happened <laugh>.

Faith (24:52):

Classic. What’s your dog’s name?

Fatima (24:55):

(DOG CONTINUES TO BARK) Charlie. He is an English springer spaniel. And so he’s all love but acts like he’s pretty tough when a doorbell or a stranger walks by. (Faith: Yes.) So…

Faith (25:05):

My dog looks like an absolute buffoon <laugh>. (Fatima: <Laugh>.) He’s like a big puppy Australian shepherd and just like, tongue hanging out the side of his mouth all the time. His name is Frodo, and like, who’s gonna take a dog seriously whose name is Frodo?

Fatima (25:20):

Frodo <laugh>.

Faith (25:20):

He does the same thing. Like, even just like, he’ll see like, any sort of cargo van go by, doesn’t care. But if the van says “Amazon,” he’s like, not today, (Fatima: <Laugh>.) not in my neighborhood <laugh>. So I feel your pain with Charlie. Sounds like he’s doing a good job, though, of protecting the house.

Fatima (25:41):

Exactly. You know, of unnecessary spending (Faith: <Laugh>.) is probably what he’s barking at.

Faith (25:47):

I hadn’t thought about that. Maybe Frodo’s my financial advisor, (Fatima: <Laugh>.) and I just have been, I haven’t learned how to speak dog.