- Hi, and thanks for watching my new podcast episode.

- Today, we'll focus on market research. Because it's so accessible in people's minds, everyone thinks they know how to do it yet very rarely sales and marketing executives apply it and when they do apply it they do it incorrectly.

- I've invited for this episode someone I regard very highly not only as an expert in market research but as a manager and a friend, someone from whom I have learned a lot. His name is Nino Ninov, and he and I worked on the same team for many years focusing on unlocking new value for one of the best known educational brands of that time, Rosetta Stone.

- When it comes to research, Nino has done it all. Even the more obscure parts of research that most folks haven't heard about. I'm excited to welcome Nino on our virtual stage today. Let's get started.

- Hi, Nino. Thanks for coming to my podcast today and joining me to speak about market research. I'm really happy to see you again. How are you doing? How are things going?

- I'm very good, thank you, thank you and thanks for having me.

- Yeah, absolutely. I know we've been talking about this for a while and I'm really excited to have this finally taking place. Actually, as a quick introduction, I was wondering if you could tell me and the audience on this podcast how you got involved with market research to start with. Like what brought you into that and what your journey has been to where you are so that people understand a little bit of the context of why you're approaching certain things the way you are when you describe the questions I'm gonna ask you.

- Yeah, of course. So it's a combination of factors that led me to where I am today. Started maybe with my first master's which was in sociology, and where I learned a lot of the same techniques that are used in marketing research. We studied questionnaire design, writing questions, focus groups, analytical techniques, statistics, et cetera, so I was fully prepared. And after I graduated, actually a few friends of mine started a marketing research company. They invited me to join them, I did. It was a lot of fun. Later when I came to the United States, I went back to business school, got a degree, an MBA. But after that, I went back to marketing research because I'm fascinated with the whole process of extracting insights from a sea of data that sometimes looks incomprehensible and difficult to make sense of. But if you are diligent, if you use the right techniques, you can derive truly, interesting and sometimes very unexpected insights, and that's a fascinating process. So after the business school, I worked for a year for the Marine Corps as a research analyst. Then I went and worked for Rosetta Stone for 13 years where I was the head of consumer insights and marketing data science. After this, for a while I was a consultant focused again on researching data science, and then worked for a year at Wayfair and now I work for Gaia Herbs, where I lead the consumer insights and marketing analytics function.

- Great. Thanks for walking us through your experiences. That definitely makes it easier for people to understand what attracts people to market research and the type of background that they have. Also you mentioned all your different engagements that you've had with different companies. I have put a little marketing plug here at the bottom of the screen, a call to action for people to come to cerebrations.info to learn more about you and maybe find out how they can get in touch with you if they have any questions and they wanna talk to you more about market research. You kind of started talking about this a little bit in your introduction. You talked about what market research does and why it's fascinating to you and how it attracted you, but I'm gonna ask you again, why do we need market research?

- Like I hear it and quite often, especially from managers who have a lot of experiences in certain field, they kind of feel that they've learned a lot and it's true. They do, they have learned a lot and it's valuable experience, but it's not systematic. It's not by using scientific methods. I mean, if I can kind of make a comparison, but as consumers, when we buy products, we do this based on our experience and sometimes it's significant experience, but it's always kind of composed of many kind of encounters with different products, with different companies. But as consumers, we don't research hundreds of companies across hundreds of products, run across hundreds of different attributes, we don't conduct analysis using regressions, et cetera. We don't do that. Well, it's similar in a business setting when the experience that you have acquired as a business manager is not acquired using these systematic approaches that I just described and using these systematic and scientific methods. So the chances are that you learn a lot but your knowledge is not complete, it's partial, it's sometimes biased in ways. And that's why you need marketing research to provide this more holistic and unbiased picture of the world that you are interested in.

- Yeah, so just to kind of recap, if I understand correctly it's mostly about formalizing the approach, about having a process and someone who's doing it in an unbiased way and systematic way, not just as one of the many things that they do throughout the day.

- Yes, it's about a systematic approach. It's about making sure that you use the appropriate techniques also to answer the questions that come to you. So the experience that you have as a business manager will be very valuable, but it has to be complimented with a more systematic and scientific approach in order to fully understand and be able to answer the questions that kind of are a problem, the business problem that you encountered.

- Okay, I got it. Well, actually, my next question kind of relates to that, but also some observations that I often see myself where the moment we start talking about market research everyone thinks that it's just a matter of putting together four or five questions in a quick survey and running it. And basically, I always joke that because market research is not rocket science, it kind of goes in the opposite extreme where everyone thinks that they can do it, everyone thinks that they could be a market researcher. So what's the pitfall or what are the pitfalls of do it yourself, doing it at home market research?

- The pitfalls are similar to the same pitfalls or the pitfalls that you're gonna fall into if you do yourself home improvement projects all the time. Like I can do some things. I can change a bulb or repair an electric outlet, but recently we had to change the entire electrical system to get it up to code, and we called the professionals in order not to burn down the house. It's kind of a similar thing in marketing research. On the surface it looks easy, but there are a lot of pitfalls. Some of them, for example, are not properly defining the population of interest and surveying the population that you cannot generalize from because your target audience is somewhat different, or not using the appropriate research techniques because you don't know all of them, but for different types of questions, there are many different research techniques that are most appropriate. One pitfall is to misuse actually complex research techniques like conjoint analysis, which on the surface looks easy, but it's not that easy and has a lot of flavors. Another one is not to use properly the powerful capabilities of the modern survey platforms. They're so complex nowadays that even marketing researchers don't have enough experience, let's say in Qualtrics, often cannot use the full extent about the capabilities out there. So there are many things. And as we talk about the different kind of phases of research, we gonna mention more and more kind of these pitfalls.

- Okay, yeah. You mentioned conjoint and that's a very interesting part of research that most people may have not heard about. I'm gonna get back to that in a little bit. I'm gonna ask you about the different types of research, but one thing that I wanted to ask you before that, because you started going in that direction as well answering my current question is you said you have to have the right person to do that for you. You can't -- like you said the analogy of you fixing a light bulb at home, but then having to do your entire circuit, that's a different story. You need someone to do it, otherwise your house may burn down. So if we think about a market researcher, for those of my listeners here who are currently thinking about doing it, but they don't know where to start, and they're trying to figure out what kind of person they need for whether to hire as a permanent like a full-time employee or maybe as a consultant for a project, what do they need to look for when they select a market researcher?

- There are three or four characteristics of the good marketing researcher that they need to look for. One is technical skills, it's obvious. But you can look at their education, you can look at their prior experience, you can ask them to give you samples of prior research that they have done to make sure that they possess the technical skills that are needed. Another one is business knowledge. If you're gonna do marketing research, you should have good knowledge of what marketing is and how marketing is done. Another one is industry knowledge. It's not absolutely necessary for example, for a researcher to be an expert in a particular industry, but it helps if they have some experience doing research in these industries. Every industry is idiosyncratic in ways and marketing is done differently so that helps a lot. And then kind of personal and professional integrity and trustworthiness. Eventually stakeholders can learn about that by talking to the researchers and talking to their previous clients and previous stakeholders. And an important one is to select a researcher that knows how to push back, not just to be an order taker. Often managers, especially managers with a lot of experience and especially people who come from consulting, they know a lot about research and they come and they tell the researcher, "Oh, these are the methods that you should use," and they even write the questions for the researcher. And researchers who don't push back on that, they're just order takers and their research may not be and usually is not as good as it should be. So these are some of the important characteristics of a good marketing researcher that people should look for.

- So basically the last one is don't hire someone who is there to please you, hire someone who's gonna tell you what needs to be done.

- Absolutely, absolutely. Hire somebody who is ready to push back and who will question. And don't get upset about that, actually this is good thing.

- Yeah, okay. Well, actually that segue is also a natural question that I was gonna ask you, how can we -- those of us that are looking to hire market researcher ensure that we are good clients. I mean, you kind of said a little bit of that, but what else do we have to look out for to stop ourselves from doing so that we're not biasing or in any other way impacting the research negatively.

- That's a great question actually. The good client for marketing research first and foremost is intellectually curious and is ready to give up knowledge that is wrong or outdated. And they accept surprises. Sometimes the research will deliver great surprises to them, they don't reject them intuitively, immediately, instinctively, but they think through them. They may not accept them at the end of the day, but important thing is to kind of go through the process of thinking about them. Also a good client for marketing research is the one that engages, not just kind of send the questions and tell the researcher, "Come later, give me the answers." You gotta be ready to invest time, especially in the early stage, the first stage of when the objectives are defined and the approaches and the methods are defined. This is a very important stage when you should be ready to invest time, and if you don't the research will not be as good as it could be. So these are some of the most important things that a client should kind of expect them to deliver and kind of contribute to the research in order for the research to be successful.

- Okay. It's interesting because I mean, I can speak from my own experience, but what I see a lot of times is there's a lot of... It kind of goes in kind of several cycles like up and downs in terms of the sentiments within the organization before they agree on doing research. And again, I'm saying this from a point of view of when I am involved with a company that is not very good at research, a company that's not very used to doing that. Not a company that's actually has formalized the process and it's almost given for that. And what I see is initially there's resistance. It's like what we discussed earlier, I don't need that, I know everything about the markets. Then there is excitement because everyone agrees. Okay, so we have a question that we don't know or several questions so let's get all super excited about it, and it takes some time to start the project. I mean, it takes time to find the right agency to work with or even the panels, whatever you need to procure for your survey or for focus groups, et cetera. And during this time because there's a low and no one is doing anything outside of the person who's involved in the projects hundred percent, everyone starts kind of backing out of it and they forget about it and it becomes something that's on the back burner, second thought. And then when the real research happens, almost no one gets engaged. And then there is a report and it kind of ends there. So it's interesting that you mentioned that it's important for everyone to be involved, to be inquisitive, to go back to the researcher and help with that. I just don't see it, but I wanted to make a comment that's something everyone should be aware of.

- Well, it's also responsibility of the researcher to keep people engaged. I mean, you can do this. You can establish the expectations early on in the first meeting when everybody's still excited. Kind of lay out all the steps in the process where people should be engaged and make them understand that without this engagement this research will not come to a good end. So if the expectations are set right at this early stage, people usually participate and engage, they do.

- Good. Well, actually you're reading my mind because you started already talking about the early stage of the project and what the steps are there. So let me kind of expand it to the full market research project. What are the key steps? What are the key stages in market research project?

- So the most important, the first one is when you define the objectives. In my experience, everything starts usually with one or two truly interesting and important questions. And then in this first stage, as you said, everybody gets excited, more stakeholders get involved. What happens is that questions keep getting added and added and something happens that I call mission creep, and you end up as a researcher with a long list. So it's very important to... This is where the researcher should push back and ask again and again, what is really important, what is good to know, what is nice to know, and try to limit the scope of the research. The best research projects are with problems, research problems that are well defined and identified. This means that when you write objective and then anybody who reads it, they interpret it the same way. There is no ambiguity there. Also you accounted for when you wrote all the objectives for these adjacencies, so you cover everything that is needed to be addressed in order to answer the question so that you don't leave anything outside. In a way you kind of limit the scope, but at the same time you make sure that the scope is big enough to answer the questions. And one of the characteristics again, of a good client is that they are able to separate the truly important from the nice to have and good to have, and to accept that some questions won't be answered. So that's kind of the first big step in marketing research. The second is to define really well the population. Talk as a marketing researcher, you gotta talk with your stakeholders and clients to truly understand who they're targeting for, whom they wanna learn, and then you define properly the population of interest. Often mistakes happen there especially do-it-yourself research. And then you have to select the appropriate research method. And on the the first stage, the very first stage, this is simply going through past research and existing data. Often people just jump to conclusion, let's just go and do a new research, let's go have another survey focus group. But you have to go to reviewing past research and sometimes reviewing data. If you are doing a pricing study, for example, and people are interested how sales volume reacts to changes in prices, well, maybe there is existing data or a sales and marketing data and pricing data that you can use instead of going and kind of conducting a primary research project. But if you go through this first space and the past research and the existing data cannot answer the questions, then you go and say, all right, let's now think about the primary research. And then you run into the next big problem, what kind of research is it?

- All right. So we've covered the why and the how. And several times we alluded to the types of research and I'm sure people are dying to find out what is actually research? The what after we covered the why and the how. Before I get there though, I just want to get back just a few moments ago to something you said because it kind of resonated with me. You were talking about as one of the steps that are important part of the project, that you need to define what are the targets that you want to go after. And one observation that I see more on the B2B side, especially on the enterprise B2B, like the really large enterprise sales is that there is a tendency by the sales teams and by the CEOs to go after the leaders in the industry. To basically say I just want to talk about the largest players in... So don't find anyone, don't recruit anyone in a focus group that doesn't represent one of the top five or top 10 organizations in my industry because I wanna find out what these people are doing. They're the leaders and they're the only ones that I care about. And same by survey, from a survey perspective just target those people. Get to find out what people within these organizations do. And to me, that's not representative of the full market. Yes, they are the leaders, but like we all know they're leaders, they're laggards, and there are people in between. And the market as a whole is much bigger than just focusing on any of these groups. Si I just thought it resonated with me a lot when you were talking about this, but do you have any comments about this? Like any suggestions on how to address that?

- In general, it's a fine balancing act because you have to answer the questions for a particular audience, that is the target audience. But at the same time you don't wanna hyper target because then you're gonna miss a lot. You're not gonna talk to people who are in the market and they have a different perspective. Sometimes when you talk with second tier or third tier participants kind of from smaller companies, you're gonna learn about how they see the big companies and how they think about attacking the big companies, and that's valuable insight too for you as a seller in this market.

- Yeah, yeah. No, I actually I can relate to that too. We just had recently a focus group and one of the participants in that represented a very small organization compared to everyone else. Luckily she was not intimidated by this at all. She admitted very early on she represents a much smaller organization, but she had so much contributions to this that most of the times the others were agreeing with her and in a way she was leading the conversation. So I don't think just because she's constrained by what's happening in her organization by size doesn't necessarily mean that she doesn't have the views of the others and I think it's very powerful to have that conversation as well.

- It is very powerful, it's powerful to have. And then you kind of mentioned this one, talk about focus groups, diversity of opinions, different perspective.

- Yeah. All right, so let's get to questions that we've been alluding to a few times. What are the main types of market research? What's the what in market research?

- So we kind of all of us intuitively know this. I mean, there is quantitative and qualitative research, there is mixed methods when combine the two. The main difference basically between quantitative and qualitative is what type of questions you're trying to answer. When you start with the questions, and most of your questions as a researcher that you get from the stakeholders is what percent of the people like this versus that? What is the share of this versus that? How many? When you hear these type of questions, this is obviously quantitative research. And the qualitative research mostly answers the what and why questions. When you hear the question, what people do in order to, for example, learn foreign languages? Or why they engage in foreign language learning? This is the type of questions that call for qualitative research. The reality is that often the questions will be mixed. There will be first type of questions and the second type because people wanna know what people like and then they wanna know why do they like it? Why do they prefer this versus that? So most research projects really you have to use several different methods. I mean, I've reviewed different types of projects and I've done projects in which we have to do focus groups and IDIs, individual interviews, and some ethnography, and user experience, and surveys, and several different types of surveys using several different type of research techniques. But kind of you start with these questions and then based on the questions that you get as the researcher, you determine what types of research is either qualitative, quantitative. And then again, within each one of them, what are the most appropriate research techniques?

- It's interesting. One thing that I always kind of keep in mind and bring up in a conversation where someone asks me when do we do qualitative? When do we do quantitative? And I wonder if you agree with me or not on this, but the way I see it is a lot of times I would love to get the power of statistical significance by doing a survey. I mean, I'd like to know the mass numbers to be able to say the percentages. But before I get there, I need to know what questions to ask, what questions will make sense. I mean, if it's a new area for me, let's say we're going after a new segment or a new product, I don't know if I'm asking the right questions to get these mass answers. So it's probably better to do a qualitative first, so have the luxury to ask people, to find out what's right and what's wrong. And once we identify these, then go and do the mass survey to support that.

- You're absolutely right. I mean, that's some of the common sequences in research, especially when you do something in the new area, new types of questions, new industry, you launch new products. You start with let's figure out the right questions. So you may need to conduct some individual interviews or often it's good to conduct these face focus groups. Let's say you wanna learn about all the possible strategies that people employ in order to learn a foreign language. So research settings could be focus groups or individual interviews, and you have to still interview enough people. The fact that it's qualitative doesn't mean that you can do five people and you're done. There are some guidelines here, for example, about 10% of the adults in the United States engage in foreign language learning. So if you wanna elicit a complete list of all the different strategies and techniques that they use to learn foreign language, you have to interview 30 people, then you're gonna elicit most likely all of them. If you interview only 10, you run about 35% risk that you're gonna miss an important strategy that people use, but you are not gonna get it because you interviewed too few people. So that's another instance where there is specialized knowledge here and you are better talk to a professional. How many people I need for my focus groups? How many people I need for my individual interviews?

- Okay, I see. So when you talked about different types of research, you mentioned a few other like more aesthetic concepts and names. Also earlier on you talked about conjoint. So let's actually if you don't mind, can we just dive a little deeper into the conjoint analysis? Because I think it's interesting and most people don't know about it, or at least they don't know that this is the name for it.

- Yeah, absolutely. So conjoint is research techniques that is very different from the standard questionnaires. Let's say in the standard questionnaire, survey questionnaire, you gonna ask a question about attributes of different products and you basically ask people, so these are the 20 attributes that describe a particular product. Now, rate each one of them on a scale from one to 10 how important are they to you, and they start rating. The reality in real world, consumers don't do this. We put them in a situation that is atypical, that doesn't happen in the real world. What do they do in the real world? Well, they make a choice. That's exactly what conjoint does. In conjoint you ask people to make choices between several products that you describe with these attributes. And people are very good at making choices. They do it all the time. They do it intuitively on the back of their head often subconsciously they kind of evaluate the different attributes, the different criteria, how they're important, and then they jump to a conclusion, they make a choice. And then you can derive with usually some type of regression latent or vision or latent class regressions, the relative importance of the different attributes. You don't need to ask directly people, but you put them in a more realistic choice situation, which is kind of replicates the real world choice situations between different products and then use derived importance techniques to derive the importance of the different attributes. It's very good for exactly product research, product development research. It's very good for pricing research, where asking people specifically about prices doesn't produce if you ask directly, doesn't produce the best results, especially in some categories that people don't wanna be perceived as oh, I'm cheap, I am saving on something that is very important. They'll give you responses that are not very realistic, conjoint can help there a lot. And keep in mind, on the surface conjoint is relatively easy, especially if you use one of the advanced research platforms. They kind of help you a lot to quickly construct the conjoint, but it is not as easy. I've been to conferences where power users of conjoint. And I'm a user of conjoint, I've used it for 20 years regularly. And in these power conferences, power-user conferences, I kind of hear things that I've never thought about in the ways you can use the technique in order to kind of get deep insight. So use it, please use it and use professionals, work with professionals, particularly people that specialize in conjoint.

- Yeah, okay. Thank you. I guess the layman description, one sentence description that I do for conjoints when I talk to someone and I don't have a lot of time, I know they're not gonna spend a lot of time thinking through it is I always tell them if I give you the option to buy a car and I give you five price points and I give you five options, you're probably gonna say that you want all the five features of the car and you wanna pick the lowest price. You're always gonna have say that. Why would you say anything else? But it's not the reality, that's not what happens in real world. So how do you tease out these trade offs then, that's what conjoint does.

- Exactly. Conjoint is particularly good examining these trade offs that people do make in the real world when they trade between oh, I want this feature but that's the price, and then I gotta think about what's more important for me. And conjoint can reflect this situation and captures it well.

- Yeah, absolutely. Okay. So one other question that I had that goes a little deeper into the types of research, we mentioned focus groups and in-depth interviews, IDIs a lot. So sometimes I think for most people it's hard to determine when they should do one versus the other. And at least what I've seen as well when I talk to people that are not experienced with research is they have a tendency to say, "All right, let's just do a focus group because it's easier." I mean, I'm gonna just do it once. I'm not gonna spend... You mentioned earlier 30 interviews, 30 subjects, that's like 30 one-hour commitments from someone's time. It's hard, it's expensive, it's hard to coordinate if you have more people in the room. For focus group, you can get seven or eight of these folks in one setting. So when do you use one versus the other? Or like what's the main deciding factor for one versus the other?

- The main reason for using individual interviews, there are several reasons. The main reason is when you really wanna dig into the why, why people do something. In individual interviews you often use laddering techniques in which you kind of start, and when you start asking people why do you engage in this activity? Why do you prefer this product, for example, versus another product? And often the first thing that we hear it's kind of surface level ready answers that people have. And if you wanna dig behind that and go deeper and try to reach to the deepest kind of deep seated motives why people engage in certain activities, then the individual interviews are a better way than the focus group. So learning about motivations especially at the very deep level, individual interviews are the best method. Allows for probing, allows for kind of a lot of creative work from the interviewer. You need experienced interviewers who are capable of taking full advantage of these techniques. Focus groups are particularly good when you try to answer the question of what people do in order to accomplish something, learn a language, or stay healthy and fit. And one of the most valuable features of the focus groups is the social dynamics that is happening in the focus group. I've heard opinions from professional researchers that they wanna kind of eliminate the social dynamics because it interferes with surfacing the true beliefs that people have, that otherwise people kind of looking around the room will be impacted by other people, will not tell you really what they think, what they like and dislike. But the reality is that in the real world, the way we perceive products and evaluate them or marketing messages, it happens in social environment. It happens in conversations with other people. They influence us, we influence them. And the focus groups that the social dynamics of the focus groups is an opportunity actually to replicate this real world experience and capture it in the focus groups. So that's why in the good focus groups there is a lot of conversation between the participants in the focus groups, not just answering the questions one at a time, the questions that the facilitator asks that kind of talk to each other, And a good facilitator will engage them in this conversation, will ask them to comment on each other's opinions in a safe and nonjudgmental environment.

- Yeah, actually it's a very good point. I just saw it again in one of our recent focus groups where actually the person who was supposedly the most experienced in that field, coming with the most experience in terms of years but also working for probably the most renowned of the three organizations represented, he initially dismissed one of our hypotheses, he was against it, he thought that this is not gonna work out well. And then as the conversation progressed and other people got involved, the other participants got involved and they started sharing their experiences, they actually started bringing up a lot of reasons for that idea, why they think it's good. And to his benefit, he came back after that and said, "You know what, I know initially I said that this is a bad idea. But now that I heard everyone and it's not because I hear it from you guys, you're asking these questions, you have an agenda, but I hear it from my peers, I am actually changing my mind. I actually agree with what they're saying." And that's something that you're not gonna get from an in-depth interview 'cause they're just gonna talk about the things that they believe and what they hear.

- So, I mean, that was a great example of what I just talked about, that the social dynamics of the focus groups kind of providing additional insights that otherwise you cannot get if you just interview one person at a time.

- Yeah, great. So my final question that I always ask, and it's basically because I want all these podcast episodes to be entertaining, but at the same time also educational. And the most important thing about an educational one is to retain some of that, whatever you hear on these episodes, what are you gonna remember a few months later if you ever remember that you had listened to this episode? So Nino, if you had a way to influence our listeners now and your wish was that they would retain something, what would you like them to retain from this? What do you like them to remember the most?

- So for business managers, marketing managers who listen this, be involved in the research process, stay engaged, work with the researchers, use professional researchers. It's a profession with a lot of technical skills. It's becoming increasingly complex to the point when even within, as I said, within conjoint, for example, there are so many options that you have to work with the professionals so people could truly kind of understand and know all the options and have the necessary skills. And be open minded, that's extremely important. In the research you're gonna get counterintuitive insights that go against the grain, that go against the existing beliefs. You don't need to accept them right away, but be ready to discuss them, don't reject them immediately. And the best marketing people that I've worked with had these open minds and they were ready to discuss these counterintuitive, and they were ready to change course when necessarily based on the research techniques.

- Great, all right. Well, thank you very much. This has been a very insightful conversation. We ran a little bit over our time, but I think overall it's been very useful. I hope it's gonna be very useful for my listeners, but also for me just as a refresher about all my experiences that I've seen in the past with market research to kind of put everything in context. So thank you Nino for joining me today and for your time. And I hope we get to speak again later in the foreseeable future, speak again soon.

- It was a pleasure, Emil.

- All right, thank you.

- All right.

- Bye-bye.

- Bye.