3 Primary Themes in Early Product Testing

Erman Misirlisoy, PhD
8 min readJul 6, 2023
Photo by Girl with red hat on Unsplash

As tech companies develop new products and features, they’ll typically want user feedback to revise and validate concepts. Testing product concepts early and often helps to pivot ideas without wasting too many resources. But where should a researcher focus their questioning when there’s not yet an interactive prototype to run a full usability test with?

Some of the most crucial feedback — especially when it comes to testing early concepts — centers around value and basic functionality. Do prospective users agree that a team’s new idea will help to improve their experience?

One of the most widely used research methods for collecting this feedback is a qualitative user interview, but it’s not always easy to keep an interviewee’s attention focused on value. Generic questions such as “what do you most like/dislike about this concept?” will often be met with comments about visual design such as preferred colors or text layout.

Feedback about visual design is of course valuable but it’s usually not the main priority in early testing, especially since designs are intended to be rough and immature. Value and basic functionality are the hardest-hitting questions when seeking early signs of product-market fit.

Based on studying how researchers around the industry prioritize questions in concept testing and how much value my own product teams have derived from research insights in the past, I think there’s a small set of broadly-applicable and high-impact themes to rely on. Although there are hundreds of individual questions you could ask about value and functionality, the most common and actionable ones tend to cluster into three primary domains, which I will outline here.

The CUB themes

In the visual below, you’ll see three simple themes that are relevant from the earliest stages of concept testing: comprehension, utility, and barriers (let’s call this the CUB framework after my favorite baseball team). This generally excludes deeper questions about technical usability or visual design, which might be more relevant for an advanced and interactive prototype. Instead, the researcher might have a few mock screens to demonstrate the functionality of a new idea or even just a pen-and-paper sketch. In any case, the purpose is to get…

--

--

Erman Misirlisoy, PhD

Research Leader (Ex-Instagram / Chief Scientist at multiple startups). Author of the User Insight Newsletter: https://userinsight.substack.com/