Expert vs. consumer sensory panels: best of both worlds?

In this series of articles, NIZO Food Research Division Manager and FoodNavigator advisory panel member René Floris discusses some of the big issues in today’s food industry.

While manufacturers are working to meet market demand for healthier, more sustainable foods, at the end of the day, if consumers don’t like a product, they won’t buy it. It has to meet their sensory requirements for flavour, texture and overall quality. Margreet Rippen, Expert Sensory, and Els de Hoog, Expert Flavour & Texture, both from NIZO, describe the role of sensory panels in product development, the differences between consumer and expert evaluations, and why it is important to pick the right type of panel based on the insight you need.

What are sensory panels, and what insight can they provide?

An expert sensory panel is a group of carefully selected and trained individuals who evaluate and provide sensory feedback on a food, ingredient or beverage. Using a learnt sensory lexicon, they provide an objective assessment of specific sensory attributes, delivering in-depth analysis and sensory blueprints of the sample or ingredient. This insight can help manufacturers make informed decisions about optimising product formulations, selecting ingredients, and identifying how to adapt processing conditions focussing on improving specific desired attributes or, conversely, eliminating unwanted characteristics.

Consumer sensory panels, on the other hand, are generally organised by panel research agencies. They are made up of untrained individuals representing a certain population, who are invited to evaluate the product. The value they provide is normally whether they “like” or “dislike” a product.

How do the sensory panels evaluate a product?

Expert sensory panel participants are asked to taste several products or formulations. They then provide scores for the generated and pre-defined sensory characteristics using a scale or ranking system. For example, they may be asked to score flavour attributes of the product, such as “woody”, “musty/earthy”, “astringent”, on a scale of 0–100, where 0 means they don’t perceive the attribute at all, and 100 means the attribute is very intense.

While consumer panels are usually asked to score each product or formulation on their “liking” for it, today they may also be asked to give their opinions on other product-related aspects such as colour, packaging and cooking instructions. Sometimes, the consumers are asked to score some attributes chosen by the panel research agency.

How are panels trained and organized?

The expert panellists are first trained to describe their personal sensory experience, thus generating attributes for a sensory lexicon for a particular product. A standardised vocabulary ensures that every panel member is using the same word to describe what they perceive in the product. For example, one panellist may note what they consider a “foxy/musky” characteristic, while another finds that same characteristic reminds them of a certain shrub, and another brings up notes of blackcurrant. In the lexicon, all of these might be referred to as “ribes”.

The panel leader is providing explanations to the trained NIZO expert panel at the round table.

They are also trained on how to use the scale system in a standardised way, to ensure consistency and repeatability. This is key for accurate and reliable evaluations, and for differentiating between often subtle differences in products. NIZO maintains a stable panel of dedicated evaluators who work on a weekly basis and participate in ongoing trainings which further increases consistency.

In contrast, a consumer panel is set up per event. The participants are not trained, as they will mainly give their subjective judgement on “liking”. Consumer panels need a higher number of participants (generally 80 – 100 consumers) than expert panels (10 – 12 panellists) to ensure reliable data.

How do the results of consumer panels and expert panels differ?

We recently carried out a study with both a consumer panel and an expert panel to compare the outcomes when evaluating the same six products on flavour and texture. We then performed a multivariate preference mapping analysis on the response datasets and created principal component analysis (PCA) plots for each. The consumers also indicated their preferred product, which was plotted, as well.

On the plot of the results from the consumer panel, all six products grouped together, making it impossible to differentiate the products based on the attributes (which were chosen by the panel research agency). This indicated that the consumers could not truly distinguish the products from each other. Their preference also overlapped with all of the attributes, meaning that no information could be given on why the consumers preferred that product.

The plot from the expert panel, however, was much more spread out, with each product having its own, defined space in the graph. The attributes were spaced widely from each other, indicating a clear sensory differentiation between the products. We could see which products were stronger and/or weaker for which sensory attributes.

Example of a spider plot: Attributes which differ significantly between the pasteurised and UHT heated soy milk are indicated with a ‘*’ (*p<0.05, **p<0.01, ***p<0.001).

Interestingly, when we plotted the preference data from the consumer panel onto the expert panel graphic, we could clearly distinguish the specific attributes associated with the “liking”, and thus advise the manufacturer on how to develop their product to the meet this consumer preference.

What can you do with the data from an expert panel?

The level of sensitivity and detail from an expert panel allows you to go far beyond basic preference: you can identify problems such as quality defects or off-flavours, you can monitor shelf-life by evaluating changes in sensory characteristics of your product over time, you can benchmark your product against others in the market and understand how it is different, and create sensory blue prints of one’s ingredient or product.

You can also use the specific attribute insight to steer product development, in terms of both formulation and processing, to get the attributes you want. For example, if the panel indicates your product has high levels of astringency, you can focus on reducing that, whether by increasing the amount of fat, carbohydrates or sugar, or by using a technique such as fermentation. After all, you need to know what the problem is before you can solve it.

How does a manufacturer decide which panel is better for new product development?

It’s not a question of “better”, it’s about picking the right tool for the job at hand. And when talking about product development, the question is quite challenging. In this situation, you need to create a unique market position, and only expert sensory panels can tell you what is unique about your product. But you also have to match the customers’ desires and expectations. Combining the expert panel data on attributes with customer panel data on preferences could deliver on this goal, allowing you to take concrete measures to steer your product development in the “right” direction.