Methods
In June 2023, we recruited an online convenience sample using the survey research platform Cint as part of a parent study examining the impact of the use of different nutrient profile models in Brazil’s FOPL system. Participants were eligible if they were 18 years or older, resided in Brazil, and were responsible for at least 50% of their household’s food purchases. The panel company used purposive sampling to obtain a sample whose age, gender, and regional distribution was comparable to that of the Brazilian population. The study received approval from the University of North Carolina at Chapel Hill’s Institutional Review Board (#21-2129) and the University of Campinas Ethics in Research Committee. The study design, measures, hypotheses, and analytic plan were registered prior to data collection on ClinicalTrials.gov (NCT #05842460).
Procedures
All participants provided free and informed consent. The parent study comprised a shopping task in a fictional online supermarket, an emerging platform for conducting nutrition research [36, 37], followed by a self-administered online survey programmed into Qualtrics survey software. The current study was embedded into the survey. Both studies were conducted entirely in Brazilian Portuguese. To avoid contamination, only participants originally assigned to the parent study’s control group (in which products did not carry any FOPLs) participated in the present study. This procedure resulted in 1,004 participants in the current study (Fig. 1).
Participants were randomly assigned to one of two labeling conditions using the Qualtrics randomizer function: nutrients-only labels, in which products carried only Brazil’s current nutrient warning labels, or nutrients plus ultra-processed labels, in which each product carried the same nutrient warning labels and all products carried an additional label that read “WARNING: ULTRA-PROCESSED.” Since the ultra-processed warning label is novel to consumers, we chose to use the marker word “warning” to make this label more attention-grabbing [38] while still generally adhering to Brazil’s FOPL visual identity (i.e., color scheme and shapes). Participants saw images of four products in random order carrying their assigned labels: cookies, salty snacks, a yogurt, and flavored milk (Fig. 2). Cookies and salty snacks are among the UPFs that most contribute to energy intake in Brazil [27, 39]. Yogurt and flavored milk are also widely consumed in Brazil [40], and despite often being ultra-processed and high in sugar, tend to be perceived as healthier in studies from several countries [40–44]. To enhance realism, our stimuli displayed real products from established brands in Brazil that had been available in the online supermarket used in the parent study.
Measures
At the start of the survey, participants answered the open-ended question “What do you think the term ‘ultra-processed’ means?” and rated their confidence in their understanding of the term “ultra-processed” on a 5-point scale. These measures were placed before the experiment to capture participants’ prior familiarity with the concept of UPFs without the influence of ultra-processed warning labels on products. Participants then proceeded to answer 30 additional questions for the parent study about their experience in the online store and products displayed in the store, which were placed before the experiment as a wash-out.
Next, after seeing each product, participants rated their intentions to buy the products (“How likely would you be to buy this product in the next week, if it were available in a store?”) as the primary outcome. As secondary outcomes, participants identified whether they believed the products to be ultra-processed or not (“Do you think this product is ultra-processed?”), rated the products’ perceived healthfulness (“How healthy does this product seem to you?”), and rated the perceived message effectiveness (PME) of the warning labels with three items: “How much do these labels discourage you from wanting to consume this product?”, “How much do these labels make you concerned about the health effects of consuming this product?”, and “How much do these labels make consuming this product seem unpleasant to you?”. PME is a measure commonly used in message-development studies and has been shown to predict actual message effectiveness [45–48]. Response options to all outcomes except UPF identification were provided on a Likert-style scale ranging from the lowest value (coded as 1) to the highest value (coded as 5); response options to the UPF identification outcome were coded dichotomously (i.e., “no” or “I’m not sure” vs. “yes”). Lastly, the survey also collected information on demographic characteristics.
Analysis
First, we developed a codebook to code the open-ended responses. This code-development process used a hybrid inductive-deductive approach [49–51] while reviewing around 25% of the responses, guided both by the 2015 Dietary Guidelines for the Brazilian Population’s definition of UPFs [26] and by insights obtained from the data. We captured descriptors used by participants and matched such descriptors to their corresponding UPF or non-UPF characteristics (Table 1). Once the codebook was developed, two Portuguese-speaking coders independently coded all responses and resolved any discrepancies by consensus.
To gauge the level of understanding of the UPF concept in the sample, we classified all participants based on what descriptors they mentioned in their responses (Table 2). Because the Dietary Guidelines for the Brazilian Population define UPFs as “industrial formulations made entirely or mostly from substances extracted from whole foods, derived from food constituents, or synthesized in laboratories,” [26] we considered lack of whole foods and presence of synthetic substances as the two key UPF characteristics in this analysis. Participants who mentioned descriptors related to both key UPF characteristics (i.e., at least one lack of whole foods descriptor and at least one synthetic substances descriptor) were thus classified as having a high understanding of the concept. Next, we classified those who mentioned at least one non-UPF characteristic as having a low understanding of the concept, unless they had also mentioned the two key UPF characteristics. Lastly, we verified that all other participants had mentioned at least one descriptor related to a UPF characteristic (i.e., lack of whole foods, synthetic substances, unhealthy products, or other UPF characteristics) and proceeded to classify them as having a moderate understanding of the concept.
For the experimental portion of this study, given our pre-determined sample size of around 1,000, 80% power, and a critical alpha of 0.05, we calculated that we would be able to detect a minimum effect size of d = 0.15 for the primary outcome (i.e., purchase intentions). We determined that this sample size would be sufficient, given that, on a smaller preliminary study using ultra-processed warning labels, we found an effect size of d = 0.22 on a single-item PME measure [34].
For continuous outcomes (i.e., purchase intentions, perceived healthfulness, PME), we verified that reliability across products was sufficient (Cronbach’s α > 0.7, Table S1) and averaged participants’ responses to each question across the four products. For PME, the only outcome measured through more than one item (Cronbach’s α = 0.92), we also averaged participants’ responses across items. We conducted independent samples t-tests to examine the statistical significance of the differences between labeling conditions. For the identification of UPFs outcome, we created a dichotomous variable indicating whether each participant correctly identified all four products as ultra-processed or not, and then conducted a chi-squared test for significance testing. Additionally, for each of these outcomes, we conducted exploratory analyses (not included in the prospective trial registration) to determine if there were differences between labeling conditions for each individual product. These exploratory analyses used the same statistical testing approach as our primary analyses – i.e., independent samples t-tests for continuous outcomes and chi-squared tests for the dichotomous outcome.
We also conducted moderation analyses to examine whether demographic characteristics (i.e., age, gender, educational attainment, self-reported health status), self-reported confidence in one’s understanding of the term “ultra-processed,” and low understanding of the term “ultra-processed” measured through the open-ended response moderated the effect of the ultra-processed warning label on identification of UPFs and purchase intentions. All moderation analyses were included in the prospective trial registration except for moderation by self-reported confidence in understanding of the term “ultra-processed,” which we added post-hoc to provide a more comprehensive understanding of how results may have differed among participants. We fit a series of logistic regressions for the dichotomous outcome (i.e., UPF identification) and linear regressions for the continuous outcome (i.e., purchase intentions), with separate models for each potential moderator. Each model included the trial arm, the moderator, and their interaction. Moderator variables were dichotomized to maximize statistical power due to the limited number of participants in certain response categories, as well as to simplify interpretation of results. Additionally, to account for multiple comparisons, we performed the Holm-Bonferroni correction on the p-values associated with the interaction terms.
We used complete case analysis to address any missing data, resulting in the exclusion of 7 participants from the analysis of UPF identification, 9 participants from purchase intentions, 8 participants from perceived healthfulness, and 18 participants from PME. Analyses were conducted using G*Power3.1 (power calculations) and Stata/SE version 17 (all other analyses) with a critical alpha of 0.05.
CC BY 4.0. ""Warning: ultra-processed": an online experiment examining the impact of ultra-processed warning labels on consumers' product perceptions and behavioral intentions" by D'Angelo Campos A, Ng SW, Duran AC et al., The international journal of behavioral nutrition and physical activity (2024). https://doi.org/10.1186/s12966-024-01664-w. Licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Peer-reviewed and open access; full text from PubMed Central (PMC11462959). Headings and layout adapted for reading; figures and tables are in the original.