Resource

Campaign Message Testing: A Practical Guide for Impact Teams

ÂLI OKUMUŞOĞLU

Campaign message testing means putting your draft messages in front of the people you want to reach, before you spend on production and media, to learn what they understand, believe and are moved by. Done at two points, early on the concept and later on the finished content, it is the cheapest way to avoid funding a campaign built on assumptions. A strategy-first partner can design the test, read the results and turn them into a sharper campaign, which is how we work at Dante.

TL;DR

  • Message testing gets feedback from your intended audience on a strategy, copy or content to check it gets the response you expect (ICPA toolkit).
  • Test twice: at concept level before you pay for production, and at content level once visuals and video exist (ICPA, 2023).
  • Use interviews and focus groups to understand why; use surveys and A/B tests to compare options.
  • Test one element at a time, and read small-sample results as a guide, not a forecast.
  • The first goal is to find what does not work, including messages that backfire.

Table of Contents

What message testing is, and what it is not

A campaign strategy sets the purpose, the audience and the narratives. Message testing is a distinct step inside that cycle: it checks which narratives or specific messages resonate with a given group, usually after background research, audience segmentation and a design phase (EU RANPS paper). If you are still shaping the wider plan, our guide to behaviour change communication covers that stage.

The point is humility. A programme’s objectives are not the same as the message most likely to achieve them, a warning quoted in the IFRC’s messaging guidance, which also quotes the view that one should no more release an untested communication than an untested product.

What message testing is, and what it is not

What to test, and when

Commonly tested items include pitches, narratives and messages, stories and protagonists, slogans and hashtags, visuals, video and website content (ICPA). The PIRC and ILGA-Europe guide suggests isolating one “active ingredient” at a time: a value, a metaphor, a messenger, an image or a touch of humour. Change three things at once and you will not know which one moved people.

Timing matters as much. Test the concept before large production spend, then test the content once it exists. ICPA’s line is blunt: any testing is better than no testing.

The WHO adds a useful discipline: make sure the message is technically accurate and approved first, and be clear what it should achieve, whether awareness, support, behaviour change or busting a myth. That matters for scientific and policy-heavy messages, where a catchy line must not drift from the facts.

Choosing a method

The Busara guide for civil society organisations separates formative research, which helps you understand, from summative research, nearer the end, which gives more conclusive answers.

Method Best for Stage
In-depth interviews Why people react as they do Any stage
Focus groups Spotting confusing or backfiring ideas Formative
Surveys Comparing messages across a segment Formative or summative
A/B tests Choosing between live variants Summative, or ongoing

Online A/B testing can run before launch and keep improving content during a campaign (EU RANPS paper). Within surveys, a monadic design shows each respondent one message and gives a more defensible, benchmarkable answer. A comparative design is fine for an early directional check, but respondents anchor to whichever message they saw first or last (SurveyMonkey).

A step-by-step routine

  1. Write a testable question. Link one feature you are testing to one expected outcome, and decide whether the research is formative or summative (Busara).
  2. Define the audience. Segment by age, sex and culture, as the WHO suggests, and by the channel and messenger they trust.
  3. Draft a few clearly different messages. A message map helps: what you want people to think, feel and do, a main statement from their perspective, and supporting proof points (US EPA workbook).
  4. Run the test. Use a script that probes comprehension, strong and weak points, personal relevance and anything confusing or sensitive. Avoid leading questions (CDC guide).
  5. Read for what fails first. ICPA argues the first goal is to find what is not working, because messages built for unfamiliar audiences often rest on assumptions that do not hold.
  6. Refine and retest. Fold the lessons into strategy and creative, then launch and evaluate.

This is where an experienced team earns its place. Writing sharp variants, framing questions neutrally and translating findings into a revised narrative is the work our strategy and campaign teams do for impact organisations.

How many people do you need?

Sample size depends on the method, and the guidance varies:

  • Small-scale checks: the CDC suggests two focus groups of four people, or a “5-5-5” survey of five people, five questions, five minutes.
  • Qualitative saturation: around 9 to 17 participants is often enough to stop hearing new themes (SurveyMonkey).
  • Focus groups per segment: ICPA suggests at least two per audience segment.
  • Quantitative scores: at least 100 respondents per message in a monadic design, more for subgroup comparisons. ICPA uses 1,000, the typical national poll size, as its standard.

Bigger is not automatically better. The OECD cautions that lots of data from one type of group can magnify sampling bias, and that very large samples can flag tiny, inconsequential differences. Small qualitative samples guide decisions but should not be used to predict how the whole audience will behave (US EPA).

What testing looks like for NGOs and nonprofits

Engagement is not impact. In a study of Facebook split tests for a conservation organisation, wild dog stories drew more traffic to the donation site than the classic elephant story, yet only the elephant video produced a donation. The lesson: test the outcome you care about, not only clicks.

Prior beliefs can also mislead. A direct-mail experiment found that adding scientific impact information had no average effect on giving, though large prior donors responded differently from small ones. Segment your results.

And where emotion is high, interviews catch what surveys miss. The PIRC guide shows how a well-meant line can trigger the opposite of the intended reaction.

To measure results, track inputs, outputs, outcomes and impact, and prefer actual behaviour to self-reporting where possible (US EPA workbook). If you want a broader frame for stories, see our nonprofit storytelling strategy, and for common wording traps, three brand messaging mistakes to avoid.

What testing looks like for NGOs and nonprofits

Testing inside a wider plan

Testing is one stage of a cycle that runs from purpose to evaluation. Placing it well, and tying it to reporting, is easier when the whole communications plan is in view. The ESG communications strategy guide is a useful companion for sustainability-led organisations connecting message choices to a broader plan.

Our perspective

We work only with organisations creating positive impact, and the pattern we see is consistent in how we approach it: the strongest campaigns start by being honest about what the audience does not yet believe.

For us, testing is not a research add-on. It is how strategy becomes precise. A message that is clear to your team can be invisible, confusing or even alienating to the people you need, and a small, well-framed test is the fastest way to find out.

That is why we pair strategy with creative and delivery. The same team that drafts the narrative can plan the test, read the reaction and sharpen the campaign. Complexity is our daily material, and our job is to hand you clarity you can act on.

Ali

Bring us your draft messages

If you are about to launch, apply for funding or brief an agency, Dante’s impact campaigns combine strategy, creative and digital delivery for NGOs, EU projects, startups and public institutions. Our communication consulting and content services can help you shape the messages worth testing. Writing a brief? Our guide to writing a communications RFP helps.

FAQ

What is campaign message testing?

It is a process of getting feedback from your target audience on a campaign strategy, copy or content to see whether it produces the response you expect. It mainly informs the formative stage, before the campaign is fully built.

Which message testing methods work best for small nonprofits?

Start with a handful of interviews or small focus groups to understand reactions, then run a short survey or a paid A/B test to compare options. The CDC’s small-scale approach of five people, five questions and five minutes shows that a first test need not be large.

When should we test our messages?

Twice: at concept level, before you invest heavily in production, and at content level, once images, video and copy exist. Paid campaigns can also use the opening weeks to try every option, then pause to review data before committing the rest of the budget.

How many respondents do we need?

For qualitative work, roughly 9 to 17 participants often reaches saturation. For a quantitative score, aim for at least 100 respondents per message in a monadic survey, and more if you compare subgroups.

Sources

Join our newsletter

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
let's launch

Let's talk

UP