💡
EA market testing (public)
  • Introduction/overview
    • Introduction & explanation
    • 👋Meet the team
    • 📕Content overview
    • Progress/goals (early 2023)
      • EAMT progress & results
      • Goals, trajectory, FAQs
  • 🤝Partners, contexts, trials
    • Introduction
    • Giving What We Can
      • Pledge page (options trial)
      • Giving guides - Facebook
      • Message Test (Feb 2022)
      • YouTube Remarketing
    • One For the World (OftW)
      • Pre-giving-tues. email A/B
        • Preregistration: OftW pre-GT
    • The Life You Can Save (TLYCS)
      • Advisor signup (Portland)
    • Fundraisers & impact info.
      • ICRC - quick overview
      • CRS/DV: overview
      • 📖Posts and writings
    • University/city groups
    • Workplaces/orgs
    • Other partners
    • Related/relevant projects/orgs
  • 🪧Marketing & testing: opportunities, tools, tips
    • Testing Contexts: Overview
    • Implementing ads, messages, designs
      • Doing and funding ads
      • Video ads/Best-practice guidelines
      • Facebook
      • Targeted ad on FB, with variations: setup
    • Collecting outcome data
      • Facebook ads interface
        • Pivot tables
      • Google analytics interface
      • Google A/B, optimize interface
      • Reconciling FB/GA reports
      • Survey/marketing platforms
    • Trial reporting template
  • 🎨Research Design, methodology
    • Methods: Overview, resources
    • "Qualitative" design issues
    • Real-world assignment & inference
      • Geographic segmentation/blocked randomization
      • Difference in difference/'Time-based methods'
      • Facebook split-testing issues
    • Simple quant design issues
    • Adaptive design/sampling, reinforcement learning
    • 'Observational' studies: issues
    • Analysis: Statistical approaches
  • 🧮Profiling and segmentation project
    • Introduction, scoping work
    • Existing work/data
      • Surveys/Predicting EA interest
      • Awareness: RP, etc.
      • Kagan and Fitz survey
      • Longtermism attitudes/profiling
      • Animal welfare attitudes: profiling/surveying
      • Other data
    • Fehr/SOEP analysis... followup
      • Followup with Thomas Ptashnik
    • Further approaches in progress
      • Profiling 'existing traffic'
  • 📋(In)effective Altruistic choices: Review of theory and evidence
    • Introduction...
    • The challenge: drivers of effective/ineffective giving
      • How little we know...
    • Models, theories, psych. norms
    • Tools and trials: overview
      • Tools/interventions: principles
      • Outcomes: Effective gift/consider impact)
        • (Effectiveness information and its presentation)
        • (Outcome: Pledge, give substantially (& effectively))
          • (Moral duty (of well-off))
        • Give if you win/ conditional pledge
      • Academic Paper Ideas
  • Appendix
    • How this 'gitbook' works
      • Other tech
    • Literature: animal advocacy messaging
    • Charity ratings, rankings, messages
    • "A large-scale online experiment" (participants-aware)
  • Innovationsinfundraising.org
Powered by GitBook
On this page
  • Summary
  • Pre-trial reporting template
  • General idea, main 'hypothesis' (if there is one)
  • Point of contact (at organization running trial)
  • Timing of trial (when will it/did it start and end, if known)
  • Digital location where project 'lives' (planning, material, data)
  • Environment/context for trial
  • Participant universe and sample size
  • Key treatment(s)
  • Treatment assignment procedure
  • Outcome data

Was this helpful?

Edit on GitHub
Export as PDF
  1. Partners, contexts, trials
  2. Giving What We Can

Message Test (Feb 2022)

PreviousGiving guides - FacebookNextYouTube Remarketing

Last updated 2 years ago

Was this helpful?

Summary

Main Question: Do some message themes work better than others for drawing visitors to Giving What We Can’s landing page?

Main findings: 'Social proof messages' on Facebook ads were most effective at generating landing page views per dollar compared to other message themes (effectiveness, services, giving more, and values).

Future directions: There were significant differences in 'link clicks per dollar' on the different messages by age. We recommend a systematic test to determine if age makes a difference in the relative effectiveness of social proof and values messages. Future studies could explore why the social proof message was more effective in this study than the previous giving guide study and the importance of the message to “join” the movement as social proof.

Possible connection between this trial and the Giving guides - Facebook: Note that the two best-performing messages both prompted the user to “join” a movement or a group of people (perhaps an elite group); but beware .

to report below.

Pre-trial reporting template

General idea, main 'hypothesis' (if there is one)

In this test, we are aiming to find out if one 'theme' of messages resonates better with our target audience than others.

If we knew which 'themes' were most effective with our advertising, then we could create more ads on this theme and improve our conversion.

Specifically, which of the following themes resonate with our target audience the most:

  • effectiveness

  • giving more

  • social proof

  • values

  • services

On choosing an objective of this test, originally I planned to use link clicks, but this is not the most high quality indicator of conversion, and when I tried to use newsletter signups Facebook warned me that I might not see any conversions at all... So instead, the campaign will optimise for landing page views, which is slightly better than a link click and will generate enough conversions that we should [see?] we statistically significant results.

Point of contact (at organization running trial)

Grace Adams

Timing of trial (when will it/did it start and end, if known)

Trial will run for 7 days on GWWC's ad account, from 9.30am AEDT Friday 25 Feb to 9.30am AEDT Friday 4 Mar.

Digital location where project 'lives' (planning, material, data)

Environment/context for trial

This test will take place on Meta platforms including Facebook and Instagram

Participant universe and sample size

  • We are targeting a "Philanthropy and Lookalikes (18-39)" audiences, based in UK, US or Netherland

  • Estimates from Facebook: Reach is expected to be 1.4K-4.1K per day (7 days) per ad set (5 ad sets) = 49K-143K

  • Estimates from Facebook: Conversion is expected to be 10-30 landing page views per day (7 days) per ad set (5 ad sets) = 350-1050

Key treatment(s)

We are using the GWWC Brand Video by Hypercube as the creative across all tests. Although it did not perform as well as our other ads in the Giving Guide campaign, I think that it will interfere less with our messages we aim to test.

Mock up of ad:

Treatment assignment procedure

  • This test has been set up as an A/B test through Facebook, testing each campaign head to head, each campaign covers one theme, with the different ads as a child.

  • This will allow us to test which theme was better, not just which individual ads

  • A/B testing on facebook will ensure that the audiences fall into an individual treatment group

Outcome data

  • Primary measure will be cost per landing page view, but secondary measures such as CPC, 3 second video plays, email sign ups will also be tracked

  • Data will live on Meta ads platform


Working document can be found but all important details will be listed in this brief

We are going to test a set of messages for each theme, please see them in the

🤝
here
google doc linked
ex-post theorizing
Link