Testing Contexts: Overview

Testing 'implementation strategies'

Contexts to test outreach messages

Contexts allowing individual randomization & tracking of medium-term outcomes

1. GWWC web site at point of email signup

2. Email lists

   • immediate: subject headers w/ 'open rates' as dependent variable

   • medium-term: all outcomes tied to email

Contexts for 'Immediate outcomes' (clicks etc)

Facebook; But the targeting algorithm may frustrate randomization. (see Real-world assignment & inference.) Can it be switched off?

Contexts allowing randomization by geography

See Geographic segmentation/blocked randomization

This is helpful for ease of attribution if the important outcomes can be tracked by ZIP code/post code/address.

• Online display advertising

• Google search

• YouTube

• LinkedIn

• Facebook (presumably)

Testing Rich Content

How to test rich content?

We can use some of the same strategies as above to test "rich content", i.e., short or even long talks, book chapters, podcasts, and so forth.

However, we may also want richer more detailed 'qualitative' feedback...

Paid participants may allow richer feedback (see discussion)

• Emails might be an opportunity

• Surveys with professional participants

• Surveys with undergraduates

Here generalizability may be a challenge, particularly extending inference from convenience samples to larger and more general populations. "Might be good to think of creative ways of doing that though, e.g., looking at which content creates the most extreme enthusiasm."

What to test in 'rich content'

Does the messenger matter?

• Does the messenger demographics and appearance matter?

• Does it depend on the audience?

• What’s the optimal length?

Message customization (heterogeneity and targeted marketing)

We haven’t thought about this much but it seems important – it might be worth, for example, having different messaging for different cause areas and letting them be algorithmically targeted.

Imagery/non-content considerations

How many images to include on a page?

• How much text to include in a page?

• How many buttons?

• How many choice options?

The 'mysterious sauce' ... JS knows about (Video ads/Best-practice guidelines)... we don't always have a "theory" but it might be meaningful.

Targeting

See also Profiling: Discussion

Question: If our aim is to change the culture of giving in general, what kind of people should we be targeting?

1. Influencers (People with lots of social influence)

2. Low-hanging fruit (i.e., people who are naturally predisposed towards effective giving, pledging, & EA)

Idea: Compare different outreach methods on the basis of "cost per pledge" (or per "whatever-metric-we-use"). (Outcomes: Outcome: Effective gift... & Outcome: Pledge... )

Ideas/methods for targeting: platforms and audiences

Some audiences and approaches to targeting

• Public lists of political donations (e.g, archive.org)

  • ... donors to candidates sympathetic to a relevant cause area

• Internet activity ... those who watch/read/search for:

  • Videos relevant to a cause area

  • Reddit threads relevant to a cause area

  • Magazines/news sites relevant to a cause area

  • Search/visiting webpages about charity effectiveness/merit (e.g., Charity Navigator) 👍

• Education

  • Courses/degrees/majors relevant to a cause area

    • (e.g., development econ/studies, animal behavior, AI)

  • People at high-status institutions (future influencers/policymakers)

• Exploiting social network structure

  • Targeting "influencers" and "central" people (on the basis of "number of followers" / friends / etc.)

• Key search terms (google 'effective giving' etc)

• Podcast listeners (philanthropy, economics, development & global health ...)