Trial reporting template
For each proposed/ongoing/past trial, we should report the following minimal details, with links (proposed template)
Last updated
Was this helpful?
For each proposed/ongoing/past trial, we should report the following minimal details, with links (proposed template)
Last updated
Was this helpful?
For each proposed/ongoing/past trial, let's try to report the following minimal details, with links (proposed template) If you don't have time and you have another clear presentation of most of this, please link or embed it.
Firstly, what is this promotion trying to do (e.g., 'encourage signups for giving pledges')?
But more importantly, what are you trying to learn here... What might you have a better understanding of after the trial than you did before the trial?
E.g.,
Specifically:
Does the opportunity we offer to sign up for an 'accountability partner' increase or decrease the rate at which people DO XXX activity?
Does it lead to greater overall XXXlinked donations per visitor over the next 1 year interval?
Generally:
Does 'social accountability' help to encourage XXX activities and promises and the fulfillment of these? Does the 'fear of being held accountable' discourage people from making commitments?
(Optional: brief on background theory and previous evidence)
You can enter more than 1 person here, including an external organizer (like JS Winchell), but ideally, also someone inside the organization.
Add 'academic/research lead' here if there is one
The present Gitbook/and a nested Github repo folder could be ideal. Please give a precise link so others could access it.
(Is it on a web page, a google advertisement, a physical mailing, etc)
Who will be targeted or who do you expect to be part of the trial?
(Somewhat optional) How many people (or 'units') do you expect to be involved (median guess)?
(Optional): How many do you expect will have a 'positive outcome' (e.g., a 'conversion')?
Description, link exact language/content if possible
At what level is it varied? (individual visitors, postal codes, days of the week, etc)
How are treatments assigned ('blocked randomization', 'adaptive/Thomson sampling', etc.)
If you are using a 'set Google, Facebook etc algorithm', just input the settings you used here, and/or link the (Google, FB, etc) explanation
How many/what shares are assigned to each treatment?
What measures (outcomes, other features) will be collected?
When and how
Where will the data be stored, who will have access
Planned analysis methods, preregistration link, IRB link, connection to other projects and promotions
Did it go as planned? Any departures? (Timing, randomization, design changes, etc)
How much/what data was collected? How many observations?
Where is the data stored (also link/adjust the above), who has it, and under what conditions?
"Partners and stakeholders opinions": were they happy with the trial? Did they seem to think it was a success?
Simplest statement (e.g., "3% donated in the treatment versus 2.2% in the control, with an average amount raised of $4.3 in the treatment and $3.1 in the control')
Preliminary interpretation, with statistical test if possible (e.g., 'google Optimize states an 80% chance that the treatment outperformed the control', a Fisher's exact test yields a p=0.06 that a positive donations was more likely in the treatment than the control)
"Full analysis"
Who/what when will it be done?
Link to 'where' it will be done (both the 'follow up the pre-analysis plan, and the full write-up, if applicable)
Possibly: Briefly characterize the overall confusions/state of analysis here (state the date last updated)
Feeding synthesis and meta-analysis
Which generalizable questions does this inform?
Is data sharable? Key comparable outcomes?
What other work/trials does this relate to?
State of meta-analysis