Comment on page
Trial reporting template
For each proposed/ongoing/past trial, we should report the following minimal details, with links (proposed template)
For each proposed/ongoing/past trial, let's try to report the following minimal details, with links (proposed template) If you don't have time and you have another clear presentation of most of this, please link or embed it.
Please keep your answers brief -- if you want to give more detail (which is not necessary) please link a later section or external page. _
Short version of this template as Google Doc HERE (link copy-opens a new version for you to work in)
Firstly, what is this promotion trying to do (e.g., 'encourage signups for giving pledges')?
But more importantly, what are you trying to learn here... What might you have a better understanding of after the trial than you did before the trial?
E.g.,
Specifically:
(Optional: brief on background theory and previous evidence)
You can enter more than 1 person here, including an external organizer (like JS Winchell), but ideally, also someone inside the organization.
Add 'academic/research lead' here if there is one
The present Gitbook/and a nested Github repo folder could be ideal. Please give a precise link so others could access it.
(Is it on a web page, a google advertisement, a physical mailing, etc)
- Who will be targeted or who do you expect to be part of the trial?
- (Somewhat optional) How many people (or 'units') do you expect to be involved (median guess)?
- (Optional): How many do you expect will have a 'positive outcome' (e.g., a 'conversion')?
Description, link exact language/content if possible
- At what level is it varied? (individual visitors, postal codes, days of the week, etc)
- How are treatments assigned ('blocked randomization', 'adaptive/Thomson sampling', etc.)
- If you are using a 'set Google, Facebook etc algorithm', just input the settings you used here, and/or link the (Google, FB, etc) explanation
- How many/what shares are assigned to each treatment?
- What measures (outcomes, other features) will be collected?
- When and how
- Where will the data be stored, who will have access
Planned analysis methods, preregistration link, IRB link, connection to other projects and promotions
- 1.Did it go as planned? Any departures? (Timing, randomization, design changes, etc)
- 2.How much/what data was collected? How many observations?
- 3.Where is the data stored (also link/adjust the above), who has it, and under what conditions?
- 1."Partners and stakeholders opinions": were they happy with the trial? Did they seem to think it was a success?
- 2.Simplest statement (e.g., "3% donated in the treatment versus 2.2% in the control, with an average amount raised of $4.3 in the treatment and $3.1 in the control')
- 3.Preliminary interpretation, with statistical test if possible (e.g., 'google Optimize states an 80% chance that the treatment outperformed the control', a Fisher's exact test yields a p=0.06 that a positive donations was more likely in the treatment than the control)
- 4."Full analysis"
- 1.Who/what when will it be done?
- 2.Link to 'where' it will be done (both the 'follow up the pre-analysis plan, and the full write-up, if applicable)
- 3.Possibly: Briefly characterize the overall confusions/state of analysis here (state the date last updated)
- 5.Feeding synthesis and meta-analysis
- 1.Which generalizable questions does this inform?
- 2.Is data sharable? Key comparable outcomes?
- 3.What other work/trials does this relate to?
- 4.State of meta-analysis