# Trial reporting template

*For each proposed/ongoing/past trial, let's try to report the following minimal details, with links (proposed template)* If you don't have time and you have another clear presentation of most of this, please link or embed it.

## "Concise reporting template"

{% hint style="info" %}
Please keep your answers brief -- if you want to give more detail (which is not necessary) please link a later section or external page. \_
{% endhint %}

{% hint style="info" %}
Short version of this template [as Google Doc HERE](https://docs.google.com/document/d/1N1_An6mgUl5TpwHlYOe4S70qABfOVXhJCG4ceFmIJZ4/copy) (link copy-opens a new version for you to work in)
{% endhint %}

### **General idea, main 'hypothesis' (if there is one)**

*Firstly, what is this promotion trying to do (e.g., 'encourage signups for giving pledges')?*

*But more importantly, what are you trying to **learn** here... What might you have a better understanding of after the trial than you did before the trial?*

*E.g.,*

> *Specifically:*
>
> * *Does the opportunity we offer to sign up for an 'accountability partner' increase or decrease the rate at which people DO XXX activity?*
> * *Does it lead to greater overall XXXlinked donations per visitor over the next 1 year interval?*
>
> *Generally:*
>
> * *Does 'social accountability' help to encourage XXX activities and promises and the fulfillment of these? Does the 'fear of being held accountable' discourage people from making commitments?*

*(Optional: brief on background theory and previous evidence)*

### Point of contact (at organization running trial)

*You can enter more than 1 person here, including an external organizer (like JS Winchell), but ideally, also someone inside the organization.*

*Add 'academic/research lead' here if there is one*

### Timing of trial (when will it/did it start and end, if known)

### Digital location where project 'lives' (planning, material, data)

The present Gitbook/and a nested Github repo folder could be ideal. Please give a precise link so others could access it.

### Environment/context for trial

(Is it on a web page, a google advertisement, a physical mailing, etc)

### **Participant universe and sample size**

* Who will be targeted or who do you expect to be part of the trial?
* (Somewhat optional) How many people (or 'units') do you expect to be involved (median guess)?
* (Optional): How many do you expect will have a 'positive outcome' (e.g., a 'conversion')?

### Key treatment(s)

Description, link exact language/content if possible

### Treatment assignment procedure

* At what level is it varied? (individual visitors, postal codes, days of the week, etc)
* How are treatments assigned ('blocked randomization', 'adaptive/Thomson sampling', etc.)
  * *If you are using a 'set Google, Facebook etc algorithm', just input the settings you used here, and/or link the (Google, FB, etc) explanation*
* How many/what shares are assigned to each treatment?

### **Outcome data**

* What measures (outcomes, other features) will be collected?
* When and how
* Where will the data be stored, who will have access

### **Optional/suggested additions**

*Planned analysis methods, preregistration link, IRB link, connection to other projects and promotions*

## Ex-post: Reporting results (brief)

### Implementation and data collection

1. Did it go as planned? Any departures? (Timing, randomization, design changes, etc)
2. How much/what data was collected? How many observations?
3. Where is the data stored (also link/adjust the above), who has it, and under what conditions?

### Basic results/outcomes

1. "Partners and stakeholders opinions": were they happy with the trial? Did they seem to think it was a success?
2. Simplest statement (e.g., "3% donated in the treatment versus 2.2% in the control, with an average amount raised of $4.3 in the treatment and $3.1 in the control')
3. Preliminary interpretation, with statistical test if possible (e.g., 'google Optimize states an 80% chance that the treatment outperformed the control', a Fisher's exact test yields a p=0.06 that a positive donations was more likely in the treatment than the control)
4. "Full analysis"
   1. Who/what when will it be done?
   2. Link to 'where' it will be done (both the 'follow up the pre-analysis plan, and the full write-up, if applicable)
   3. Possibly: Briefly characterize the overall confusions/state of analysis here (state the date last updated)
5. Feeding synthesis and meta-analysis
   1. Which generalizable questions does this inform?
   2. Is data sharable? Key comparable outcomes?
   3. What other work/trials does this relate to?
   4. State of meta-analysis