What to do with the data after you collect it (and what you should put in a pre-analysis-plan).
Notes from slack:
I’m finding some issues like this in analyzing rare events … not quite that rare, but still a few per thousand or a few per hundred.
I’m taking 2 statistical approaches to the analysis (discussion, code, and data in links):
Randomization inference (simulation) … for a sort of equivalence testing here
I think either of these could be ‘flipped around’ to be used for power calculation or ‘the Bayesian equivalent of power calculation’
My colleague Jamie Elsey has some expertise with the latter; we’re putting together our discussion HERE, although it’s mainly frequentist and not Bayesian ATM.
There are reasons 'some pre-registration' or at least 'declaring your intentions in advance' is worth doing even if you aren't aiming at scientific publication
https://gitlab.com/dsbowen/conditional-inference/-/blob/master/examples/bayes_primer.ipynb