Adaptive design/sampling, reinforcement learning
Last updated
Last updated
Dillon writes: I've run some very promising MTurk pilots using my adaptive experimentation software. Compared to traditional random assignment, it increases statistical power, identifies higher-value treatments, and results in more precise estimates of the effectiveness of top-performing treatments. From simulations, I estimate that the gains from adaptive experimentation are approximately equivalent to increasing your sample size by 2x-8x (depending on the distribution of effect sizes).
This would allow us to run studies like Eric Schwitzgebel + Fiery Cushman's study on philosophical arguments to increase charitable giving much more effectively
Dillon Bowen: End of 3rd year of decision processes in Wharton PHd.
Here is a stats package for estimating effect sizes in multi-armed experiments. https://dsbowen.gitlab.io/conditional-inference/
I just made a getting started video: Welcome to Hemlock - YouTube
...running experiments with many arms and winnowing out the 'best ones' to learn the most/best.
See: adaptive design, adaptive sampling, dynamic design, reinforcement learning, exploration sampling, Thompson's sampling, Bayesian adaptive inference, multifactor experiment
Discrete vs continuous: switches vs. knobs
In our cases of the ‘options are discrete’, many knobs to turn, although some are discrete. There is a different version of this for discrete vs continuous
If we can order the different treatments (arms/knobs) as 'dimensions' we can infer more... Can do better thinking of them as a ‘multifactor experiment’ rather than 2 unrelated … several separate dimensions
"Model running in the background" trying to figure out ‘things about the effectiveness of the interventions you might use’
“Ex-post regret versus cumulative regret” … latter suggests Thompson sampling (Does Thompson's sampling take into account the length of the future period?)
Ex-post … Use machine learning to consider which characteristics matter and how much they matter … although he doesn’t know of papers that have looked at this, but assumes there are adaptive designs that incorporate this.
Statistical inference can be challenging with adaptive designs, but this is a ripe area of research
Dillon: has a paper on traditional statistical inference after an adaptive design.
Goals 'what kinds of inference':
The arm you using relative to (? the average arm?)
Which factors matter/joint distribution ….. Bayesian models