Should research projects be improved and updated 'in the same place', rather than with 'extension papers'?
Small changes and fixes: The current system makes it difficult to make minor updates – even obvious corrections – to published papers. This makes these papers less useful and less readable. If you find an error in your own published work, there is also little incentive to note it and ask for a correction, even if this were possible.
In contrast, a 'living project' could be corrected and updated in situ. If future and continued evaluations matter, they will have the incentive to do so.
Lack of incentives for updates and extensions: If academic researchers see major ways to improve and build on their past work, these can be hard to get published and get credit for. The academic system rewards novelty and innovation, and top journals are reluctant to publish 'the second paper' on a topic. As this would count as 'a second publication' (for tenure etc.), authors may be accused of double-dipping, and journals and editors may punish them for this.
Clutter and confusion in the literature: Because of the above, researchers often try to spin an improvement to a previous paper as very new and different. They do sometimes publish a range of papers getting at similar things and using similar methods, in different papers/journals. This makes it hard for other researchers and readers to understand which paper they should read.
In contrast, a 'living project' can keep these in one place. The author can lay out different chapters and sections in ways that make the full work most useful.
But we recognize there may also be downsides to _'_all extensions and updates in a single place'...
Some discussion follows. Note that the Unjournal enables this but does not require it.
By “Dynamic Documents” I mean papers/projects built with Quarto, R-markdown, or JuPyTer notebooks (the most prominent tools) that do and report the data analysis (as well as math/simulations) in the same space that the results and discussion are presented (with ‘code blocks’ hidden).
I consider some of the benefits of this format, particularly for EA-aligned organizations like Open Philanthropy: Benefits of Dynamic Documents
“Continually update a project” rather than start a “new extension paper” when you see what you could have done better.
The main idea is that each version is given a specific time stamp, and that is the object that is reviewed and cited. This is more or less already the case when we cite working papers/drafts/mimeos/preprints.
See #living-kaizend-research-projects, further discussing the potential benefits.
'Dynamic Documents' are projects or papers that are developed using prominent tools such as R-markdown or JuPyTer notebooks (the two most prominent tools).
The salient features and benefits of this approach include:
Integrated data analysis and reporting means the data analysis (as well as math/simulations) is done and reported in the same space that the results and discussion are presented. This is made possible through the concealment of 'code blocks'.
Transparent reporting means you can track exactly what is being reported and how it was constructed:
Making the process a lot less error-prone
Helping readers understand it better (see 'explorable explanations')
Helping replicators and future researchers build on it
Other advantages of these formats (over PDFs for example) include:
Convenient ‘folding blocks’
Margin comments
and links
Integrating interactive tools
Some quick examples from my own work in progress (but other people have done it much better)
