LogoLogo
  • The Unjournal
  • An Introduction to The Unjournal
    • Content overview
    • How to get involved
      • Brief version of call
      • Impactful Research Prize (pilot)
      • Jobs and paid projects with The Unjournal
        • Advisory/team roles (research, management)
        • Administration, operations and management roles
        • Research & operations-linked roles & projects
        • Standalone project: Impactful Research Scoping (temp. pause)
      • Independent evaluations (trial)
        • Reviewers from previous journal submissions
    • Organizational roles and responsibilities
      • Unjournal Field Specialists: Incentives and norms (trial)
    • Our team
      • Reinstein's story in brief
    • Plan of action
    • Explanations & outreach
      • Press releases
      • Outreach texts
      • Related articles and work
    • Updates (earlier)
      • Impactful Research Prize Winners
      • Previous updates
  • Why Unjournal?
    • Reshaping academic evaluation: Beyond accept/reject
    • Promoting open and robust science
    • Global priorities: Theory of Change (Logic Model)
      • Balancing information accessibility and hazard concerns
    • Promoting 'Dynamic Documents' and 'Living Research Projects'
      • Benefits of Dynamic Documents
      • Benefits of Living Research Projects
    • The File Drawer Effect (Article)
    • Open, reliable, and useful evaluation
      • Multiple dimensions of feedback
  • Frequently Asked Questions (FAQ)
    • For research authors
    • Evaluation ('refereeing')
    • Suggesting and prioritizing research
  • Our policies: evaluation & workflow
    • Project submission, selection and prioritization
      • What research to target?
      • What specific areas do we cover?
      • Process: prioritizing research
        • Prioritization ratings: discussion
      • Suggesting research (forms, guidance)
      • "Direct evaluation" track
      • "Applied and Policy" Track
      • 'Conditional embargos' & exceptions
      • Formats, research stage, publication status
    • Evaluation
      • For prospective evaluators
      • Guidelines for evaluators
        • Why these guidelines/metrics?
        • Proposed curating robustness replication
        • Conventional guidelines for referee reports
      • Why pay evaluators (reviewers)?
      • Protecting anonymity
    • Mapping evaluation workflow
      • Evaluation workflow – Simplified
    • Communicating results
    • Recap: submissions
  • What is global-priorities-relevant research?
  • "Pivotal questions"
    • ‘Operationalizable’ questions
    • Why "operationalizable questions"?
  • Action and progress
    • Pilot steps
      • Pilot: Building a founding committee
      • Pilot: Identifying key research
      • Pilot: Setting up platforms
      • Setting up evaluation guidelines for pilot papers
      • 'Evaluators': Identifying and engaging
    • Plan of action (cross-link)
  • Grants and proposals
    • Survival and Flourishing Fund (successful)
    • ACX/LTFF grant proposal (as submitted, successful)
      • Notes: post-grant plan and revisions
      • (Linked proposals and comments - moved for now)
    • Unsuccessful applications
      • Clearer Thinking FTX regranting (unsuccessful)
      • FTX Future Fund (for further funding; unsuccessful)
      • Sloan
  • Parallel/partner initiatives and resources
    • eLife
    • Peer Communities In
    • Sciety
    • Asterisk
    • Related: EA/global priorities seminar series
    • EA and EA Forum initiatives
      • EA forum peer reviewing (related)
      • Links to EA Forum/"EA journal"
    • Other non-journal evaluation
    • Economics survey (Charness et al.)
  • Management details [mostly moved to Coda]
    • Governance of The Unjournal
    • Status, expenses, and payments
    • Evaluation manager process
      • Choosing evaluators (considerations)
        • Avoiding COI
        • Tips and text for contacting evaluators (private)
    • UJ Team: resources, onboarding
    • Policies/issues discussion
    • Research scoping discussion spaces
    • Communication and style
  • Tech, tools and resources
    • Tech scoping
    • Hosting & platforms
      • PubPub
      • Kotahi/Sciety (phased out)
        • Kotahi: submit/eval/mgmt (may be phasing out?)
        • Sciety (host & curate evals)
    • This GitBook; editing it, etc
    • Other tech and tools
      • Cryptpad (for evaluator or other anonymity)
      • hypothes.is for collab. annotation
Powered by GitBook
On this page
  • Living, "Kaizen", "permanent Beta" work
  • Advantages of 'permanent Beta' projects
  • Discussion: living projects vs the current "replication+extension approach"

Was this helpful?

Export as PDF
  1. Why Unjournal?
  2. Promoting 'Dynamic Documents' and 'Living Research Projects'

Benefits of Living Research Projects

Living, "Kaizen", "permanent Beta" work

Should research projects be improved and updated 'in the same place', rather than with 'extension papers'?

Advantages of 'permanent Beta' projects

  • Small changes and fixes: The current system makes it difficult to make minor updates – even obvious corrections – to published papers. This makes these papers less useful and less readable. If you find an error in your own published work, there is also little incentive to note it and ask for a correction, even if this were possible.

    • In contrast, a 'living project' could be corrected and updated in situ. If future and continued evaluations matter, they will have the incentive to do so.

  • Lack of incentives for updates and extensions: If academic researchers see major ways to improve and build on their past work, these can be hard to get published and get credit for. The academic system rewards novelty and innovation, and top journals are reluctant to publish 'the second paper' on a topic. As this would count as 'a second publication' (for tenure etc.), authors may be accused of double-dipping, and journals and editors may punish them for this.

  • Clutter and confusion in the literature: Because of the above, researchers often try to spin an improvement to a previous paper as very new and different. They do sometimes publish a range of papers getting at similar things and using similar methods, in different papers/journals. This makes it hard for other researchers and readers to understand which paper they should read.

    • In contrast, a 'living project' can keep these in one place. The author can lay out different chapters and sections in ways that make the full work most useful.

But we recognize there may also be downsides to _'_all extensions and updates in a single place'...

Discussion: living projects vs the current "replication+extension approach"

SK: why [do] you think it's significantly better than the current replication+extension approach?

PS (?): Are these things like 'living' google docs that keep getting updated? If so I'd consider using workarounds to replicate their benefits on the forum for a test run (e.g., people add a version to paper title or content or post a new version for each major revision). More generally, I'd prefer the next publication norm for papers to be about making new 'versions' of prior publications (e.g, a 'living review' paper on x is published and reviewed each year) than creating live documents (e.g., a dynamic review on x is published on a website and repeatedly reviewed at frequent and uncertain intervals when the authors add to it). I see huge value in living documents. However, I feel that they wouldn't be as efficient/easy to supervise/review as 'paper versions'.

@GavinTaylor: I don’t think living documents need to pose a problem as long as they are discretely versioned and each version is accessible. Some academic fields are/were focused on books more than papers, and these were versioned by edition. Preprinting is also a form of versioning and combining the citations between the published paper and its preprint/s seems to be gaining acceptance (well, google scholar may force this by letting you combine them) - I don’t recall ever seeing a preprint citation indication a specific version (on preprint servers that support this) but its seems possible.

DR: I mainly agree with @gavintaylor, but I appreciate that 'changing everything at the same time' is not always the best strategy.

The main idea is that each version is given a specific time stamp, and that is the object that is reviewed and cited. This is more or less already the case when we cite working papers/drafts/mimeos/preprints.

Gavin, on the latter 'past version accesibility' issue, This could/should be a part of what we ensure with specific rules and tech support, perhaps.

I think the issue with the current citing practice for live documents like webpages is that even if a ‘version’ is indicated (e.g. access date) past versions aren’t often very accessible.

They might also not be ideal for citing as they would be an ever-changing resource. I can imagine the whole academic system struggling to understand and adapt to such a radical innovation given how focused it is on static documents. With all of this considered, I'd like 'dynamic/living work' to be incentivised with funding and managed with informal feedback and comments rather than being formally reviewed (at least for now). I'd see living review work as sitting alongside and informing 'reviewed' papers rather than supplanting them. As an example, you might have a website that provides a 'living document' for lay people about how to promote charity effectively and then publish annual papers to summarise the state of the art for an academic/sophisticated audience.

DR: I can see the arguments on both sides here. I definitely support replications and sometimes it may make sense for the author to “start a new paper” rather than make this an improvement if the old one. I also think that the project should be time stamped, evaluated and archived at particular stages of its development.

But I lean to thinking that in many to most cases a single project with multiple updates will make the literature clearer and easier to navigate than the current proliferation of “multiple very similar papers by the same author in different journals”. It also seems a better use of researcher time, rather than having to constantly restate and repackage the same things

  1. Some discussion follows. Note that the Unjournal enables this but does not require it.

PreviousBenefits of Dynamic DocumentsNextOpen, reliable, and useful evaluation

Last updated 1 year ago

Was this helpful?