replicateEverything
  • Studies
  • Replicate
  • Contribute
  • Feedback
  • About
Explore different types of study

1. Choose a study
Local study
Tip: type or select local to use the study checked out in the app's current working directory (no DOI or registry lookup needed).

2. Tables & figures

  • Output
  • Code
  • Pipeline

Contribute a replication

We invite contributions of replications to replicateEverything . Once a study is checked and registered, users can inspect code and run replications live from this Shiny app and from the R package.
Hover underlined terms for copy-paste examples.
1. Prep your replication repository
1.1 Write a replication.yml
The key to a compatible registry entry is a replication.yml — a text file that maps the parts of a replication archive: paper metadata, maintainer, engines, and each step's code, data, and outputs. replicateEverything reads that yaml to Display, Code, and Run. View template example from rep-template .
Yaml elements that matter for registry compatibility:
  • Maintainer. Every study must declare maintainer: (name + email) — shown as [maintainer] on the Studies tab.
  • Collections. List one or more collections: tags (e.g. APSR , PED , World Bank , IPI ) so readers can filter the registry.
  • Engines / languages. Declare top-level languages: and per-step engine: ( r , stata , and/or python ). Paired ids (e.g. tab_1 and tab_1_stata ) let both engines appear in Shiny.
  • Steps. Each table, figure, or transform is one steps: entry linking code , data / inputs , and outputs . Prefer a type: format child for display formatting.
  • Analysis helpers. R steps define make_*() / format_*()
    Analysis helpers
    make_tab_1 <- function(data) {
  glm(onset ~ warl + gdpenl + lpopl, data = data,
      family = binomial())
}

format_tab_1 <- function(object) {
  modelsummary::modelsummary(
    object,
    output = "kableExtra",
    stars = TRUE
  )
}
         ; Stata/Python steps use the scripts named in yaml. Footers that call those helpers are optional.
1.2 Structure your repository folder
Choose one of two layouts for the study materials.
Folder-backed study repository. Dedicated Git repo with code/ , data/ , and outputs/ ; scripts on disk linked from yaml; R, Stata, and/or Python. Good default for delivered archives and multi-engine papers. Start from rep-template or create code/ / data/ / outputs/
Study repo layout
replication.yml
data/repdata.dta
code/tab_1.R
outputs/tab_1.html
outputs/manifest.json
tests/testthat/
tests/substantive/
 plus root replication.yml
Template study yaml
paper:
  study_handle: rep-template
  title: "Minimal folder-backed template study"
  year: "2026a"
  authors: ReplicateEverything Team
  source_repository: https://github.com/replicate-anything/rep-template
  abstract: >
    Unpublished template study for replicateEverything. Demonstrates a minimal
    folder-backed layout: CSV data, one estimatr table, HTML display output,
    and substantive tests. No journal article DOI.
  study_url: https://github.com/replicate-anything/rep-template
  dependencies:
    - estimatr
    - knitr

maintainer:
  name: Macartan Humphreys
  email: macartan.humphreys@wzb.eu

collections:
  - IPI

repo: replicate-anything/rep-template

languages:
  - r

steps:
  - id: tab_1
    type: table
    label: Simple table
    description: OLS of Y on X with robust SEs (estimatr::lm_robust)
    engine: r
    inputs:
      - data/data.csv
    code: code/tab_1.R
    format: format_tab_1
    outputs:
      - outputs/tab_1.html
    dependencies:
      - estimatr
      - knitr

  - id: tab_1_format
    type: format
    parent: tab_1
    code: code/tab_1.R
 .
Package-backed study. Materials live in an R package (LazyData / data/ , functions in R/ ); R only (Stata belongs in the folder-backed model). Convenient when many tables/figures share packaged datasets. Include replication.yml
Package registry stub
paper:
  doi: https://doi.org/10.1371/journal.pone.0278337
  title: Public support for global vaccine sharing...
  package: rep1371journalpone0278337
  package_repo: replicate-anything/rep-10.1371-journal.pone.0278337
  package_ref: main
repo: replicate-anything/rep-10.1371-journal.pone.0278337
maintainer:
  name: Jane Maintainer
  email: maintainer@example.org
collections:
  - IPI
languages:
  - r
 (and inst/replication.yml ) with paper.package , maintainer: , and collections: . Layout sketch: R/ / inst/ / data/
Package study layout
DESCRIPTION
R/make_tab_1.R          # export make_* / format_* only
inst/replication.yml
inst/replication_code/  # optional scripts
inst/report/artifacts/  # baked Display outputs
data/                   # packaged datasets
tests/testthat/
tests/substantive/
 . Do not define or ship run_replication() , list_replications() , load_artifact() , or get_code() in the study package — those verbs live only in replicateEverything . Export only the make_*() / format_*()
Study package helpers
# Study package: export analysis helpers named in yaml only
make_tab_1 <- function(data) { ... }
format_tab_1 <- function(object) { ... }

# Consumers call replicateEverything (these verbs are not in the study package):
library(replicateEverything)
check_replication(".")
run_replication(doi, "tab_1")
get_code(doi, "tab_1")
load_artifact(doi, "tab_1")
 named in yaml (plus any true study helpers).
Common guidance. Keep tests next to the study materials. Use tests/testthat/ for smoke tests. When published or known benchmarks exist, write substantive tests : add tests/substantive/<id>.R
Substantive benchmarks
# tests/substantive/tab_1.R  (published / known benchmarks)
substantive_check_tab_1 <- function(object, tolerance = 1e-5) {
  # compare coefficients / estimates on object from make_tab_1()
  ...
}

# tests/testthat/test-tab_1.R sources it after a live run:
fit <- replicateEverything::run_replication("local", "tab_1")
source("tests/substantive/tab_1.R", local = TRUE)
substantive_check_tab_1(fit)
 with a substantive_check_<id>() and call it from tests/testthat/ after a live run. check_replication() reports coverage of these checks. See vignettes folder-replication-checklist and package-replication-checklist for full checklists.
1.3 Bake outputs
Common to both layouts: run build_study_outputs()
Bake Display outputs
library(replicateEverything)

# Folder- or package-backed — writes Display outputs + manifest.json:
build_study_outputs(".", install_deps = TRUE)

# Folder studies → outputs/
# Package studies → inst/report/artifacts/ (or inst/report/outputs/)
 so Display can load precomputed HTML/figures quickly. Folder studies write under outputs/ ; package studies write under inst/report/artifacts/ (or inst/report/outputs/ ). One entrypoint covers both.
2. Check locally
2.1 Validate and run tests using testthat
Validate structure and run testthat :
  • check_replication()
    Validate a study
    library(replicateEverything)

# Structure, outputs, optional live runs:
check_replication(".", full_replication = FALSE)

# testthat smoke tests (and substantive checks when present):
testthat::test_dir("tests/testthat")
         — structure, outputs, substantive-check coverage, and optional live runs.
  • testthat::test_dir("tests/testthat") — smoke tests, plus tests/substantive/
    Substantive benchmarks
    # tests/substantive/tab_1.R  (published / known benchmarks)
substantive_check_tab_1 <- function(object, tolerance = 1e-5) {
  # compare coefficients / estimates on object from make_tab_1()
  ...
}

# tests/testthat/test-tab_1.R sources it after a live run:
fit <- replicateEverything::run_replication("local", "tab_1")
source("tests/substantive/tab_1.R", local = TRUE)
substantive_check_tab_1(fit)
         when benchmarks exist.
2.2 Check the replicateEverything API
You can now check that the replicateEverything functions play well with your repo — the same verbs Shiny uses — without claiming the study exports them:
  • check_replication()
    API play-well checks
    library(replicateEverything)

# Exercise the same APIs Shiny uses against your study:
list_replications("local")          # or the study DOI
run_replication("local", "tab_1", format = TRUE)
get_code("local", "tab_1")
load_artifact("local", "tab_1")
         already exercises yaml and materials; also call list_replications() , run_replication()
    Run one replication
    library(replicateEverything)

# Exercise the same APIs Shiny uses against your study:
list_replications("local")          # or the study DOI
run_replication("local", "tab_1", format = TRUE)
get_code("local", "tab_1")
load_artifact("local", "tab_1")
         , get_code() , and load_artifact() with doi = "local" when your working directory is inside the study (folder) or after the package is installed.
3. Connect with the registry
Choose one path:
  • Contact the registry maintainer. Send the address of the study repo (or package) once it passes local checks. The maintainer typically: re-validates with check_replication() , runs sync_study_to_registry()
    Maintainer sync
    # Registry maintainer — after the study repo is ready:
library(replicateEverything)
options(replicateEverything.registry_root = "../registry")

check_replication("../path/to/study")
sync_study_to_registry("../path/to/study")  # writes studies/<folder>.yml
refresh_registry("../registry", audit = TRUE)  # rebuilds index.csv + audit
# commit registry: studies/<folder>.yml, index.csv
         to write studies/<folder>.yml from your yaml, then refresh_registry()
    Rebuild index + audit
    # Registry maintainer — after the study repo is ready:
library(replicateEverything)
options(replicateEverything.registry_root = "../registry")

check_replication("../path/to/study")
sync_study_to_registry("../path/to/study")  # writes studies/<folder>.yml
refresh_registry("../registry", audit = TRUE)  # rebuilds index.csv + audit
# commit registry: studies/<folder>.yml, index.csv
         and deploys.
  • Open a pull request on the registry. First sync a local copy of the registry, then use: check_and_bake_study()
    Prepare then PR
    # Contributor — before opening a registry PR:
library(replicateEverything)
options(replicateEverything.registry_root = "../registry")

check_and_bake_study(".", build_artifacts = TRUE)
check_replication(".", full_replication = FALSE)

# Sync a stub into your local registry checkout, rebuild index, then PR:
sync_study_to_registry(".")
build_registry_index("../registry")
# open PR on replicate-anything/registry with studies/<folder>.yml + index.csv
         , sync_study_to_registry()
    Write stub
    # Contributor — before opening a registry PR:
library(replicateEverything)
options(replicateEverything.registry_root = "../registry")

check_and_bake_study(".", build_artifacts = TRUE)
check_replication(".", full_replication = FALSE)

# Sync a stub into your local registry checkout, rebuild index, then PR:
sync_study_to_registry(".")
build_registry_index("../registry")
# open PR on replicate-anything/registry with studies/<folder>.yml + index.csv
         , build_registry_index()
    Compile index
    library(replicateEverything)
options(replicateEverything.registry_root = "../registry")

build_registry_index("../registry")
# writes index.csv with collections, maintainer_*, languages from stubs
         . Then open a PR on replicate-anything/registry pushing the following new elements: an addition to studies/<folder>.yml and an updated index.csv .

Feedback

Help us improve this prototype. Report bugs, ideas, or general feedback on GitHub — we read every issue.

Report on GitHub

Open an issue with a category label:

  • Bug report
  • Feature request
  • Other / general

Package: replicateEverything issues · Registry / study contributions: registry issues

Send feedback

Plain text only (max 2,000 characters).

This is a project of IPI WZB led by Macartan Humphreys , Cord Masche, and Vernon Washington.

replicateEverything

This demo app is bundled with the replicateEverything R package. Browse studies, display precomputed artifacts, and run live replications.

  • Why replicateEverything?
  • Live demo (this app)
  • Package documentation
  • Replication registry
  • Study repositories

Run interactively with replicateEverything::run_shiny_app() or deploy with save_local_shiny() . See the Shiny demo app vignette for details.

replicateEverything 0.7.58 · pkg b275a6f · app b275a6f · lib /wzb/samba/user/ipi/R/x86_64-pc-linux-gnu-library/4.4/replicateEverything stamp version: 0.7.56 · installed: 0.7.58 [possibly stale]