Skip to content

R_VERSION_PLAN.md

Latest commit

 

History

History
334 lines (259 loc) · 15 KB

R_VERSION_PLAN.md

File metadata and controls

334 lines (259 loc) · 15 KB
Raw

Building an R Version

A practical plan for porting this starter kit to R. Written so a student or instructor can hand the plan to Claude Code, OpenAI Codex, GitHub Copilot, or Perplexity Computer and end up with a working R dashboard at parity with the Python one.

Not a tutorial. This is a design and decision document. The Python codebase remains the reference implementation; this doc maps it to the R equivalents and flags where the languages diverge.

Why an R Version?

R has two genuine advantages for an analytics dashboard like this:

  1. Statistical and time-series modeling is more idiomatic in R. Phase 4 (on-time performance, headway distributions) maps cleanly onto forecast, tsibble, fable, and base lm/glm.
  2. Many MS Analytics curricula are R-first. Students who already know dplyr, ggplot2, and Shiny should be able to extend their existing skill set rather than learn a parallel Python stack.

R is not a better choice for the API client, the capture loop, or the deployment story — those are roughly a wash. Pick R when the analytics matter more than the plumbing.

Stack Recommendation — Two Paths

Path A: Shiny + golem (recommended for full interactivity)

A faithful port of the four-tab dashboard. Same UX, R-native code.

Layer Package Why
API client httr2 Modern successor to httr; pipeable, retry-aware
Async / caching memoise + cachem Mirror of Streamlit's @st.cache_data
HTTP fixtures httptest2 For testing without hitting live API
Data wrangling dplyr + tidyr Tidyverse standard
Storage duckdb + DBI + dbplyr Same DuckDB file as Python; dbplyr lets you write dplyr that compiles to SQL
Plotting plotly (R port) or ggplot2 + plotly::ggplotly() Same charts as Python
Maps leaflet For optional bus-stop pin layer (Phase 5)
UI framework bs4Dash or stock shiny bs4Dash gives Bootstrap 4 cards/tabs out of the box
App framework golem Production-quality Shiny structure
Reproducibility renv Pinned package versions, equivalent to uv.lock
Testing testthat (3rd ed.) + shinytest2 Unit + end-to-end UI tests
Linting lintr + styler Style consistency

Path B: Quarto Dashboards (simpler, less interactive)

Quarto Dashboards are newer (released 2024). Single-file authoring with R, Python, or Julia. Less interactive than Shiny but a fraction of the code.

Use Quarto if:

  • You want a static or near-static dashboard (refresh on page load, not reactive)
  • You're comfortable accepting fewer interactive widgets
  • You like authoring in .qmd files mixing prose, code, and output
  • You want a single deploy artifact (HTML, optionally with shinyrt for limited interactivity)

Use Shiny if:

  • You want the existing dashboard's reactivity (typeahead station search, refresh button, auto-refresh)
  • You're building toward Phase 4/5 where complex interactivity helps
  • You want classmates to interact with live data, not a snapshot

This plan focuses on Path A. Path B is a 1–2 day rewrite once Path A exists and someone wants the simpler version.

Project Layout (golem convention)

wmata-dashboard-r/
├── DESCRIPTION                # package metadata; equivalent to pyproject.toml
├── NAMESPACE                  # auto-generated by roxygen2 — do not edit
├── README.md
├── .Renviron.example          # template; copy to .Renviron, add WMATA_API_KEY
├── .gitignore                 # excludes .Renviron, renv/library, *.duckdb
├── renv.lock                  # locked package versions (like uv.lock)
│
├── R/                         # all R source — golem looks here automatically
│   ├── api_client.R           # ←  wmata/client.py
│   ├── storage.R              # ←  wmata/storage.py
│   ├── capture.R              # ←  wmata/capture.py
│   ├── helpers_ui.R           # line_badge(), incident_card_html(), etc.
│   │
│   ├── mod_rail.R             # ←  app/dashboard.py — Rail tab as a Shiny module
│   ├── mod_bus.R              # ←  Bus tab
│   ├── mod_system.R           # ←  System tab
│   ├── mod_history.R          # ←  History tab
│   │
│   ├── app_ui.R               # top-level UI: sidebar + tabs
│   ├── app_server.R           # top-level server
│   └── run_app.R              # exported entry point: wmataDashboard::run_app()
│
├── inst/
│   ├── app/www/
│   │   └── style.css          # ←  the inline <style> from app/dashboard.py
│   └── extdata/
│       └── demo_data.rds      # demo-mode fallback data
│
├── exec/
│   └── capture.R              # CLI capture script — Rscript exec/capture.R --once
│
├── tests/
│   ├── testthat/
│   │   ├── test-api-client.R
│   │   ├── test-storage.R
│   │   └── test-capture.R
│   └── testthat.R
│
├── docs/                      # symlink or copy from the Python project
└── data/                      # gitignored
    ├── cache/
    └── wmata.duckdb

Component Mapping (Python → R)

Python file R file Notes
wmata/client.py R/api_client.R Replace requests with httr2. Functions stay the same names.
wmata/storage.py R/storage.R Use duckdb::dbConnect() + DBI::dbExecute(). The DuckDB file is binary-compatible — same data/wmata.duckdb works from both languages.
wmata/capture.py R/capture.R Sys.sleep() for the loop; withCallingHandlers() for per-iteration error handling.
scripts/capture.py exec/capture.R Use optparse package for CLI args.
app/dashboard.py (sidebar + header + KPIs) R/app_ui.R + R/app_server.R Top-level shell.
Each Streamlit with tab_X: block R/mod_X.R One Shiny module per tab. Modules give clean namespacing and reusability.
Inline <style> block inst/app/www/style.css Loaded via tags$link(rel="stylesheet", href="style.css").
.streamlit/config.toml inst/app/www/style.css (theme parts) + bs4Dash::dashboardPage(skin = ...) Shiny doesn't have a single config file for theme; CSS is the truth.
tests/run_tests.py tests/testthat/*.R testthat 3rd edition with parallel runner.
pyproject.toml DESCRIPTION + renv.lock Package metadata + locked deps.

Sketch — R/api_client.R

#' WMATA API client.
#'
#' Thin wrapper around the WMATA REST API. All functions take api_key as the
#' first argument; no global state.
#'
#' @import httr2
#' @export
get_predictions <- function(api_key, station_code) {
  req <- request("https://api.wmata.com/StationPrediction.svc/json/GetPrediction") |>
    req_url_path_append(station_code) |>
    req_headers(api_key = api_key) |>
    req_timeout(10) |>
    req_retry(max_tries = 2)

  resp <- req_perform(req)
  body <- resp_body_json(resp)

  trains <- body$Trains %||% list()
  if (length(trains) == 0) {
    return(tibble::tibble(Line = character(), DestinationName = character(), Min = character()))
  }
  dplyr::bind_rows(lapply(trains, as.data.frame))
}

The Min field stays a character column — same data quirk as Python.

Sketch — R/mod_rail.R (Rail tab as Shiny module)

#' Rail tab module.
#' @export
mod_rail_ui <- function(id) {
  ns <- NS(id)
  tagList(
    div(class = "section-header", textOutput(ns("station_header"), inline = TRUE)),
    uiOutput(ns("predictions_table")),
    plotly::plotlyOutput(ns("arrival_chart"), height = "260px")
  )
}

#' @export
mod_rail_server <- function(id, api_key, station_data, selected_station) {
  moduleServer(id, function(input, output, session) {

    predictions <- reactive({
      req(selected_station(), api_key)
      code <- station_data()[[selected_station()]]$code
      get_predictions(api_key, code)
    }) |>
      bindCache(selected_station(), x = ., n = 30)  # 30 sec cache

    output$station_header <- renderText({
      paste0("Next Trains — ", selected_station())
    })

    output$predictions_table <- renderUI({
      df <- predictions()
      if (nrow(df) == 0) return(div(class = "empty-state", "No live predictions."))
      pred_table_html(df)
    })

    output$arrival_chart <- plotly::renderPlotly({
      build_arrival_chart(predictions())
    })
  })
}

The reactive cache (bindCache) is the moral equivalent of Streamlit's @st.cache_data(ttl=30).

Reactive vs Top-to-Bottom — the One Mental Model Shift

Streamlit Shiny
Execution model Whole script re-runs on every interaction Reactive graph; only affected nodes re-evaluate
State across runs st.session_state dict Reactive values + module namespacing
Caching @st.cache_data(ttl=N) bindCache(), memoise::memoise(), cachem
Empty state guard if data: ... req(data)
Loading st.spinner() shinybusy::add_busy_spinner()

Once you internalize "reactives are recipes the framework re-cooks when ingredients change," Shiny becomes pleasant. Until then it feels backwards.

What R Does Better Here

  • Phase 4 analytics. forecast, fable, tsibble for headway and on-time time-series. dbplyr to push aggregations into DuckDB without writing SQL strings.
  • Statistical visualization. ggplot2 is genuinely better than matplotlib/plotly for grouped bar charts, faceted small multiples, and publication-grade figures. Use plotly::ggplotly() to keep interactivity.
  • Pipe-friendly API client. httr2 reads more naturally than requests for chained transforms.

What Python Does Better Here

  • Streamlit's "every event re-runs the script" model. Easier to teach beginners than Shiny reactives.
  • More deployment options on the free tier. Streamlit Community Cloud, Hugging Face Spaces, Render all have Streamlit-native flows; R deploys are doable but fewer turnkey choices.
  • Slightly more polished DuckDB story. Python duckdb package supports more DataFrame interop than R's at present (but both are fine for this project).
  • AI-tool fluency. Today's agentic coding tools are slightly stronger on Python out of the box. R works fine but expect to brief the agent more carefully.

Testing Strategy

# tests/testthat/test-api-client.R
test_that("get_predictions returns expected shape", {
  skip_on_cran()
  skip_if_offline()
  skip_if(Sys.getenv("WMATA_API_KEY") == "", "no API key")

  trains <- get_predictions(Sys.getenv("WMATA_API_KEY"), "A01")

  expect_s3_class(trains, "data.frame")
  expect_true(all(c("Line", "DestinationName", "Min") %in% names(trains)))
  expect_type(trains$Min, "character")  # Min is a string, not int
})

UI tests with shinytest2:

test_that("Rail tab renders predictions for Metro Center", {
  skip_if(Sys.getenv("WMATA_API_KEY") == "", "no API key")
  app <- shinytest2::AppDriver$new(run_app(), name = "rail-metro-center")
  app$set_inputs(`sidebar-station` = "Metro Center")
  app$wait_for_idle()
  expect_match(app$get_text("[data-testid='station-header']"), "Metro Center")
})

Aim for a coverage parallel to the Python suite: shape checks at L1, end-to-end at L3, an L2 integration test for capture.

Deployment Options for R

Option Best for Notes
Local Personal use Rscript -e 'wmataDashboard::run_app()'
shinyapps.io Free hosted Shiny Posit's free tier — 5 apps, 25 active hours/month
Posit Connect Cloud Newer free tier GitHub-integrated, simpler than Connect (the on-prem product)
Hugging Face Spaces Free + ML adjacency Supports Shiny via Docker SDK
Docker + Fly.io / Render Full control Build with rocker/shiny base image
Quarto Pub If you went with Path B Static deploy of the rendered dashboard

The local-first principle from docs/DEPLOYMENT.md applies here too: most students should start by running it on their laptop and only deploy once they need to share.

Working With Agentic AI Tools

A good prompt for porting one component:

Port wmata/storage.py from this Python codebase to R/storage.R. Use the duckdb, DBI, and dplyr packages. Match the function signatures (connect, init_schema, insert_predictions, get_capture_stats, get_arrival_history, get_recent_runs). Preserve the schema exactly — the same DuckDB file should be read/writable from either language. Add a tests/testthat/test-storage.R that mirrors H1–H3 from the Python tests/run_tests.py.

Tips:

  • Hand the agent the Python source file first, then this plan, then the target R file path
  • Ask for one component at a time — don't try to port the whole project in a single prompt
  • Run testthat::test_file() after each component before moving on
  • For Shiny modules, port the Python tab, then verify in the running app, then write shinytest2 tests
  • Keep the same DuckDB file so you can flip between Python and R freely while developing

Estimated Effort

For a student with intermediate R skills working with an AI assistant:

Component Hours
DESCRIPTION + renv + scaffolding 1
R/api_client.R + tests 2
R/storage.R + tests 2
R/capture.R + exec/capture.R 2
R/mod_rail.R (first module — most learning) 3
R/mod_bus.R, R/mod_system.R, R/mod_history.R 4
Top-level UI/server, sidebar, KPI row 2
inst/app/www/style.css (port the design tokens) 1
shinytest2 end-to-end tests 2
First deployment to shinyapps.io 1
Total ~20 hours

That's roughly two long weekends or a sprint week. The first three components establish the patterns; the rest go faster.

Open Questions for the Porter

  • Which UI framework? bs4Dash gives the most modern look out of the box but adds a layer; pure shiny keeps it simple. The Python version's heavy CSS customization argues for going more bare-bones in R and rebuilding from CSS.
  • Demo mode handling. The Python version embeds DEMO_* constants in dashboard.py. In R, save them as inst/extdata/demo_data.rds and load on startup if WMATA_API_KEY is empty.
  • Hot reload. Shiny's reload story isn't as smooth as Streamlit's. Consider golem::run_dev() which sets up a watch loop, or just live with runApp() and Ctrl-C / re-run.
  • Time zones in DuckDB. Both R and Python store captured_at as TIMESTAMP. Confirm timezone handling is consistent — Sys.time() vs datetime.now(timezone.utc) may differ; standardize on UTC at write time.

What You'll Get

A parallel R implementation that:

  • Runs the same dashboard with the same data, written in idiomatic R
  • Reads the same DuckDB file the Python capture writes (so they can run side-by-side)
  • Demonstrates Shiny modules, httr2, bs4Dash, golem, renv, testthat 3, and shinytest2 — a curriculum-grade modern R stack
  • Provides a side-by-side comparison for instructors teaching both languages