Skip to content

README.md

Latest commit

 

History

History
316 lines (244 loc) · 13.6 KB

README.md

File metadata and controls

316 lines (244 loc) · 13.6 KB
Raw

irw: A Python Package for the Item Response Warehouse

This repository hosts the Python package irw, which provides programmatic access to the Item Response Warehouse (IRW), an open repository of harmonized item response data hosted on Redivis.

Project map: ARCHITECTURE.md in ben-domingue/irw — which repo owns what, where the data lives, and which document is authoritative when two disagree.

Installation

Recommended: Use a virtual environment (prevents conflicts with other packages):

# Create and activate a virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install the package
python -m pip install --upgrade pip
python -m pip install irw

To upgrade an existing install:

python -m pip install --upgrade irw

Development install

To run against unreleased code on main:

python -m pip install "git+https://github.com/itemresponsewarehouse/Python-pkg.git"

Note that this install does not upgrade cleanly: pip resolves the version from the clone, sees it already installed and skips, even under --upgrade. Use it when you want a specific commit, not as a way to stay current.

Requirements

  • Python 3.9 or higher
  • pip

If you encounter any installation issues, please open an issue.

IMPORTANT: Redivis Authentication

The IRW tables are hosted on Redivis, a data management platform. To access these datasets, you'll need to:

  1. Have a Redivis account (create one at https://redivis.com/?createAccount if you don't have one).

  2. Authenticate using the Redivis Python Client:

    1. When you first use a function in irw that connects to Redivis (e.g. list_tables()), a browser window will open, prompting you to sign in to your Redivis account.
    2. After signing in, click Allow to grant access for the Redivis Python Client.
    3. Once authentication is successful, close the browser window. You will see the message "Authentication was successful" in console.

Note: You only need to authenticate once per session. For detailed instructions, refer to the Redivis Python Client documentation.

Usage Examples

See the examples/ directory:

  • example.py - Complete workflow example
  • available_methods.md - Reference guide for all available methods

Example workflow:

import irw

# Get database information
irw.info()

# View available tables
tables = irw.list_tables()
tables_with_metadata = irw.list_tables(include_metadata=True)

# Get table info
irw.info("agn_kay_2025")  # Table metadata

# Fetch a table
df = irw.fetch("agn_kay_2025")
# Convert to response matrix
resp_matrix = irw.long2resp(df)

# Explore available filters
filters = irw.get_filters()  # Returns list of filter names
irw.describe_filter('construct_type')  # Get values for a specific filter

# Filter and fetch tables
filtered = irw.filter(n_responses=[1000, None], construct_type="Affective/mental health")
dfs = irw.fetch(filtered)

# Get BibTeX citation
irw.save_bibtex("agn_kay_2025")  # Returns BibTeX entry
# Download table
irw.download("agn_kay_2025", path="data.csv")

# Browse collections: labelled groupings of tables
irw.collections()                     # all collections, with coverage and table counts
tabs = irw.collection("depression")   # the table names in one collection
irw.collection_members(tables="frac20")  # which collections is this table in?

# Simulate IRW-shaped data (no network, no Redivis quota)
sim = irw.simdata(n_id=500, n_item=20, model="2PL", seed=1)   # id / item / resp
pairs = irw.simdata_comp(n_agent=100, n_pairs=10000, nu=0.1)  # agent_a / agent_b / winner

# Check your own table against the IRW format before depositing it
report = irw.validate("my_table.csv")  # needs `pip install irw-validate`
report.ok                              # False if anything blocks

# Which version of IRW is this? (cite this number)
irw.version()                         # newest IRW version and its dataset pins
irw.version("2026-08-01")             # what was live on that date

Caching and releases

Table listings and metadata are cached in memory so repeated calls do not re-query Redivis. The cache is keyed on each dataset's current version tag, so a release invalidates it: a process that has been running since before a release does not go on serving what the release withdrew. That matters most for item text, where withdrawals are how IRW stops distributing instrument wording it may not distribute.

Version tags are re-checked at most once every 300 seconds per dataset. Set IRW_VERSION_TTL_SECONDS to change that window -- 0 re-checks on every lookup, at the cost of one small metadata request each time. To drop everything cached in the current process:

from irw.utils.redivis.cache import metadata_cache
metadata_cache.clear()

Nothing is cached on disk, so a new process starts cold.

Validating your own data

irw.validate() checks a table against the IRW format standard — the rules in datastandard.md — and reports what would block a deposit. It uploads nothing and contacts nothing.

import irw, irw_validate

report = irw.validate("my_table.csv")     # or a DataFrame
report.ok                                  # False if anything blocks
print(irw_validate.format_report(report))

The checks live in a separate package, irw-validate, which is not installed with this one:

pip install irw-validate

It is kept separate on purpose. This package is a read client and most of its users never deposit anything, so the checker is fetched only by those who need it — and it needs pandas and nothing else: no Redivis account, no credentials, no network. irw.validate() raises an ImportError naming that command if it is missing.

Severity depends on the profile: profile="upload" (the default) is the deposit gate, and profile="core" is the five-check subset that IRW's standalone R script implements, for agreement between the two.

MCP server

IRW can run as a local, read-only Model Context Protocol server for an MCP-capable research assistant (issue ben-domingue/irw#1713). Eight tools:

Tool What it does Costs Redivis quota?
search_tables free-text search plus irw.filter()'s own filters, passed straight through; a summary card per hit, each saying whether it is tagged no
describe_filter what one filter means and which values it takes no
describe_table statistics, tags, bibliography for one table no
get_processing_notes the header of the script that built the table, from the IRW GitHub repository: whether id links across waves, what a cov_* means, what was excluded no (no login either)
fetch_table a bounded page of rows, bounded on the wire a page
get_itemtext a bounded page of item text with a rights object: response-data licence, the instrument-rights rule, and the table's public notes small
list_collections the labelled collections no
get_citation BibTeX for the original data producers no

search_tables does not define its own filter vocabulary. It takes a filters object, checks the names against irw.get_filters() and hands it to irw.filter(), so the server cannot drift from the package and cannot offer a smaller filter set than it has. describe_filter exposes irw.describe_filter() so an assistant can look up a tag vocabulary rather than guess at a spelling. One deliberate difference from irw.filter(): no default density filter is applied, because its [0.5, 1] default silently removes sparse tables from a search nobody asked to be about density.

Every response carries an observed irw_version and irw_released_at when the public manifest is available, plus retrieved_at. This does not pin reads from independently versioned Redivis tables: data_pinned is explicitly false. Processing-script links do use an immutable Git commit. The server's own version is the irw package version, not the data.

The MCP server requires Python 3.10 or newer because the current MCP SDK does. It does not make OpenAI calls and does not require an OpenAI key; the host application is responsible for the model. Redivis authentication is still handled by the irw package.

python3 -m venv .venv
.venv/bin/python -m pip install ".[mcp]"
.venv/bin/python -c "import sys; from pathlib import Path; print(Path(sys.executable).with_name('irw-mcp'))"

These commands install the local checkout. For an index installation, use "irw[mcp]>=0.1.4" only after that release is published; 0.1.3 does not include this server. On Windows, use .venv\Scripts\python.exe instead.

Authenticate with Redivis before first use. The Redivis SDK's interactive browser login cannot complete inside an MCP server, so the server refuses to start a call without credentials (error code authentication_required) rather than hanging. Either run one call in a regular terminal -- python -c "import irw; irw.list_tables()" -- which caches credentials in ~/.redivis, or set REDIVIS_API_TOKEN in the MCP host's environment.

Configure an MCP host to start the installed launcher by its absolute path. Add the irw entry to any existing mcpServers object; do not replace other servers. Tokens set in a terminal are not necessarily inherited by a desktop application: configure the host's environment without committing secrets.

{
  "mcpServers": {
    "irw": {
      "command": "/absolute/path/to/.venv/bin/irw-mcp",
      "args": []
    }
  }
}

fetch_table and get_itemtext return bounded pages (default 100 rows, offset for the next page, has_more and truncated fields; maximum 1,000 response rows and 500 item-text rows). These are storage-order previews, not random samples. Follow next_offset; a full page does not prove another row exists. Ordinary response-data windows are bounded on the wire: fetch_table passes max_rows and columns to irw.fetch(), which hands both to Redivis's read session. Each page reads offset + limit rows, including the prefix discarded for later pages; this sum is capped at 10,000. Item text may require fetching an entire text shard before paging. Rows come back columnar -- columns names the fields and each entry of rows is a list of values in that order -- which is about half the response size of repeating every column name on every row.

Because only the window is downloaded, total_rows is null and has_more reports whether the window came back full; total_rows_estimate carries the catalogue's response count. Claiming the window size as the table's size would be the more convenient answer and the wrong one.

wide=true and dedup=true are the exception. Both are computed over the whole table -- dedup can only drop the duplicates it can see, and the reshape uses whatever rows it is given -- so they cannot be bounded to a page. Those calls download the table, say so in warnings, and are refused above 1,000,000 responses (error table_too_large) or when size is unknown (table_size_unknown), following the size threshold the agents briefing (llms.txt) gives researchers.

Responses have a 256 KiB application-JSON limit. Pages shrink only by complete records and include next_offset; indivisible oversized records return an error, not shortened wording or citations. columns selects output columns, including item columns after wide=true reshaping. Numeric filter values must be finite numbers or one/two-element numeric lists; malformed filters fail.

Processing notes are best-effort header extraction, not a complete execution trace or proof of suitability. Ambiguous script families are reported without choosing a script. Item text distinguishes available, unavailable, and fetch_failed; an expected text retrieval failure is not an empty dataset.

The tool descriptions carry the traps the briefing documents: tags are incomplete, so an untagged table is not a non-match; longitudinal is a grep of the variable string; response direction is not recoded across items; duplicate id-item rows can be real data; and the deposit licence is not an instrument licence. The per-filter caveats come from the package's own FILTER_DESCRIPTIONS, so a caveat added there reaches an assistant without anyone editing the server. Item text may be reconstructed or incomplete; verify it against the original source, and read rights.public_notes (withdrawn wording, machine translations, known mismatches) before using it.

rights.response_data_license is IRW's Derived License for the response data. IRW records an Original License for the source deposit separately, and it is not carried in the metadata the package reads, so rights reports it as null with a note rather than passing the derived licence off as an answer about the source.

The process uses stdio, so it is intended to be launched by a local MCP host. It is not a hosted HTTP endpoint and cannot be called directly by a static GitHub Pages browser widget.

Development

Setting up Development Environment

  1. Clone the repository:

    git clone https://github.com/itemresponsewarehouse/Python-pkg.git
cd Python-pkg

  2. Create a virtual environment:

    python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

  3. Install development dependencies:

    pip install -e .

Feedback and Contributions

If you encounter issues or have suggestions for improving irw, please submit them on the GitHub Issues page. Contributions are welcome!