Skip to content

Latest commit

 

History

History
256 lines (190 loc) · 10.9 KB

File metadata and controls

256 lines (190 loc) · 10.9 KB

OfficeQA Benchmark Research

Date: 2026-05-12 Researcher: Jessica Irwin (Gru.ai) Source: Databricks Blog | Paper (arXiv) | GitHub


1. Overview

OfficeQA is Databricks' open-source benchmark for evaluating AI agents on grounded reasoning — answering questions that require searching, extracting, and analyzing information from complex real-world document collections.

Unlike academic benchmarks (ARC-AGI, MMLU, Humanity's Last Exam) that test abstract reasoning or parametric knowledge, OfficeQA focuses on economically valuable enterprise tasks: answering precise questions grounded in proprietary, unstructured data.

Key Stats

  • Corpus: ~89,000 pages of U.S. Treasury Bulletins (1939–present)
  • Document types: Scanned physical PDFs (pre-1996) + digital-native PDFs
  • Data density: 26+ million numerical values across the corpus
  • Questions: 246 total (133 Pro + 113 Easy)
  • License: Apache 2.0 (code) / CC-BY-SA 4.0 (data)

2. Benchmark Design

Two Tiers

Tier Count Description
OfficeQA Pro 133 Hard questions — the primary benchmark for frontier models
OfficeQA Full 246 Includes 113 easier questions for system optimization

"Easy" questions are those answered correctly by both leading frontier agents (GPT-5.1 and Claude Opus 4.5 at time of initial release). Pro questions require more complex multi-step reasoning.

Question Characteristics

  • 62% require analytical reasoning beyond basic arithmetic
  • 22% involve internet search for external values
  • 11% require data from 3+ separate bulletins
  • 3% require visual reasoning over charts/figures
  • All questions have single, unambiguous correct answers
  • Questions span 1–20+ pages of source material

Question Creation Process

  1. Annotators generate questions by referencing corpus documents
  2. A second annotator independently reproduces the answer using the same documents
  3. AI agent conflict resolution through a three-reviewer system
  4. Questions answerable by parametric knowledge alone are filtered out
  5. Difficulty classification based on frontier agent performance

Databricks partnered with SuperAnnotate for annotation infrastructure.

Evaluation Methodology

Deterministic exact-match evaluation with configurable tolerance:

Absolute Relative Error (%) = |predicted - actual| / |actual| × 100
Tolerance Description
0.0% Exact match (default for reporting)
0.1% Minor rounding differences
1.0% Small numerical variations
5.0% Approximate answers

Non-numerical answers use fuzzy text matching with punctuation normalization.


3. Performance Results

LLM Baselines (No Agent, Direct Prompting)

Configuration Claude Opus 4.6 GPT-5.4 Gemini 3.1 Pro
Prompt Only (no docs) 2.26% 0.75% 2.26%
With Web Search 3.01% 11.28% 3.01%
Oracle PDF Pages (raw) 36.09% 56.39% 51.13%
Oracle Parsed Pages 57.14% 65.41% 56.39%

Key insight: Without document access, frontier models score ~2%. Even the best parametric knowledge cannot answer grounded questions.

Agent Baselines (Full Corpus Access)

Agent Framework Model Format Accuracy
Claude Agent SDK Opus 4.6 Raw PDF 48.12%
Claude Agent SDK Opus 4.6 Parsed 54.14%
Codex CLI GPT-5.4 Raw PDF 36.09%
Codex CLI GPT-5.4 Parsed 56.39%
Gemini CLI Gemini 3.1 Raw PDF 18.05%
Gemini CLI Gemini 3.1 Parsed 29.32%

Best agent result: Claude Opus 4.6 with parsed corpus = 54.14% (full corpus), 66.92% (oracle pages)

Document Parsing Impact

Databricks' ai_parse_document tool dramatically improves results:

  • Claude Opus 4.5: +30.2 percentage points with parsing
  • GPT-5.1: +9.1 percentage points with parsing
  • Average relative performance gain: 16.1%

Human Performance (30-question subset)

Setting Accuracy Avg. Latency
Oracle Pages (Human) 51.1% 19.2 min
Oracle Pages (Agent) 71.1–72.2% 2.1–5.3 min
Full Corpus (Human) 34.6% 31.4 min
Full Corpus (Agent) 50.0–56.7% 3.5–15.4 min

Agents consistently outperform humans in both speed and accuracy on this benchmark, though humans excel at identifying layout-based formatting issues in raw PDFs.


4. Five Critical Failure Modes

1. Temporal Revision Verification

Agents prematurely converge on the first numerically plausible value instead of identifying the most recent revision. Treasury Bulletins frequently revise earlier figures, and agents fail to verify temporal currency, leading to cascading errors.

2. Parsing Faithfulness

Even state-of-the-art document parsing introduces errors:

  • Misread numbers (OCR artifacts)
  • Corrupted text from scanning
  • Misaligned table topology (columns/rows shifted)
  • These errors propagate through downstream calculations

3. Visual Understanding

"Agent systems lack the granular resolution to correctly interpret dense financial charts and struggle to map visual trend lines with high precision."

Only 3% of questions require visual reasoning, but agents fail disproportionately on these.

4. Analytical Reasoning

Agents retrieve contextually misaligned alternatives, apply mathematically valid but contextually wrong formulas, and ignore explicit calculation requirements stated in the question.

5. Latency

  • 23.6 minutes average per question over full PDF corpus
  • 3.9 minutes with pre-parsed documents
  • Enterprise use cases demand faster turnaround

5. Comparison with Other Benchmarks

OfficeQA vs. GDPVal (OpenAI)

Dimension OfficeQA GDPVal
Creator Databricks OpenAI
Focus Grounded reasoning over documents Economically valuable work product
Scope Deep (single domain, financial docs) Broad (44 occupations, 9 industries)
Tasks 246 questions with deterministic answers 1,320 tasks requiring multimodal deliverables
Corpus ~89K pages of Treasury Bulletins Various professional scenarios
Evaluation Deterministic exact-match with tolerance Human expert pairwise comparison
Output Numerical/text answers PDFs, spreadsheets, presentations, etc.
Question "Can AI accurately reason over enterprise data?" "Can AI replace a knowledge worker?"

OfficeQA vs. ARC-AGI-2 / Humanity's Last Exam

Dimension OfficeQA ARC-AGI-2 / HLE
Task type Document grounded QA Abstract reasoning / expert knowledge
Data dependency Requires external corpus Self-contained
Enterprise relevance High (directly mimics real work) Low (academic/theoretical)
Evaluation Numerical accuracy Pattern matching / expert verification

Bottom line: GDPVal asks "can AI replace a knowledge worker?" (breadth). OfficeQA asks "can AI accurately answer questions grounded in real documents?" (depth). They are complementary.


6. Feasible Approaches to Build Similar Benchmarks

Approach 1: Public Document Corpora (Lowest Cost)

Use freely available, information-dense collections:

  • SEC EDGAR filings (10-K, 10-Q, proxy statements)
  • Patent databases (USPTO full-text)
  • Government reports (GAO, CBO, census)
  • Academic papers (arXiv, PubMed)
  • Legal case files (PACER, state courts)
  • Municipal budgets and city planning documents

Approach 2: Synthetic-then-Verify Pipeline

  1. Use a frontier LLM to generate candidate questions from documents
  2. Have the LLM attempt to answer them
  3. Human annotators verify/correct answers (~3-5x cheaper than full human authoring)
  4. Filter questions solvable by parametric knowledge alone

Approach 3: Enterprise Partner Data

Partner with companies for anonymized internal docs:

  • HR policy manuals + compliance questions
  • Technical documentation + troubleshooting scenarios
  • Financial reports + analytical queries
  • Contract repositories + legal reasoning tasks

Approach 4: Multi-Modal Extension

Extend beyond PDFs:

  • Spreadsheets with formulas and pivot tables
  • Slide decks with embedded charts
  • Email threads with attachments
  • Database exports (CSV/JSON)
  • Mixed-format document packages

Approach 5: Domain Expert Crowdsourcing

Using platforms like Scale AI or Surge AI:

  1. Recruit domain experts (accountants, lawyers, analysts)
  2. Provide a document corpus
  3. Have them write questions they'd actually ask in their job
  4. Cross-validate with a second expert
  5. Filter by difficulty using AI agent baselines

Cost Estimates

Method Cost per Question 250 Questions
Full human annotation $50–100 $12.5K–25K
Synthetic + human verify $15–30 $3.75K–7.5K
Domain expert crowdsource $30–60 $7.5K–15K

Design Principles (from OfficeQA methodology)

  • Questions must require the corpus (filter out parametric knowledge)
  • Answers should be deterministic and verifiable
  • Include multi-step reasoning (not just lookup)
  • Mix difficulty levels based on agent performance
  • Use strict tolerance evaluation (0.0% error default)
  • Include both scanned and digital documents
  • Cross-annotator verification is essential

7. Relevance to Gru.ai

OfficeQA highlights critical gaps in AI agent capabilities for enterprise document understanding:

  1. Document parsing quality is a major differentiator (+16-30pp accuracy gains)
  2. Agent frameworks matter — Claude Agent SDK consistently outperforms alternatives
  3. Real enterprise work requires grounding in proprietary data, not parametric knowledge
  4. Building domain-specific benchmarks could differentiate Gru.ai in vertical markets
  5. The Grounded Reasoning Cup (Spring 2026) presents an opportunity for visibility

Potential Actions

  • Evaluate Gru.ai agent performance on OfficeQA Pro
  • Build a domain-specific benchmark for a target vertical
  • Investigate document parsing improvements for enterprise use cases
  • Consider entering the Databricks Grounded Reasoning Cup

References

  1. Databricks Blog: Introducing OfficeQA (Dec 2025)
  2. arXiv Paper: OfficeQA Pro (Mar 2026)
  3. GitHub: databricks/officeqa
  4. SuperAnnotate Case Study: How SuperAnnotate Helped Build OfficeQA (Feb 2026)
  5. HuggingFace Papers: OfficeQA Pro (Mar 2026)