Skip to content

Sync DocETL and DocWrangler config parameters #429

@shreyashankar

Description

@shreyashankar

Problem

  • There is no single place to view or edit DocETL global config (default model, rate limit, system prompt, etc.). DocWrangler can set only some params, and they drift.

Proposal

  1. Single source of truth in DocETL: new GlobalConfig and Python API.
  • Add docetl.api.get_global_config() and docetl.api.update_global_config(overrides: dict).
  • Load/validate via a schema; persist to file/env; optionally expose read-only endpoint in server.
  1. DocWrangler integration
  • Fetch read-only/display all params.
  • Allow setting an optional docwrangler_config_overrides: dict for allowed keys; merge with DocETL defaults.
  • Enforce allowlist (e.g., default model, system prompt, temperature, max tokens, rate limit).

Sync semantics

  • Merge: DocWrangler overrides take precedence; timestamp updates for audit.
  • Validate types/ranges; apply to running jobs where safe; otherwise next run.

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentationenhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions