Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -251,6 +251,7 @@ pip install "flowcept[mongo]" # MongoDB support
pip install "flowcept[webservice]" # REST API and web UI
pip install "flowcept[dask]" # Dask adapter
pip install "flowcept[mlflow]" # MLflow adapter
pip install "flowcept[codex]" # Codex session-log adapter
pip install "flowcept[rabbitmq]" # RabbitMQ MQ
pip install "flowcept[kafka]" # Kafka MQ
pip install "flowcept[telemetry]" # CPU/memory telemetry
Expand Down
5 changes: 5 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,6 +142,7 @@ Adapter setup is additive:
flowcept --init-settings --dask -y
flowcept --init-settings --mlflow -y
flowcept --init-settings --tensorboard -y
flowcept --init-settings --codex -y
```

These commands add `adapters.<name>` to the current settings file.
Expand Down Expand Up @@ -262,6 +263,10 @@ Adapters:
- `examples/tensorboard_example.py`
- `notebooks/tensorboard.ipynb`
- `tests/adapters/test_tensorboard.py`
- Codex adapter:
- `examples/codex_example.py`
- `tests/adapters/test_codex_interceptor.py`
- DPL skill docs: `resources/skills/agent-loop-provenance/README.md`

Agentic provenance / MCP:

Expand Down
1 change: 1 addition & 0 deletions docs/cli-reference.rst
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ Adapter flags are additive:
flowcept --init-settings --dask -y
flowcept --init-settings --mlflow -y
flowcept --init-settings --tensorboard -y
flowcept --init-settings --codex -y

They add ``adapters.<name>`` to the current settings file instead of replacing it.

Expand Down
2 changes: 2 additions & 0 deletions docs/prov_capture.rst
Original file line number Diff line number Diff line change
Expand Up @@ -170,6 +170,7 @@ Supported adapters:
- **MLflow** — `MLflow example <https://github.com/ORNL/flowcept/blob/main/examples/mlflow_example.py>`_
- **Dask** — `Dask example <https://github.com/ORNL/flowcept/blob/main/examples/dask_example.py>`_
- **TensorBoard** — `TensorBoard example <https://github.com/ORNL/flowcept/blob/main/examples/tensorboard_example.py>`_
- **Codex** — `Codex example <https://github.com/ORNL/flowcept/blob/main/examples/codex_example.py>`_

Install the extras you need (see `installation <setup.html>`_), then configure the adapter in your settings file.
Adapters capture runs, tasks, metrics, and artifacts and push them through Flowcept’s pipeline (MQ → DB).
Expand Down Expand Up @@ -663,6 +664,7 @@ References & Examples
- MLflow adapter: https://github.com/ORNL/flowcept/blob/main/examples/mlflow_example.py
- Dask adapter: https://github.com/ORNL/flowcept/blob/main/examples/dask_example.py
- TensorBoard adapter: https://github.com/ORNL/flowcept/blob/main/examples/tensorboard_example.py
- Codex adapter: https://github.com/ORNL/flowcept/blob/main/examples/codex_example.py
- Loop instrumentation: https://github.com/ORNL/flowcept/blob/main/examples/instrumented_loop_example.py
- LLM/PyTorch model: https://github.com/ORNL/flowcept/blob/main/examples/llm_complex/llm_model.py
- MCP Agent tasks: https://github.com/ORNL/flowcept/blob/main/examples/agents/aec_agent_mock.py
Expand Down
12 changes: 6 additions & 6 deletions docs/schemas.rst
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ It is described in:

PROV-AGENT names the main building blocks you see in modern AI systems:

- **Activities** such as Campaign, Workflow, Task, AIModelInvocation, and AgentTool
- **Activities** such as Campaign, Workflow, Task, AIModelInvocation, and ToolInvocation
- **Agents** such as an AI agent or a human user
- **Data Objects** such as domain data, prompts, responses, scheduling info, and telemetry
- **Relations** such as *used*, *wasGeneratedBy*, *wasAssociatedWith*, *wasAttributedTo*, and *wasInformedBy*
Expand Down Expand Up @@ -64,21 +64,21 @@ Use the :class:`~flowcept.commons.vocabulary.PROV_AGENT` enum to set these value
Captured automatically by :class:`~flowcept.instrumentation.flowcept_agent_task.FlowceptLLM`.
``used.prompt`` stores the input; ``generated.response`` stores the output;
``custom_metadata.llm_usage`` stores token counts.
* - ``PROV_AGENT.AGENT_TOOL``
- ``agent_tool``
- A tool execution by an AI agent (*AgentTool* in PROV-AGENT).
* - ``PROV_AGENT.TOOL_INVOCATION``
- ``tool_invocation``
- A tool execution by an AI agent (*ToolInvocation* in PROV-AGENT).
Captured automatically by the
:func:`~flowcept.instrumentation.flowcept_agent_task.agent_flowcept_task` decorator
applied to MCP tools and LangGraph tool nodes.
``used`` stores tool arguments; ``generated`` stores the return value.

The ``wasInformedBy`` relation — an ``AgentTool`` activity informing an ``AIModelInvocation`` — is
The ``wasInformedBy`` relation — a ``ToolInvocation`` activity informing an ``AIModelInvocation`` — is
the key link for root-cause analysis and downstream impact tracing in PROV-AGENT. In Flowcept this
is expressed through the ``agent_id`` field: every task with the same ``agent_id`` belongs to the
same AI agent and can be queried together to reconstruct the full agent provenance graph.

The UI uses ``subtype`` to visually distinguish AI agent activities from regular workflow tasks.
Filter for ``subtype == "ai_model_invocation"`` or ``subtype == "agent_tool"`` to isolate agent
Filter for ``subtype == "ai_model_invocation"`` or ``subtype == "tool_invocation"`` to isolate agent
interactions from the provenance database.

Figure
Expand Down
15 changes: 15 additions & 0 deletions docs/setup.rst
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ Good practice is to cherry-pick the extras relevant to your workflow instead of
pip install flowcept[mlflow] # MLflow adapter
pip install flowcept[dask] # Dask adapter
pip install flowcept[tensorboard] # TensorBoard adapter
pip install flowcept[codex] # Codex session-log adapter
pip install flowcept[rabbitmq] # RabbitMQ message queue
pip install flowcept[kafka] # Kafka message queue
pip install flowcept[nvidia] # NVIDIA GPU runtime capture
Expand Down Expand Up @@ -250,6 +251,20 @@ Adapter flags are additive:
flowcept --init-settings --dask -y
flowcept --init-settings --mlflow -y
flowcept --init-settings --tensorboard -y
flowcept --init-settings --codex -y

Codex declared provenance
~~~~~~~~~~~~~~~~~~~~~~~~~

The Codex adapter can capture standard Codex session logs without extra
assistant instructions. To capture declared PROV-Agent-Loop semantics such as
execution plans, plan steps, loop iterations, evaluations, criteria, decisions,
beliefs, memories, and lessons learned, install the bundled
``resources/skills/agent-loop-provenance`` skill and enable
``adapters.codex.declared_provenance_enabled`` in the settings file.

See ``resources/skills/agent-loop-provenance/README.md`` for the skill
installation and usage instructions.

Custom Settings File
---------------------
Expand Down
45 changes: 45 additions & 0 deletions examples/codex_example.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
"""Observe a Codex session JSONL file with Flowcept.

Setup:
pip install "flowcept[codex]"
flowcept --init-settings --codex -y

Then edit your Flowcept settings file:
adapters.codex.file_path: /path/to/codex/session.jsonl
adapters.codex.declared_provenance_enabled: false

With ``declared_provenance_enabled: false`` the adapter runs in OPL mode and
captures the provenance it can infer directly from the Codex JSONL log.

For DPL mode, install the bundled agent-loop-provenance skill before starting
the Codex session, then set:
adapters.codex.declared_provenance_enabled: true

The skill instructions live in:
resources/skills/agent-loop-provenance/README.md

Run this observer while the Codex session is active, or replay an existing JSONL
by pointing ``adapters.codex.file_path`` to that file.
"""

from time import sleep

from flowcept import Flowcept
from flowcept.configs import settings


if __name__ == "__main__":
# Configure adapters.codex.file_path in settings.yaml before running this.
file_path = settings["adapters"]["codex"]["file_path"]
declared = settings["adapters"]["codex"].get("declared_provenance_enabled", False)
print(f"Codex JSONL path: {file_path}")
print(f"Declared provenance enabled: {declared}")

with Flowcept("codex", save_workflow=False) as flowcept:
print("Codex adapter running. Press Ctrl+C to stop.")
try:
while True:
sleep(2)
print(f"records in buffer: {len(flowcept.get_buffer())}")
except KeyboardInterrupt:
print("Stopping Codex adapter...")
Loading
Loading