OCM Skills Plugin — Skill 8 of 8 Status: Complete — Phase 1 and Phase 2 tested. Modes 3 and 4 published with known failure modes (see below).
Executes the annual panel review cycle and ongoing panel governance decisions. Synthesises performance scorecard data, billing compliance records, and step-out patterns into panel-level recommendations. Produces the documentation for firm exits, coverage gap assessment, and competitive refresh processes.
Covers the full panel governance workflow: annual health assessment through to firm exit and slot refresh.
| Mode | Trigger | Output |
|---|---|---|
| 1 — Panel Health Assessment | "Review our panel" / "Annual panel review" / "Is our panel working" | Panel Health Report + GC Panel Review Note |
| 2 — Firm Exit Management | "Exit [firm] from the panel" / "Write the exit notice" / "Formal panel removal" | Firm Exit Notice and Transition Plan + Internal Exit Record |
| 3 — Coverage Gap Analysis | "We have a gap" / "We don't have anyone for [area]" / "Coverage gap" | Coverage Gap Report + Remediation Options Note |
| 4 — Panel Refresh Brief | "Refresh the panel" / "Fill the gap" / "Competitive process for [area]" | Panel Refresh Brief + RFP Scope Note |
Improvement plan before exit. The domain knowledge encodes the improvement plan requirement: exit on performance grounds without a prior improvement plan is harder to defend. Mode 2 doesn't make this a binary gate — the Internal Exit Record has a "Prior improvement plan" field that documents the history.
Mode 2 Type 2 placeholder. The Firm Exit Notice uses [Firm] as a deliberate review gate — it stays as [Firm] in the external document even when the firm name is provided in session. The internal exit record uses the firm's actual name. Same pattern as the Formal Non-Compliance Notice in invoice-review-compliance (Skill 6).
Step-out as a leading indicator. Step-out frequency connects to three diagnoses: coverage gap, underperforming panel firm, or panel discipline failure. This is the methodology connecting Mode 3 (gap identification) back to Mode 1 (panel health) and forward to Mode 4 (refresh).
Mode 4 as hand-off document. The Panel Refresh Brief is framed as a scoped input to rfp-pitch-management (Skill 3), not a standalone decision. Template includes a hand-off instruction block naming the appropriate Skill 3 mode as the next step.
Consequence framework. Mode 1 Panel Health Report maps each firm to one of four recommendations: Retain, Watch, Improvement Plan, or Exit Review. Each recommendation has defined trigger criteria and consequences.
Modes 3 and 4 have documented failure modes identified during Phase 2 testing. Four instruction-based fix attempts on Mode 3 and three on Mode 4 produced no change in the failure pattern.
Mode 3 failure pattern: Produces consulting prose with numbered strategic options instead of the structured Coverage Gap Report template. Closes with offer or question menu.
Mode 4 failure pattern: Produces Skill 3 execution documents (RFP, longlist, capabilities questionnaire with named firms) instead of the Panel Refresh Brief template. Web search runs pre-document. Account-level context may appear.
Root cause: Rich domain knowledge overrides template instructions when input is conversational (Mode 3) or execution-oriented (Mode 4). The model ignores first-token anchors and IS/IS NOT definitions when it has sufficient domain knowledge to generate plausible alternative content.
Impact: Modes 3 and 4 will produce useful content on the right topic but may not follow the template structure. Modes 1 and 2 are clean.
Future revision path: Two-step intake exchange before document production, or fundamentally different design accepting conversational register and routing to structured output differently.
| Stage | Skill |
|---|---|
| Billing rules | engagement-terms-billing-guidelines (Skill 1) |
| Panel design | panel-design-selection (Skill 2) |
| Firm selection | rfp-pitch-management (Skill 3) |
| Fee arrangements | fee-arrangement-structuring (Skill 4) |
| Matter instruction | matter-allocation-instruction (Skill 5) |
| Invoice review | invoice-review-compliance (Skill 6) |
| Performance management | performance-scorecard (Skill 7) |
| Panel review | panel-review-rationalisation (Skill 8) ← this skill |
| Skill | Connection |
|---|---|
| performance-scorecard (Skill 7) | Scorecard tier per firm feeds Mode 1 Panel Health Assessment. Primary data input. |
| invoice-review-compliance (Skill 6) | Billing compliance rate and violation records feed Mode 1. |
| matter-allocation-instruction (Skill 5) | Step-out log and matter allocation records feed Mode 1 and Mode 3 gap identification. |
| panel-design-selection (Skill 2) | Defines original panel structure and selection criteria against which review assesses each firm. |
| rfp-pitch-management (Skill 3) | Receives Mode 4 Panel Refresh Brief as input. Executes the competitive process to fill an identified slot. |
| engagement-terms-billing-guidelines (Skill 1) | OCG adherence is a panel retention criterion in Mode 1. |
Phase 1 and Phase 2 test prompts and assertion sets are in the build log.
panel-review-rationalisation/SKILL.md— skill instructionspanel-review-rationalisation-README.md— this document
Apache 2.0 — LegalOps Consulting Limited