Skip to content

[OPIK-4720] Prompt create permission#5909

Open
anastasiapyzhik wants to merge 12 commits intomainfrom
nastassia/OPIK-4720/prompt-create-permission
Open

[OPIK-4720] Prompt create permission#5909
anastasiapyzhik wants to merge 12 commits intomainfrom
nastassia/OPIK-4720/prompt-create-permission

Conversation

@anastasiapyzhik
Copy link
Copy Markdown
Contributor

@anastasiapyzhik anastasiapyzhik commented Mar 27, 2026 •
edited
Loading

Details

Add prompt_create permission to the frontend and apply checks across all prompt creation entry points in both v1 and v2.

Permission infrastructure:

  • Add PROMPT_CREATE to ManagementPermissionsNames enum
  • Add canCreatePrompts to the Permissions interface, defaults, hook, and provider

Prompts Page (v1 & v2):

  • Hide "Create new prompt" button when user lacks canCreatePrompts permission
  • Hide the empty-state "Create new prompt" link as well

Playground (v1 & v2):

  • Disable the Save button in PlaygroundPrompt when !canCreatePrompts && !selectedChatPromptId (no permission and no existing prompt loaded)

Optimizations (v1 & v2):

  • Disable the Save button in OptimizationsNewPromptSection with the same logic

AddNewPromptVersionDialog (v1 & v2):

  • Hide the "Update existing / Save as new" toggle when user lacks create permission — dialog defaults to "update" mode
  • Gate canSaveNewPrompt validation with canCreatePrompts so the submit button is disabled for new prompts

LLMPromptMessageActions (v1 & v2):

  • Disable the Save button when !canCreatePrompts && !promptId

Change checklist

  • Permission enum added to ManagementPermissionsNames
  • canCreatePrompts wired through useUserPermission → PermissionsProvider → Permissions interface
  • All usePromptCreateMutation consumer paths guarded
  • Both v1 and v2 updated consistently

Testing

  1. With permission: Verify all prompt creation flows work as before (create from Prompts page, save as new from Playground/Optimizations, save as new from AddNewPromptVersionDialog)
  2. Without permission:
    • Prompts page: "Create new prompt" button should be hidden
    • Playground: Save button disabled when no prompt loaded
    • Optimizations: Save button disabled when no prompt loaded
    • AddNewPromptVersionDialog: Toggle hidden, submit disabled for new prompts; updating existing prompts still works

Issues

OPIK-4720

Documentation

N/A

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 27, 2026

📋 PR Linter Failed

❌ Incomplete Issues Section. You must reference at least one GitHub issue (#xxxx), Jira ticket (OPIK-xxxx), CUST ticket (CUST-xxxx), DEV ticket (DEV-xxxx), or DND ticket (DND-xxxx) under the ## Issues section.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 27, 2026

📋 PR Linter Failed

❌ Incomplete Issues Section. You must reference at least one GitHub issue (#xxxx), Jira ticket (OPIK-xxxx), CUST ticket (CUST-xxxx), DEV ticket (DEV-xxxx), or DND ticket (DND-xxxx) under the ## Issues section.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 27, 2026

📋 PR Linter Failed

❌ Incomplete Issues Section. You must reference at least one GitHub issue (#xxxx), Jira ticket (OPIK-xxxx), CUST ticket (CUST-xxxx), DEV ticket (DEV-xxxx), or DND ticket (DND-xxxx) under the ## Issues section.

@anastasiapyzhik anastasiapyzhik marked this pull request as ready for review March 27, 2026 11:05
@anastasiapyzhik anastasiapyzhik requested a review from a team as a code owner March 27, 2026 11:05
anastasiapyzhik and others added 5 commits March 27, 2026 12:07
@anastasiapyzhik
Merge branch 'main' into nastassia/OPIK-4720/prompt-create-permission
afe3e00
@anastasiapyzhik
Merge branch 'main' into nastassia/OPIK-4720/prompt-create-permission
2f1539d
@yaricom @anastasiapyzhik
[OPIK-5296] [P SDK] Update OpikClient.create_optimization() to accept…
e655e6d 
… project_name (#5917)

* [OPIK-5296] Add `project_name` to optimization lifecycle and verification logic in Python SDK

- Added `project_name` parameter to optimization creation, update, and verification methods.
- Updated SDK and E2E test cases to validate `project_name` propagation.
- Introduced backward compatibility tests for OPIK v1.x to ensure new functionality does not break existing workflows.

* [OPIK-5296] Fixed import

* [OPIK-5296] Add `project_name` propagation validation to `get_optimization_by_id`

- Updated `get_optimization_by_id` to include `project_name` resolution using associated project's name.
- Enhanced E2E tests to validate `project_name` propagation via assertions.
- Added logging for project retrieval failures during optimization fetching.

* Change logging level from `debug` to `warning` for failed project retrieval in `get_optimization_by_id`.
@miguelgrc @YarivHashaiComet
@claude @anastasiapyzhik
[OPIK-4655] [FE] Playground Redesign (#5771)
9a11ea5 
* [OPIK-4843] [FE] Playground Redesign PR 1: Prompt header and layout restructuring (#5623)

* [OPIK-4843] [FE] Playground Redesign PR 1: Prompt header redesign

Restructure Playground page layout with full-height 'add variant'
 strip, compact prompt headers and simplified component hierarchy.

* [OPIK-4843] [FE] Address PR review: fix import order and remove unnecessary useMemo

* [OPIK-4843] [FE] Center add variant button with prompt section and fix full-height strip

* [OPIK-4843] [FE] fix nested tooltips, use Button component, show actions on hover, prompt colors

- Restructure PromptsSelectBox compact view to avoid nested tooltips
- Replace raw button with Button component for detach action
- Move tooltip inside PromptModelConfigs (remove external wrapper)
- Replace Tag with custom span for prompt letter colors using CSS variables
- Show actions only on the prompt that's being hovered
- Add @media(hover:hover) for mobile-friendly hover actions
- Clean up unused tooltip key/defaultOpen in PlaygroundOutputActions
- Add hover group styling to PlaygroundAdDVariant

* [OPIK-4843] [FE] Use fixed text colors for prompt badges and match Figma icon color

- Replace CSS variable text colors with fixed values for consistent
  contrast regardless of theme (dark text on light bg, white on dark bg)
- Update loaded prompt icon color to match Figma spec (#b8e54a)

* [OPIK-4844] [FE] Playground Redesign PR 2: Message container redesign (#5681)

* [OPIK-4844] [FE] Playground Redesign PR 2: Message container redesign

- Redesigned the prompt message container
- Inline variable hint widget with styled green {{ braces in CodeMirror
- Variable popover shows real preview values from first dataset item
- Bottom action bar only renders when media or variable actions exist
- Fix "+ Message" button spanning full width in prompt optimization
- Exclude transient datasetSampleData from localStorage persistence
- Delete unused PromptMessageMediaSection component

* [OPIK-4844] [FE] Address PR and product review comments

- Switch action buttons from ghost to minimal variant for correct icon colors
- Fix role dropdown re-activating focus state via onCloseAutoFocus
- Add scroll-to-element on message duplicate and scroll-to-bottom on add
- Fix variable insertion position when editor not focused
- Preserve editor focus when clicking add variable button
- Update variable tooltip text
- Remove permanent action visibility on unsaved prompt changes
- Keep bottom action bar visible while media popover is open
- Remove card shadow except during drag
- Update prompt header buttons to minimal variant
- Rename opik-variable CSS classes to comet-variable

* [OPIK-4844] [FE] Move JsonValue/JsonObject types to @/types/shared

Move JSON types from JsonTreePopover component to shared types to fix
dependency cruiser violation (store importing from components).

* [OPIK-4844] [FE] Add comment explaining variable hint useEffect

* [OPIK-4845] [FE] Playground Redesign PR 3: Top Bar + Run on Dataset Modal + Per-Prompt Run (#5750)

* [OPIK-4845] [FE] Playground Redesign PR 3: Top Bar + Run on Dataset Modal + Per-Prompt Run

Restructure the playground page with a new top bar, "Run on dataset" modal dialog,
per-prompt run/stop, and redesigned output display.

Top Bar (PlaygroundHeader):
- Extract header from PlaygroundPrompts into standalone PlaygroundHeader component
- Add experiment chip (dataset name + version, pencil to edit, X to leave)
- "Test on dataset" button opens RunOnDatasetDialog
- Run all / Re-run / Stop all with inline hotkey display (Shift+Enter)
- Ghost Reset button separated by vertical separator
- Full-width gray background extending beyond content area
- "Leave experiment mode?" confirmation with link to Experiments page

Run on Dataset Modal (RunOnDatasetDialog):
- Dataset selector with version support (DatasetVersionSelectBox)
- Metrics combobox (MetricSelector rewrite) with purple RemovableTag chips,
  overflow handling (+N), smooth scroll on expand, delete without toggle
- Filters button for dataset column filtering
- Experiment name prefix auto-fill (DatasetName-YYYY-MM-DD), persisted in store
- Validation: disabled when prompts invalid, running, or dataset empty
- Tooltips on disabled Run button explaining the reason

Per-Prompt Run (non-experiment mode):
- isRunning: boolean -> isRunningMap: Record<string, boolean> in store
- runSingle/stopSingle in useActionButtonActions
- Per-prompt Run/Stop button in output header strip
- Per-prompt validation with tooltip (no model, empty messages)

Output Redesign (PlaygroundPromptOutput):
- Colored square + "Output A" label matching prompt colors
- Provider icon + pretty model label (from useLLMProviderModelsData)
- Clock icon + duration, Coins icon + token count (from output.usage)
- Model/provider/usage captured at generation time, not from current prompt
- Empty state: "No runs yet" with colored icon
- White background, no card wrapper, border-r matching prompt columns
- Value + usage cleared on re-run for immediate loading animation

Experiment Mode Layout:
- Conditional layout: experiment mode separates prompts from outputs
- Table takes full screen width in experiment mode
- Add variant strip only spans prompts area (not outputs)
- PlaygroundExperimentOutputActions: progress bar OR experiment results header
- Experiment results header with link to comparison page, using stored prefix
- Colored dot indicators on output table column headers

Architecture & Refactoring:
- useActionButtonActions lifted to PlaygroundPage (single instance)
- Deferred run pattern (pendingRun state) for dialog-triggered runs
- onRunAll + onDeferredRunAll callbacks (immediate vs wait-for-items)
- Keyboard shortcut (Shift+Enter) in PlaygroundPage, run only (no stop)
- transformDataColumnFilters extracted to lib/filters.ts
- File reorganization: MetricSelector, useActionButtonActions,
  usePromptDatasetItemCombination moved to PlaygroundPage level
- PlaygroundOutput renamed to PlaygroundPromptOutput
- PlaygroundOutputActions renamed to PlaygroundExperimentOutputActions
- experimentNamePrefix added to Zustand store
- ConfirmDialog.description changed from string to ReactNode
- HotkeyDisplay xs size variant added
- DatasetVersionSelectBox: version name/hash right-aligned in trigger
- DEFAULT_LOADED_DATASETS imported from shared source
- MAX_VERSIONS_TO_FETCH exported from useValidatedDatasetVersion

Store Changes (PlaygroundStore):
- isRunning -> isRunningMap with per-prompt selectors
- Added usage field (duration, totalTokens, model, provider) to PlaygroundOutput
- Added experimentNamePrefix state
- Removed pendingRunAll (replaced with local state)

* [OPIK-4845] [FE] Extract buildDatasetFilterColumns helper

Extract shared dataset filter column builder to lib/filters.ts to
deduplicate the pattern across RunOnDatasetDialog and dataset pages.
Accepts optional includeId parameter for dataset pages that need an
ID filter column.

* [OPIK-4845] [FE] Use correct column types in dataset filter columns

Use mapDynamicColumnTypesToColumnType instead of hardcoded COLUMN_TYPE.string
so filter operators match the actual column data types.

* [OPIK-4846] [FE] Playground Redesign PR 4: Split-Scroll Results Table (#5755)

* [OPIK-4846] [FE] Playground split-scroll results table

- Split PlaygroundOutputTable into two independent DataTable instances
  (left: variables + tags, right: output columns per prompt)
- Resizable divider between table halves using re-resizable
- Each half scrolls horizontally independently
- Page scrolls vertically, table fits viewport width
- Prompts section scrolls horizontally on its own, header stays fixed
- Output column headers show provider icon + model name
- Extract usePromptModelDisplay hook for provider/model display logic

* [OPIK-4846] [FE] Sticky table headers via split header/body DataTables

Standard stickyHeader doesn't work here because each table half needs
overflow-x:auto for independent horizontal scrolling, which creates a
scroll container that captures position:sticky.

Workaround: StickyScrollTable renders two DataTables per half — a sticky
header (tbody hidden) and a scrollable body (thead hidden) with synced
horizontal scroll.

* [OPIK-4846] [FE] Fix blank output on first load

Use null instead of "" when clearing output after model validation,
so hasOutput correctly falls back to "No runs yet" placeholder.

* [OPIK-4846] [FE] Address PR review: clone before sort, output usage in header

- Clone datasetColumns before sorting to avoid mutating React Query cache
- Output column header now shows model from actual run usage instead of
  prompt config, consistent with non-experiment mode
- Add useFirstOutputUsageByPromptId store hook (needed because the header
  DataTable in StickyScrollTable has no rows to read usage from)

* [OPIK-4655] [FE] v2 rebase fixups: restore v1 PlaygroundStore compat, remove unused import

- Restore isRunning/setIsRunning in PlaygroundStore for v1 playground
  compatibility (v2 uses isRunningMap for per-prompt tracking)
- Remove unused COLUMN_TYPE import in EvaluationSuiteItemsTab after
  buildDatasetFilterColumns refactor

* [OPIK-4655] [FE] Playground product review feedback

- Disable "Test on dataset" when prompts have validation issues
- Simplify experiment results bar (clickable "Experiment results" link, no name)
- Remove experiment name prefix field from Run on Dataset dialog
- Add variant strip: fully clickable with hover highlight, auto-scroll on add
- Fix duplicate variant copying library version instead of current messages
- White background for empty space on wide screens
- Prompt header: fixed-width label, truncation balance between model/prompt selectors
- Message actions: buttons don't shrink, only prompt chip truncates
- Min prompt width increased to 400px, prompt width no longer grows on content change
- Remove header/run button separator
- Dataset chip border in experiment mode
- Experiment results bar spans full width
- Sticky grabber icon on table resize handle
- HotkeyDisplay simplified: single transparent style, removed variants

* [OPIK-4655] [FE] Run on dataset dialog: don't trigger experiment run on dataset selection

- Dialog sets dataset without running, button renamed to "Use dataset"
- User clicks "Run experiment" to start manually
- Run button: "Run experiment" before first run, "Re-run" after
- Extracted PlaygroundNoRunsYet shared component for empty output state
- Output cells show "No runs yet" before experiment runs
- Dialog padding adjustments
- Removed deferred run mechanism
- Added minWidth on experiment mode prompts wrapper

* [OPIK-4655] [FE] Address PR review: HotkeyDisplay variants, run guards, Shift+Enter

- HotkeyDisplay: replace inline rgba with variant system (default for
  transparent overlay, outline for bordered style in SME flow)
- Restore variant="outline" on SME AnnotationView and CommentAndScoreViewer
  HotkeyDisplay elements (v1 + v2)
- PlaygroundHeader: add dataset availability guard to isRunDisabled
  (disabled when dataset removed in experiment mode)
- RunOnDatasetDialog: add !datasetName to isRunDisabled to match
  handleRun early-return
- Move Shift+Enter shortcut from PlaygroundPage to PlaygroundHeader
  so it respects isRunDisabled validation

* [OPIK-4655] [FE] Address PR review: filter tests, MetricSelector Set optimization

- Add Vitest tests for buildDatasetFilterColumns and
  transformDataColumnFilters (9 test cases covering type mapping,
  data prefix transforms, edge cases)
- MetricSelector: memoize selectedRuleIds into a Set for O(1)
  membership checks instead of repeated Array.includes()

* [OPIK-4655] [FE] Product feedback: header size, prompt bg, message padding, loader

- Header title: comet-title-s (18px) → comet-title-xs (14px)
- Prompt sections: bg-soft-background for grayish prompt area,
  bg-background on main container for white empty space
- Messages without action bar: pb-3 default bottom padding
- Fix page loader: use shared Loader component instead of lucide icon

* [OPIK-4655] [FE] QA and product feedback fixes

- Fix filter flicker in RunOnDatasetDialog: defer FiltersButton onChange
  to popover close to prevent circular query/re-render cycle
- Fix dataset chip "null v44" on hard refresh: show "Loading..." while
  dataset name resolves
- Fix experiment results link: use v2 project-scoped navigation hook
- Show "Experiment results" bar in experiment mode before first run
- Prompts list refetches on mount so new prompts appear after navigation
- Add tooltip on disabled "Use dataset" button (empty/filtered dataset)
- Prevent auto-focus tooltip flash on dataset dialog open
- Header title to 14px, all buttons to 24px height (2xs)
- Dataset chip: 24px height, smaller text and icons
- +Message and Run/Stop buttons: 24px height
- Message role dropdown: muted-slate color
- Message bottom padding: pb-3 for non-action messages
- Always "Run experiment" label (remove Re-run switch)
- HotkeyDisplay: bg-[#FFFFFF33] default variant, add 2xs size
- Page loader: use shared Loader component instead of lucide icon
- Prompt sections: bg-soft-background with bg-background main container

* [OPIK-4655] [FE] Fix SSE [DONE] parse error and enable Test on dataset button after reset

- Skip SSE [DONE] end-of-stream marker before JSON.parse to prevent SyntaxError
- Remove disabled state from Test on dataset button so it's always clickable in non-experiment mode (the run itself is still gated in the dialog and header)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* [OPIK-4655] [FE] Fix metric edit link to use project-scoped URL

The edit link in the MetricSelector was navigating to
/${workspaceName}/online-evaluation which didn't match any route.
Fixed to include /projects/${projectId}/ so the Online Evaluation
page loads correctly and opens the Edit rule dialog.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* [OPIK-4655] [FE] Fix trace button click target in stale output cells

The trace button was only clickable at its bottom edge because the
scores container div overlapped it. Adding z-10 ensures the button
sits above the scores container and is fully clickable.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* [OPIK-4655] [FE] Reduce dataset list max height to keep Add new button visible

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* [OPIK-4655] [FE] Fix filters list scroll and height when inside modal dialog

- Reduce filters list max-height from 50vh to 30vh so Add filter button stays visible
- Add max-h-[70vh] to PopoverContent to cap overall popover height
- Add onWheel handler to FiltersContent to enable mouse wheel scrolling
  when rendered inside a Radix Dialog (which blocks wheel event propagation)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* [OPIK-4655] [FE] Clear outputs when entering experiment mode

Previous prompt outputs persisted when loading a dataset. Now
resetOutputMap() is called in handleRunOnDataset so outputs are
cleared before entering experiment mode, matching the behavior
when switching between datasets.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* [OPIK-4655] [FE] Improve hotkey badge visibility in dark mode

Increase hotkey display background opacity from 20% to 50% in dark
mode so the keyboard shortcut icons are visible on the disabled
Run button.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* [OPIK-4655] [FE] Preserve library prompt link when duplicating a variant

Duplicated prompts now keep the loadedChatPromptId so the library
name appears in the header. A skipInitialLoad flag prevents the
hook from overwriting the duplicated messages with the saved
library version. The flag is cleared on first mount to avoid
persisting to localStorage.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* [OPIK-4655] [FE] Fix double scroll and boundary behavior in filters list

Address baz-reviewer feedback: add preventDefault() to avoid double
scroll from both manual scrollTop and native behavior, and only
intercept wheel events when scroll remains in the wheel direction
so parent can scroll at boundaries.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* [OPIK-4655] [FE] Address baz-reviewer feedback on skip initial load

- Add proper deps to cleanup useEffect instead of empty array
- Add comment explaining why loadedChatPromptRef is set when skipping

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* [OPIK-4655] [FE] Fix linting issues

* [OPIK-4655] [FE] Address PR review: stop flag, Shift+Enter guard, scrollbar class

- Reset isToStopRef on runAll and runSingle to prevent stale no-op
  after stop→run sequence
- Add isRunning check to Shift+Enter shortcut so it doesn't fire
  while an experiment is already running
- Replace inline scrollbarWidth style with comet-no-scrollbar class
  in StickyScrollTable header

---------

Co-authored-by: yariv-h <yarivh@comet.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@petrotiurin @claude @anastasiapyzhik
[OPIK-5289] [SDK] fix: return ChatPrompt when field type is declared …
e7830e0 
…as ChatPrompt in agent config (#5916)

* [OPIK-5289] [SDK] fix: return ChatPrompt when field type is declared as ChatPrompt in agent config

When resolving prompt fields from a blueprint, both Python and TypeScript SDKs
now honour the declared field type (Prompt vs ChatPrompt) instead of relying
solely on backend template_structure metadata. This ensures a clean roundtrip:
a field annotated as ChatPrompt always comes back as ChatPrompt, and a field
annotated as Prompt always comes back as Prompt.

Implements OPIK-5289: fix return ChatPrompt in agent config

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix the actual issue

* test: fix mock setups in blueprint test for chat prompt resolution

Fixes test mocks that were failing due to incorrect attribute placement.
template_structure should be on prompt_detail, not version_detail.
Also added version_detail.type for proper enum conversion in
from_fern_prompt_version().

- test_chat_prompt_field__resolves_to_chat_prompt_object
- test_base_prompt_annotation__chat_structure__resolves_to_chat_prompt

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

* fix: check both prompt_detail and version_detail for template_structure

Be safe and check both locations for the template_structure field to
determine if a prompt should resolve as ChatPrompt or Prompt.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>

* extend exports

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
@github-actions github-actions bot added python Pull requests that update Python code tests Including test files, or tests related like configuration. Python SDK TypeScript SDK labels Mar 27, 2026
@github-actions github-actions bot removed python Pull requests that update Python code tests Including test files, or tests related like configuration. Python SDK TypeScript SDK labels Mar 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@baz-reviewer baz-reviewer[bot] baz-reviewer[bot] approved these changes

Reviewers whose approvals may not affect merge requirements

At least 1 approving review is required to merge this pull request.

Assignees

@anastasiapyzhik anastasiapyzhik

Labels

Frontend typescript *.ts *.tsx

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@anastasiapyzhik @yaricom @miguelgrc @petrotiurin