feat(tarko): support TTFT and TTLT metric#1232
Merged
Merged
Conversation
✅ Deploy Preview for agent-tars-docs ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
ulivz
force-pushed
the
feat/add-ttft-display
branch
from
bf53643 to
8446dc2
Compare
ulivz
changed the title
ulivz
changed the title
ulivz
force-pushed
the
feat/add-ttft-display
branch
from
243d0d1 to
f092ecb
Compare
ulivz
changed the title
ulivz
force-pushed
the
feat/add-ttft-display
branch
from
3e23615 to
6180f12
Compare
Add timing calculation in agent kernel to populate elapsedMs field: - Track request start time in LLMProcessor - Calculate elapsed time from request to response completion - Set elapsedMs in all assistant_message events - Handle edge cases (abort, error, max iterations) with 0ms
…play Improve timing calculation to properly track TTFT vs total response time: - Track first token time when receiving first content chunk - Calculate TTFT (Time to First Token) separately from total response time - Add totalElapsedMs field to event types and UI components - Enhance TTFTDisplay to show both TTFT and total time when different - Add detailed timing logs for debugging - Update tooltip to show timing breakdown
…ards Update timing field names to use industry-standard terminology: - elapsedMs → ttftMs (Time to First Token) - totalElapsedMs → totalResponseTimeMs (clearer naming) - Add backward compatibility support for elapsedMs - Update all components to use new field names - Improve tooltips and documentation with standard terms - Maintain API compatibility during transition period
Refactor TTFT display to be part of message footer for better UX: - Extract MessageFooter component from MessageGroup - Move TTFT display from message content to footer area - Use subtle colors and smaller size for footer integration - Maintain left alignment with timestamp and copy functionality - Remove standalone TTFT display from individual messages - Improve visual consistency with existing footer elements - Note: secretlint warnings are false positives for React key props
Simplify TTFT/TTLT metric display by removing separate components: - Remove LLMMetricDisplay with colors and animations - Remove MessageTimestamp with mixed responsibilities - Merge all footer functionality into MessageFooter - Use consistent gray styling matching timestamp/copy button
- Use ms precision for all time displays instead of s - Add distinct icons (FiZap for TTFT, FiActivity for TTLT) - Ensure consistent gray styling for both metrics - Add clear hover tooltips explaining TTFT and TTLT
- Add cursor-help for better UX indication - Improve tooltip text with detailed descriptions - Add position relative to ensure tooltip display
- Replace native title attributes with MUI Tooltip component - Add arrow prop for better visual indication - Ensure proper tooltip display across different themes
- Remove cursor-help class to avoid question mark cursor - Remove arrow prop from MUI Tooltip for cleaner appearance
- Increase font size to 13px for better readability - Use dark background with white text for higher contrast - Add padding and shadow for better visual appearance - Remove arrow prop for cleaner look
- Create tooltipProps constant for shared styling - Remove duplicate componentsProps definitions - Maintain consistent appearance across TTFT/TTLT tooltips
- Add back arrow prop for better visual indication - Use pure black background for higher contrast - Style arrow to match tooltip background - Increase shadow opacity for better visibility
- Import and use useDarkMode hook - Light background with dark text in dark mode - Dark background with light text in light mode - Adjust shadow opacity for better visibility
This reverts commit 433501c.
- Add AgentMetricOptions interface with enable flag (default: false) - Integrate metric option into AgentOptions via AgentMiscOptions - Pass enableMetrics flag through AgentRunner to LLMProcessor - Conditionally collect TTFT/TTLT metrics based on metric.enable setting - When disabled, timing metrics are not included in event streams
…utations - Only capture start time when metrics are enabled - Skip TTFT tracking when metrics are disabled - Avoid Date.now() calls and timing calculations when not needed - Reduce logging overhead when metrics collection is disabled - Ensure zero performance impact when metric.enable = false
ulivz
force-pushed
the
feat/add-ttft-display
branch
from
370b553 to
aa9964b
Compare
ulivz
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds TTFT (Time to First Token) and TTLT (Time to Last Token) support in Agent Kernel
Refs:
Also, add display for assistant messages in the Web UI. Each assistant message now shows response time with color-coded badges indicating performance levels.
Historical Versions
Version 2
Version 1
Checklist