As an Admin of OpenHands Cloud, I want the ability to understand how my users are using OpenHands. Specifically, I want to answer questions like:
- Who is actually using the tool?
- How much are they spending?
- What is the cost/benefit of usage so far? e.g., for the $ spend on tokens, how much engineering velocity have we gained? How much more efficient is the agent vs. human engineer?
- Which repos and branches are getting the most use?
Types of reports
High Level Report
A single page that is viewable by Admins and Owners that displays high level graphs and stats like:
- Total number of conversations across Org
- Number of active conversations (where runtime is active and associated with a conv)
- Average cost of a conversation
- Chart of total conversations per day over last 30 days
Conversation report
A list of conversations across the Org, viewable only by Admin and Owners, with the following fields:
- Tokens
- Spend
- User
- Conversation duration
- Start date
- Last update
- Associated PR
- PR merged?
- Stretch: Type of conversation: Feature, bug fix, refactoring, security fix, etc.
User usage report
A list of usage at a user-level, viewable only by Admin and Owners, that helps admins understand who is using OpenHands:
- Email of user
- Nbr of conversations
- First convo date
- Last convo date
- First login date
- Last login date
- Spend - MTD
- Spend - YTD
- Spend - Lifetime
- Current Budget
- Nbr of PRs merged with OpenHands
Model usage report
A report that helps users measure the efficiency of different LLMs
- Model name
- Nbr of conversations
- Tokens used
- Total cost
Sub-issues
As an Admin of OpenHands Cloud, I want the ability to understand how my users are using OpenHands. Specifically, I want to answer questions like:
Types of reports
High Level Report
A single page that is viewable by Admins and Owners that displays high level graphs and stats like:
Conversation report
A list of conversations across the Org, viewable only by Admin and Owners, with the following fields:
User usage report
A list of usage at a user-level, viewable only by Admin and Owners, that helps admins understand who is using OpenHands:
Model usage report
A report that helps users measure the efficiency of different LLMs
Sub-issues