API spec review: UserActivityHistory #5260
Conversation
| for a webpage that contains the keywords "Korean" and "recipe", and construct a query with those keywords, | ||
| a content type of "text/html", and an access time within the last 24 hours. | ||
|
|
||
| In order for an app to make use of this API, it must be Windows logo certified, and the user must provide |
There was a problem hiding this comment.
link to "how do I get the Windows logo certification" page?
There was a problem hiding this comment.
Also, need to take into account whatever consent model we end up with - one time, every time, does it include context, etc?
|
|
||
| # Background | ||
|
|
||
| The [UserActivity](https://learn.microsoft.com/uwp/api/windows.applicationmodel.useractivities.useractivity) |
There was a problem hiding this comment.
We need to update the docs on UserActivity as well to drop references to Timeline (yes, still there) and instead reference AI and stuff.
| about the document that the user is editing, so that Recall can later take the user to the document | ||
| at the same location. | ||
|
|
||
| To record user activity, an app uses [UserActivityChannel](https://learn.microsoft.com/en-us/uwp/api/windows.applicationmodel.useractivities.useractivitychannel?view=winrt-26100) |
There was a problem hiding this comment.
Why is this information here, rather than in the docs for how to use UserActivity? The person using this API wants to query existing ones, not necessarily create new ones.
| # Conceptual pages (How To) | ||
|
|
||
| The intended use case of this API is to allow you to make search queries against the user's activity history | ||
| on their local computer. The string matching in the query parameters in this API is lexical in nature, |
There was a problem hiding this comment.
Very wordy. Suggest more like "This API uses simple string matching for searches; it does not support natural language. If you want to support natural language search scenarios, use <some other WCR API?>."
| your app prior to calling this API. | ||
|
|
||
| For example, if a user types in something along the lines of, "Please find the Korean recipe I was looking at | ||
| earlier today", your app might have an agentic AI parse that input and determine that the user is looking |
There was a problem hiding this comment.
Is "agentic" meaningful here?
|
|
||
| In order for an app to make use of this API, it must be Windows logo certified, and the user must provide | ||
| their consent to allow access to their activity history. If either of these is not the case, the API will | ||
| throw an exception. |
There was a problem hiding this comment.
The API will fail - the exception is specific to the language projection.
| # API Details | ||
|
|
||
| ```c# (but really MIDL3) | ||
| namespace Microsoft.Windows.ApplicationModel.UserActivities |
There was a problem hiding this comment.
Q: Should we put this somewhere else? It's not a general-purpose API that anyone can use. It's specific to AI scenarios and will be VERY locked down as to who can call it. Do we have a top-level "User context stuff useful for AI" namespace? Do we need one?
There was a problem hiding this comment.
Beat me to it.
Microsoft.Windows.AI.UserActivities seems more apt
| { | ||
| runtimeclass UserActivityHistory | ||
| { | ||
| static IVector<UserActivityHistoryItem> Search( |
There was a problem hiding this comment.
Should we have an overload for a single query?
| static IVector<UserActivityHistoryItem> Search( | ||
| UserActivityHistoryQuery[] queries, | ||
| UserActivityHistoryOrderBy orderBy, | ||
| UInt32 maxResults); |
There was a problem hiding this comment.
Is there a material benefit to passing maxResults (e.g. perf? or making the user feel better if we include this as part of the consent prompt?).
| UserActivityHistoryOrderBy orderBy, | ||
| UInt32 maxResults); | ||
|
|
||
| static IVector<String> GetAppsWithUserActivity(); |
There was a problem hiding this comment.
Should this be a list of ProductIds rather than just strings? What data do we have from unpackaged apps recording activities (like Office)?
| for a webpage that contains the keywords "Korean" and "recipe", and construct a query with those keywords, | ||
| a content type of "text/html", and an access time within the last 24 hours. | ||
|
|
||
| In order for an app to make use of this API, it must be Windows logo certified, and the user must provide |
There was a problem hiding this comment.
Also, need to take into account whatever consent model we end up with - one time, every time, does it include context, etc?
| maxResults: 1); | ||
|
|
||
| UserActivityHistoryItem item = results.FirstOrDefault(); | ||
|
|
There was a problem hiding this comment.
RECOMMEND: Remove blank line
|
|
||
| This method synchronously retrieves a list of all the app names with data in the user's activity | ||
| history database. You can use this, for example, to show the user the list of apps that are being | ||
| queried against, so the user can understand why an app that is not recording user activity is not |
There was a problem hiding this comment.
...why an app not recording isn't showing up in the list of apps that recorded...
Typo? Should be "...what an app that is not recording user activity is not showing up..." ?
|
|
||
| ## UserActivityHistoryItem.AppName property | ||
|
|
||
| This property contains the name of the app in which the activity occurred. |
There was a problem hiding this comment.
My current implementation contains the exe path
Is this an unpackaged app or packaged?
If packaged you'll want to record the package full name. Given that you can lookup its DisplayName (and Logo) localized for the current user to view.
Is this historical? Does UserActivityHistory retain information recorded by apps after they're uninstalled? If so then you can't guarantee looking up the DisplayName. If so there are options but they have caveats so I'll wait to hear if relevant before saying more.
but it would be even better if there were a way to get the display name
p = packageManager.FindPackageForUser("", pkgfullname)
string displayName = p.DisplayName
returns the package's DisplayName localized for the calling user. Are there cases where the package isn't registered for the calling user?
|
|
||
| runtimeclass UserActivityHistoryItem | ||
| { | ||
| String AppName { get; }; |
There was a problem hiding this comment.
What is AppName?
If a packaged app is this the app's AUMID (programmatic id) or DisplayName (localized string for human consumption)?
Does this API support unpackaged apps?
This adds an API spec for UserActivityHistory, which can be used to query user activity history in supported apps from the past 28 days.