User Test with A. a Cursor user on Windows laptop

[Juice10]

Usertest with A. + JamesB.

Test with A.

• He kept comparing us to cursor all the time. He's using Kilo inside Cursor
  • He expects our prompt screen to be on the right like Cursor
• The welcome screen did not appear in Cursor (???), or he didn't see it (he had the earliest version of the extension installed so possibly related to that)
• He used it on an existing project and turned off auto approve because he didn't want it to potentially mess up the code
  • Context: He is using a new python framework that loads the cached version of the app whenever the compile step fails. He wanted to verify the code looks like it would work before saving the file so he doesn't have to go digging through the logs to fix the issue.
• He cared a lot about the diff of what changed in each step. Didn't know there was a diff button on each step, also with autocomplete on, having many diff buttons is not great.
• He was observing Memory usage while using it, his comp got slow, 70%-84% Memory usage. He's using Windows WSL. At its peak VSCode was using 4GB our of 14GB available.
• Noticed he turned on auto approve after we told him about this thing about enhancing the prompt, Wanted a more visual diff of what changed when auto approve is on, wasn't awear there's such an option, all he saw was a verbose summary.
• his errors appear in a huge log file, which meant he had to copy and paste a bunch of info from/to Kilo Code. And asking Kilo Code to read the file directly it crashed because the file was to big for the context window.
• In the readme he was confused about the 'local setup' option, he thought he has to do that AFTER installing from vs code, not INSTEAD of it.
• He was using a pretty new framework, and the AI wasn't using any of the framework docs so it was guessing as to what method names would be.
• Doesn't read the summary at the end of a task.
  • Assumption: too verbose, not useful enough?
• Needed to "Pin" our extension in cursor so that it shows up in his UI.

Test with James B. - 1st Time User

• I haven't got an account. Login doesn't make any sense.
• I'm not clear what the Kilo Code API provider is. Hide other providers, please.
• Why do I need to pay? It was a question asked when the profile page was viewed.
• Allow Kilo Code to open URI. This message in the web browser should be... Or this message is in VS Code, but in the web browser it could hint to the fact that it's going to happen. A kind of manager expectations, aka, like Zoom does it.
• Free tier messaging needs to be in the fail message, it needs to be in the API provider, and it needs to be on the profile page.
• Also, when you start typing, it should be at the very top, and then move the chat down just like cursor,
• 402 API usage limit, dot, dot, dot, buy tokens. That should be a better error message. We should help the user instead of just complaining.
• The badges on the VS Code marketplace, a Kilo Code page, are too colorful, too distracting, should all be the same color, or do away with them.
• The buttons are not connected to commands. Whenever a command is run, the prove button should be connected to it visually somehow.
• James felt nervous whenever he saw money. He said, "I'm nervous. I don't want to see the money."
• When multiple choice happened, he wasn't sure that these were buttons. Didn't look like buttons originally. Maybe they should be radio buttons or something else, but they should feel like buttons.
• Whenever we're in a new window and there's no folder context, we should prompt the user to create folder. Cursor does this by default. VS Code gets in this kind of like in between state, unfortunately.
• In cursor, we should tip the user to pin our extension so that they can find it again.
• The command mkdir -p should always be allowed in the context of the project.