-
Notifications
You must be signed in to change notification settings - Fork 3
phaseIII
In this phase, our team focused on conducting a Pilot Test to see how real users would interact with the Unstuck app. The main goal of Phase III was to test the app with actual users and gather feedback on their experience. We wanted to see what was working well and what still needed improvement. To do this, we had participants complete tasks in the app such as submitting repair requests and asking questions. Afterwards, we asked them questions about their experience to get a better understanding of any issues they ran into. Using this feedback, we were able to gain useful insights into how users interact with Unstuck and what changes could help improve the overall experience going forward.
The Unstuck Usability Team conducted a Pilot Test with users on May 5th, 2026. Before Pilot Testing, all of the Usability Team Members worked together to generate a Prototype of the Unstuck app, filled out an IRB form, create a Informed Consent Form, and generate a Protocol Script for this study. The Protocol outlines our structure of the study, including what tests each participants are asked to complete, and what data should we collect/measure during the study. During our planning for the Pilot Test, the UX Team was determined that our main focus for our study was to compare the efficiency of our Unstuck app to an competitor's app.
The participants of our study was other UX teams from the Usability Engineering class. The Unstuck UX team posted some of the requirements for the user to the slack channel, and there are 6 other students that signed up to participate in the upcoming Pilot Test. It was determined that the only requirement for this study was that the user has to be over the age of 18.
For the Unstuck Pilot Test, each participant was asked to complete certain tasks. Each task was to be completed for our Unstuck Prototype and our competitor app, Upwork. The app that the participants started with was varied, in order to help any bias that the participants had for our app.
Before the Pilot Test began, users were supposed to read the Informed Consent Form and asked to sign the form to ensure that the users understood the study and the risks, and were willing to be able to participate. Next, the test would have been followed by the Protocol outline for our test. First, our introduction was read and questions were asked to gather some background information of the user. These questions were aimed to gather some information about the experience that the users might of had with similar apps, and to what scale this app would help other users manage their car. After their responses were recorded, the script initiated the tests. For the tasks that the users were asked to complete, the primary goal of the task was to test the efficiency of the Unstuck app against a competitor.
With our app displayed on a laptop, the moderator had to read the instructions for Task 1-2, then ask the participant to complete the task whenever they are ready. Once the participant began to interact with the app, the moderator started a timer. During the interaction, the moderator took notes on the actions that the participant was doing during completing the task, the moderator stops and records the specific notes, and makes a note on what the participant was thinking during the task that the participant was doing. Once the moderator recorded the result, then they will be able to go to the next task and repeat the same process until the moderator was able to complete each task.
For Task 3-4, it was identical to the original tasks. For Task 3-4, the users were asked to open Upwork and read the instructions for that task. Once the user begins to interact with Upwork, the moderator would start a timer and begin to take notes and steps on how the user would complete these tasks. When the user believes that they have completed the task, the moderator would stop the timer and record their notes on a data spreadsheet and take which portions were good and bad about our app. Once the moderator has recorded the results, they would open back our app and repeat the same process as they did for Task 3.
Once all tasks were complete and all information from each task has been recorded, the moderator begins to debrief the participant. The debrief aimed to gather feedback from the participant about their experience in using the app. The moderator asks the participant questions about each feature that they liked or dislike on the app, as well as which feature they preferred on the app. The moderator records each participant's responses, then thanks to the participants, the Pilot Test concludes.
After the Pilot Test is finished, the moderators record the data that they collected, without any identifying information, into a data spreadsheet for reference and analysis later on.
Participants shared the likely-hood of a simple and clean designs over a saturated GUI, with other competitors the user found Unstuck to be easier to use. The pilot showed that most participants were able to complete the tasks without external assistance, proving the app Unstuck to be easy to understand.
However, participants ran into difficulties when finding certain navigation options or understanding the objective behind the buttons and levels. Tasks that required the transition through multiple screens, showed the level of complication that a new user can encounter.
Our study highlights the fact that we are missing key navigational info, and that there is much room for improvement in user education and familiarity.
Unstuck demonstrated a strong foundation, offering a straightforward way for users to request assistance with car-related issues. The simplicity of the interface and ease of navigation were consistently highlighted as strengths. That said, there is room for improvement. While users were generally satisfied with the design, the number of steps required to complete certain tasks could be reduced or streamlined. Adding tutorials or onboarding instructions could help new users get comfortable with the app more quickly.
The most significant Caveat to the sprint was our participation selection. The Unstuck team was limited with the participants, as well as the participants that were selected for doing our Pilot Test. The UX team was also inexperienced in moderating the Pilot Tests, which was contributed to how our study went.
Another caveat was that not all of our teams workflow was finished at the time of the moderation, so some users had trouble navigating the app based on that alone.