Wenqin Chen

Bobby Davis

March 2015

Each team member participated in all aspects of the project. We both sourced users for the usability tests, traded off roles as the moderator and the note taker, and jointly prepared the report write-up.

Test Description:

Our team chose to test the usability of the Jawbone UP24 fitness activity tracker device and the corresponding smartphone app. Our objective was to better understand how people interacted with both the hardware and software elements of the product. A brief description of our three test users is as follows:

  • User 1: male, experienced jawbone user, 22-28 years old, tech-savvy, health-conscious
  • User 2: male, experienced jawbone user, 22-28 years old, tech-savvy and interested in data, health-conscious
  • User 3: female, novice jawbone user, 22-28 years old, tech-savvy, health-conscious

Test Guide:

Hi, thanks for coming. We’re performing a usability test for our UX class. For this test, we’ll be asking you to perform some tasks related to the use of the Jawbone UP24 fitness tracker and app. In case you aren’t familiar, the Jawbone UP24 fitness tracker allows users to track the number of steps they’ve taken throughout the day, which the user can view on the smartphone app. The app also allows users to view and set up other features related to their daily health, including calorie intake based on food consumption, calories burned, sleep tracking, and idle alerts.

During this test, we’d like to ask your permission to record photos, audio, and video. These materials will only be used for our assignment and will only be seen by the two of us and our instructors. If you agree, please review the consent form and sign once you’re ready.

We’ll be taking video using what we call the “reverse hug” method – the laptop will be placed in front of you and configured to take video of the smartphone and your hands. For each of the tasks we will ask you to perform, please talk us through your thought process as you perform the tasks. After each task, we’ll be asking you to rank each one in difficulty on a scale of one to five.

Let’s dive into the tasks:

Task 1 (for user 3)

  • Find and download the Jawbone UP app onto your smartphone.
  • Pair the Jawbone UP to your smartphone through the app.

Task 2 (for all users)

  • Put on the Jawbone, adjust it so that it sits most comfortably on your wrist, then take it off.

Task 3:(for all users)

  • Using the Jawbone UP app, check your daily activity progress.

Task 4: (for all users)

  • Using the Jawbone UP app, set the fitness tracker’s idle alert at 15 minutes.

Task 5:(for all users)

  • Using the Jawbone UP app, set a vibration alarm for a morning wake up call.

Task 6: (for user 3)

  • Using the Jawbone UP app, log what you ate for breakfast today.

Overall, how easy or hard was your experience performing these tasks? Did you have any additional general thoughts you’d like to share?

Thanks again for taking the time to help us with this test.

Summary of Results

  • Areas/features that our test users liked:
  • Generally, each of our three test users seemed to like different aspects of the product.
  • User 3 was delighted that she could set/track her happiness level using the app.

“Happy, (sliding up the emotion slide bar) suuuper happy. Yay I like that.”

  • User 2 was delighted when he found the idle alert feature and indicated that he felt it was something he would start using.
  • User 1 purchased the product for himself a few months ago in order to track his daily walking habits.
  • Areas/features that our test users had trouble with:
  • Device sizing/fit for females
  • Problem: for user 3, she felt that the size “small” device was too small to fit comfortably on her wrist. This is despite the fact that she considers herself to have small wrists. Quote: “It is not comfortable. In fact it is quite small. I’m kind of have a small wrist. I’m amazed at how anyone can wear this.”
  • Recommendation: Consider producing another device size that sits between small and medium in order to better accommodate the wrist size of adult females.
  • Hardware/software sync
  • Problem 1: the app crashed when user 1 attempted to physically pair the device with the smartphone via the audio jack physical connection rather than through Bluetooth.This user ultimately was unable to pair the device to the app.
  • Problem 2: user 2 stated that he uses the app infrequently because he perceived the Bluetooth connection to be unreliable. This perception was built through his experience with a previous model of the device. During this test, the user was able to successfully connect his device to the app using Bluetooth, so our user acknowledged that his perceptions of Bluetooth problems likely were based on negative experiences with the previous device model. Quote: “This stupid alarm would come on at 7:30 when I didn’t need it, because it didn’t sync apparently.”
  • Unintuitive UI
  • Problem 1: When opening the app for the first time in order to pair the device, the first screen seen by user 3 displays pictures of three device types. This screen was confusing and unintuitive to our novice user, since there were no instructions or words that could help the user decide what action to take. Quote: (Long silence staring at the app screen) “I’m just trying to figure out what each of these options matches up again. The band doesn’t look like anyone of them.”
  • Problem 2: User 3had difficulty in making changes to settings, such as to the alarm clock time. At issue was the fact that settings don’t save until the user taps on the checkmark icon in the upper right corner of the screen. The user did not see the icon and returned to the previous screen without saving the new settings. Quote: “I thought it would automatically sync but it don’t. So I shouldn’t press the x and instead the check button…. I could’ve learned it the hard way if I didn’t wake up the next morning.”
  • Problem 3: On the app home screen, some of the key features are presented only after hitting the “+” icon on the bottom center of the screen. This icon appears or disappears depending on how far down the user has scrolled on the home screen. User 3 was not able to quickly find the “+” button and thus spent a substantial amount of time searching for certain features.Additionally, the interface presented after tapping the “+” icon is not intuitive. Quote: “sometimes the + is there, (scrolling down) sometimes it is not there.” (A while later) “But + is gone again.”
  • Feature overload
  • Problem 1: User 2 commented that it was difficult to find any given feature in the app because there were so many features available across three separate menus (left menu, right menu, “+” menu). This user was pleasantly surprised to find that some features he thought would be useful were already available in the app – he simply didn’t know those features already existed.Quote: (Looking at the idle alert feature) “Huh I should use this feature!”
  • Problem 2: This same user also had difficulty finding the menu on the right side of screen because his eye was never drawn to the icon in the upper right corner that opens the menu.Quote: “I didn’t know why I just completely forgot about the right menu.”
  • Feature fatigue
  • Problem: User 1 said he now would ignore the vibration morning wakeup call. It wouldn’t wake him up anymore. Quote: “I don’t know. I just sleep through it.”
  • Hunger for more data
  • Problem: Users 2 and 3 felt it would be useful if they could gain access to more granular data and metrics by tapping on summary-level data dashboards. For example, one user commented that she wished that she could tap the calorie intake graph in order to find out what foods she consumed that day, or even to compare that day’s calories with other days in order to identify trends.
  • Recommendations
  • Investigate and remedy any issues regarding device/app pairing via the smartphone audio jack. Alternatively, consider removing the audio jack pairing option if it is deemed to be extraneous. Having a feature that crashes the app is worse than not having the feature at all.
  • Consider simplifying the app home screen menus in order to ameliorate feature overload / UI clutter issues. Determine whether more features can be bundled and/or nested under higher level headings.
  • Find more effective ways to message product features to customers. Some users in this study were unaware of the existence of certain features they found to be helpful, or were unaware of product improvements in other areas such as Bluetooth syncing.
  • Make app icons more accessible and understandable. This is particularly true of the “+” menu icon on the home screen, which at minimum should be enlarged, displayed more consistently, or both.
  • Introduce more varied vibration reminders so users do not grow oblivious to them.

Link to video clip of usability tests:

Figure 1: User 2 during testing session.

Figure 2: User 1 plugging in his wristband for syncing.

Figure 3: User 3 trying to pair the device.