Group 3 – Agora Financial for Families – Round 1
Group 3Round 1: Data Analysis and Revisions to Prototype
Background
Agora Financial is in the business of providing its clients with relevant information concerning the economy and the financial problems facing our country. Agora is interested in promoting education and awareness of the consequences associated with the federal budget deficit, personal savings deficit, national trade deficit, and lack of economic leadership. They are looking for an instructional solution system that will help current and future generations make smarter financial decisions. It is Agora’s belief that by developing an instructional solution system that will educate Americans about personal finance and economics, the next generation of Americans will be better equipped to make meaningful and positive decision both within their households and for the economic health of the country.
Determination of Project Direction
Agora Financial is very concerned by Americans’ lack of education in personal finance and economics. In the family unit, information and learning tends to be passed from the parent to the child. While outside influence does play a part, this simple dyad provides the most immediate path for learning. Based on our needs analysis, we decided to look at the problem from the viewpoint of the family.
Reiteration of Project Goal
As stated in the research project management plan, there are three major considerations which must be addressed in determining the outcome of the final product.
- Promoting education of tweens/teens in areas of finance and economics by providing an online solution to help parents educate their tweens/teens and themselves about the full range of topics on finance and economics.
- Promote the Agora brand as major force on financial and economic matters.
- Develop future subscribers for Agora: parents interested in becoming more financially savvy, and then their children.
Continuing Purpose and Progress of Plan
As stated in the User Research Project Management plan, the ongoing movement of our work is to improve and refine development of our preliminary prototype and move towards an ultimate product design. The plan provides for conducting a pre-round survey, a Round 1 usability test, and a Round 2 using a combination of techniques. This paper looks at what we have completed so far and provides information concerning the outcomes.
Latest Steps
At this date, we have completed the beginning elements of the plan, and are in the process of beginning the next steps. This paper will address the following:
- The purpose and scope of the Pre-round Survey, as well as its collated findings
- The design and implementation of a Usability Test, findings, and actions taken
- Our next steps
Pre-Round Survey
To determine the topic area of highest interest to our target audience, we invited eleven parents and twelve teens in our evaluation team to complete an online survey. We used Survey Monkey and email to conduct the survey. There was a different survey for parents and tweens/teens. The surveys focused on their interest in different financial topics.
Tween/Teens top topics of interest were responsibly using cash (70%), sources of income for teens (60%), and control personal information (50%). Parents top topics of interest were developing a short term plan for spending, budgeting, and savings (80%), developing a long term plan for spending, budgeting, and savings (70%), and apply consumer skills to purchase decisions (e.g., comparison shopping) (70%).
Parents and Tween/Teens were also asked what features they would be interested in. Parents wanted bite-sized information targeted towards their teens. About 50% wanted tips on how to talk to their kids about finances. They were also interested in interactive games and activities, a personal page, and a glossary.
Of those surveyed, 80% of the parents and 70% of the tween/teens were interested in Money Management. Therefore, it was determined that we would focus on the topic of Money Management for the usability testing. The questions on the surveys and a summary of the results are provided in Appendix A, B and C.
Methodology For Round One
Round 1 – Usability Testing
Following the process laid out in the Project Management Plan, Group Three implemented usability testing of the prototype in its current state. A usability test script was developed to ensure a uniform presentation across a range of test subjects. We selected usability testing as an effective approach for addressing these goals as our questions deal with how the users interact and move throughout the site, which is the focus of usability testing. Usability tests are a very effective approach to ascertain whether users understand the design as it is intended and will help us to validate our key goals of whether our site is memorable, desirable, easy to navigate, and contains the right kind of features and resources.
The Usability Test Guide
Introduction / Our usability testing methodology encompassed observing and videotaping eleven parents (subjects) using the site. To the extent possible, we observed them using the site in their own environment such as at home or at work. Our usability testing for this phase of the project was focused on parents’ use of the site as our needs analysis indicated that parents are more likely to utilize the site than their children.Usability Script Format and Contents / Most of the usability testing was conducted at the subject’s home; however, there were exceptions when it was easier to conduct the usability test at another site. When this occurred, the group members (monitors) conducting the test changed the wording of the script to fit. In the introduction, the subject was given information concerning the reason for the test, the duration of the test, and assurances of privacy.
Statement of Informed Consent / Each subject was read the Statement of Informed Consent and asked if it was understood. (In the case of the long-distance test, the statement was given over the phone prior to the test.)
Preliminary Interview / As a benchmark, each subject was asked a series of questions to establish their familiarity with, and use of, the internet. They were also asked questions that served to establish their current interaction with their children concerning finances.
Evaluation Instructions / These instructions were designed to help the subject better supply the monitors with information. The intent was to ensure the subject understood that any impression, positive or negative, should be given, and that the purpose of the test was to determine the goodness of the site, not the abilities of the user.
Prototype Evaluation / Initial Impressions / This section of the test was designed to determine the user’s reaction to the website. A series of questions were asked concerning the front page, what the subject saw, and what he or she was considering as their next action. The subjects were asked what their anticipations were upon viewing the site, and what they thought might be available to them based on this first impression.
First and Second Tasks / Once the subject had answered all of the questions, they were given tasks to accomplish on the site. Directions were minimal in order to see how well they were able to navigate the site on their own. The first task was to try to find an area on the website that related to an interest they expressed during the Initial Impressions.
The second task was more directed. The subject was given a scenario to work from and was asked to maneuver through the site to locate information concerning the topic.
The purpose of having the two tasks was to determine if there was a difference in how the subject carried out general exploration vs. specific requirements, and if one was more difficult than the other.
Interface Tour / The Interface Tour was a more formal attempt to discern the subject’s ability to work with the website. By asking the subject to move to various areas and asking them to give their impressions, the monitors planned to compare this area against the information already gathered during ‘first Impressions’ and the two tasks to determine either collaboration or differences.
Memorability Exercise / As the title states, this was an exercise to test the impression the site had on the user. The first of the two tasks (moving from current location to the home page) was designed to see if the indicators on the site were easy to use. The second task (drawing a simple sketch of the Home Page from memory) was designed to see how great an impression the Home Page elements had made in their memory of the site.
Third Task / The third task, attempting to use the “My Profile” section to create a repository of articles, was given to determine the user’s ability to understand and make use of a save function to move articles into a password-protected space for later use.
Wrap Up / At the end of the test, two sets of questions were asked to the subject. The first set dealt with the site, and their feelings about it after having had the chance to explore it.
The second set attempted to determine if they felt the site worth sharing with others, and how they felt about the site compared with other sites they may have used. The subject was also asked for any additional suggestions on how they would like to see improvements of the site.
Because one of our test subject families lived outside the immediate testing area, we created a modified version of the script that could be read and answered without the reviewer being present. This script, along with the URL for the prototype was sent via email to the test. There was no opportunity to use videotaping for this trial.
Script Revision
One of the outcomes of early testing was a revision of the script. This was carefully done so the script contents would not change, causing a discrepancy between early results and those which were obtained after revision.
The revision eliminated some of the repetition contained in the first version of the script. As our testers administered the first version, it became apparent that the users were beginning to simply repeat earlier answers as variants of the questions were asked. By reducing the number of repetitions and streamlining the procedure, respondents’ participation became more relaxed and the flow of the testing became smoother.
Administrative
After the usability tests were administered, each result was carefully reviewed and the results of the tests were collated. The next section presents the results, and the conclusions and actions taken because of them.
Round 1 Findings
During our first round of usability testing, we approached 6 parents with at least one child between ages 12 and 16. Most participants were local and conducted in person. However, one set of parents lived at a long enough distance to require e-mail only usability testing. Each group member took the opportunity to conduct at least one usability test. All tests were video and/or audio recorded, distributed, and analyzed by our group.
General Findings
Lack of a Current Solution for Family Financial Education
Most users noted they had no current web source of financial education for their children. Those who did have a source for their children’s financial education generally relied on K-12 economics programs.
Content Choice
Most users validated our choice of focus by stating their primary focus for their children’s financial education was money management.
Location
Most users who spoke with their children about financial topics did so at home or in informal settings. A few users mentioned an interest in K-12 economics programs, or stated that they had a teen or tween currently enrolled in these programs.
Visual Presentation
Reaction to the visual aspects of the site were largely positive. Users stated they appreciated the simplicity of the layout, the bright colors, and professional look of the main page. However, opinions on certain visual aspects of the site.
White Space
Users displayed a mixed reaction to the white space of the site. Several users disliked the white space surrounding the main content of the site, or found it distracting. Others suggested adding a dark color wash to the background to help the main page stand out.
Graphics/Photos
Users displayed a mixed reaction to the graphics on the main page of the site. Female were more likely to notice the piggy bank (or find it “cute”), while men were more likely to the money graphic. A significant section of both groups found the graphics to be either (a) distracting to the content or (b) inappropriate to the content’s audience.
Underlined News links
Several users complained about the dotted underlined found beneath each News section link.
Secondary Pages
Users who commented on secondary pages of the site mentioned that it lacked the visual appeal of the first page. Although appreciative of its scope, users felt overwhelmed by the long listings of articles on these pages. Most users also ignored the secondary content (such as fun stuff) available on this page.
Navigation and Site Structure
Navigation encountered multiple blocks to usability. Users were unsure of how to find specific articles, yet they ‘wanted it now’. Users were annoyed by the multiple clicks required to get to article content.
Too Many Steps
Usability testing illustrated that users did not share the same ‘mental model’ of how to reach specific content. Several ignored the drop down menu, while others simply could not distinguish various topic areas. Users were unsure of how to find specific articles, or were unsure of which area of the site would contain that article. When we asked users to find specific topics, users ‘stumbled’ through items explicitly listed on the home page – especially from the Hot Topics section.
Contents of Topic Areas Unclear
Users were unable to distinguish the difference in content between the Teach Yourself or "Teach Your Kids". Some users also had difficulty distinguishing sub-topics. Instead, users would often click through several sections in an effort to locate this content. When the appropriate section was located, users struggled to locate the article they wanted in the lengthy list available. On secondary pages, most ignored the sub navigation containing related articles or activities, and struggled to locate these features.
Ignored Breadcrumbs
Users did not notice the breadcrumbs at all. Despite a logo return-link, they used the “back” button to locate the home page. While they mentioned the need for a Home button, they ignored the breadcrumbs!
Disliked Scrolling/long topic lists
Users had difficulty locating the section containing the articles they sought. When they located an appropriate section, they were annoyed by the scrolling required to reach their article of interest. Often, users did not scroll the length of the page before giving up.
Fun Stuff
Users could not locate Fun Stuff or did not recognize that Fun Stuff referred to games and links. Some users felt Links would have its own page, while others felt Fun Stuff would be located in the Teach Your Kids section. Few recognized that games or activities would be found in Fun Stuff.
Audience Conflicts
Some users wanted the site to be addressed only to parents. Some users wanted the site to be full of articles that would teach their kid without their help (i.e., videos, tutorials). Several users were confused about the difference between Teach Yourself and Teach Your Kids.
Features & Content
Like other sections of the site, our users also surprised us when it came to features and content.
Income and Careers
This section did not connect with users. Users found careers may be an premature topic for tweens and suggested replacing this section with college considerations (like scholarships). Adults had difficulty discerning what the content of this section would be for adults.
Hot Topics
This front-and-center listing of articles distracted from the larger content of the site. Often, users would click on Hot Topics while completely ignoring the drop-down menu or other content of the site. Unfortunately, users interpreted this section of the site as a complete list of articles.
Tips
Many parents mentioned they were excited about quick tips.
Poll
The poll on each section was ignored by most users. Even those users who mentioned the poll in a positive light did not immediately see its value.
My Profile / Lesson Plan
These features were largely ignored. Users only valued these features after they were explained by the group. In the meantime, some users were confused by the Add to Lesson Plan buttons and the My Profile link.