Group 3 – Agora Financial for Families – Round Two

Group 3Round Two: User Research and Revisions to Prototype

Background

The structure and content of the product under development are designed to meet the requirements of Agora Financial, an organization dedicated to providing its clients with relevant information concerning the economy and the financial problems facing our country. Agora has taken up the cause of education and awareness in the fight to prepare citizens with knowledge and means of coping with the consequences associated with the federal budget deficit, personal savings deficit, national trade deficit, and lack of economic leadership. It is Agora’s belief that by developing an instructional solution system that will educate Americans about personal finance and economics, the next generation of Americans will be better equipped to make meaningful and positive decisions both within their households and for the economic health of the country.

Determination of Project Direction

The focus of Agora Financial is that of education as a tool for coping with personal finance and economics. The lack of ability to adequately cope with financial problems and the rapidly shifting economy suggests a lack of informed preparation at all levels of the family unit. Agora believes that for future generations to survive the fluctuations in the marketplace, knowledge must occur. Based on needs analysis, the design and development team came to the conclusion that for generational learning to occur, education must take place at both levels of the family unit, parent and child. This simple yet powerful dyad, with the bonds and trust that are inherent provide the most immediate path for learning to take place. Both parts of the dyad profit from the experience and two results are possible: learning and tradition of familial education, and the tradition of shared knowledge passed from generation to generation.[cnhs1]

Reiteration of Project Goal

As stated in the research project management plan, there are three major considerations which must be addressed in determining the outcome of the final product. These considerations drove the process during the first round of work and continue as the focus in round two.

1.Promoting education of tweens/teens in areas of finance and economics by providing an online solution to help parents educate their tweens/teens and themselves about the full range of topics on finance and economics.

2.Promote the Agora brand as a major force on financial and economic matters.

3.Develop future subscribers for Agora: parents interested in becoming more financially savvy, and then their children.

Continuing Purpose and Progress against Plan

As stated in the User Research Project Management plan, the ongoing movement of our work is to improve and refine development of our prototype and move towards an ultimate product design.The plan provides for conducting a Pre-round survey, Round One usability testing, and Round Twouser research using a combination of techniques.

This paper looks at the second stage of the process, focusing on research concerning the user and their reaction to, and use of, the developing product.At this date, we have completed Round Two. In addition, we have also completeda follow-up 1.5 Round of usability testing to validate our prototype revisions based on our original usability test findings. This paper includes the following sections:

  • Round 1.5 Usability Testing and Feedback–This section reports on the results of validity testing for changes made in the structure of the prototype
  • User Diaries–This section provides a users’ record of their interactions with the prototype’s content and the effects on the family as information was obtained and passed within the family unit.
  • PreTest/PostTest – This section provides acomparison of user knowledge before and after interaction with the site and its content.
  • Log Analysis – This section provides statistical results obtained for users’ use and interaction with the prototype site, providing information useful in helping revise and fine-tune the site.
  • Interviews – This section provides a summary of the information gathered from our interviews that probe more deeply into whether parents are learning and sharing information with their kids as a result of interacting with our site.
  • Findings– This section presents the findings from this round of work, adjustments made and/or proposed as a result, and the current state of the work.

Round 1.5 Usability Testing and Feedback

To validate the effect of design changes that occurred after Round 1 Usability Testing, we conducted a second round of usability testing. We asked four participants who evaluated the original version of the site to review the new version of the site. This round of testing was conducted via email with a brief seven question, open-ended survey. Questions included overall impressions of the site, whether the site was easier to navigate, whether the navigation features were more helpful, whether the site was an improvement over the old site, and any additional feedback they might like to provide. We also included two usability testing questions asking users to describe how they would get to a specific section of the site. Questions asked are listed in Appendix A.

Of the four participants who received our survey, we received three responses. All agreed the new layout was a design improvement over the original site. Words used to describe the new site included "clean", "clearer", "colorful", "inviting", "better organized", "balanced", "user friendly", "simplified", and "well presented". Participants also relayed positive feedback on the new navigation structure. They agreed the navigation was easier to understand and more intuitive. One participant stated that she "didn't need to dig or look hard to find specific information".

The two usability test questions included in the survey asked participants to describe how they would locate two different types of content. The first content type was an article about 529 College Savings Plans. All participants were able to correctly describe which buttons they would click on the site to get to information on 529 College Savings Plans. Interestingly, users did not indicate they would go to "Money Management" and then "Savings" to find information on a savings plan, even though an article on the plan was included in this section. In a future version of the site, the content will likely be cross listed between both "Money Management: Saving" and "Paying for College: Educational Savings Account".

Users were then asked to identify the buttons they would click to find resources that would help them teach their kids about budgeting. Two users were able to correctly identify that they needed to use the "Money Management" drop down and then click on "Budgeting". One of these two users identified that resources would be located under "Teach Yourself" articles, and the other identified "Teach Your Kids" articles. Our third participant indicated he would click on "Starting your Kids Investing Early is Smart". This is an article that was listed as a "Featured Article" on the home page. None of the participants mentioned the "Fun Activities" section as a place where they would find resources for teaching their kids. That said, it is possible that the header "Teach Your Kids" may have drawn participants' attention towards it, especially since the question asked, "What buttons would you click on to find a resource to teach your kids about developing a budget?"

Two suggestions were made. One user noted that Customized Content was not a part of the home page. The site was later updated to include a more prominent area about Customized Content prior to Round 2 testing. Another participant noted a bug where menu items flicker when the mouse is not over the text of the menu items. This is a known bug related specifically to Internet Explorer 6 and the navigation CSS coding used. In future versions of a site, an alternative drop down navigation code may be able to address this bug.

The feedback to our site was positive overall. As one of our participants stated:

Hope we can find something like this to be available in the future. I certainly would visit and would welcome the opportunity to utilize and see it’s [sic] full benefit. This type of information is difficult to obtain all from a central location. It’s overwhelming. Thanks for allowing me to participate.

User Diaries

If it is not possible to maintain an ‘around-the-clock’ surveillance on a testing subject, the use of a subject-maintained diary can provide useful information over a period of time. By taking the time to determine what we want to learn from the subject’s reaction to an event, and using that knowledge to create a question set for the subject to answer as they carry out the work, we can derive information on how the subject affects, and is affected by, the event.

This tool was employed during the user research phase of the development phase to obtain user perspective on their experience uUsing sSite rResources. The diary was administered over a two week period in two cycles. Users were presented with a diary page to complete at the end of each week, with reminders sent out at the middle of each cycle. See Appendix B for the content of the diaries as well as the instructions that were given in accompaniment.

Both parents and tweens/teens were participants in the diaries, which were a combination of questions designed to gauge both the parents’ use of the site and their interaction with their tweens/teens.

Week One Results

During our first week of diaries, we tapped seven parents with children of various ages. Each parent had at least one child between the ages of 12-16. Parents who reviewed the site found it to be easy to navigateion, attractive, and clear. They reported that articles were easy to find and reviewed several of the articles and/or games (i.e., fun activities).

Of the seven parents who participated in the field test, two had limited interactions with the site due to travel and other activities (work, childcare, etc.). Among the remaining five, parents reported reading about budgeting and spending diaries. Most feedback on the articles and activities were positive, however, one parent mentioned the article "Getting your kids involved in developing a budget" could be simpler, and more ‘fun’.

Parental Learning in Week One

Three parents reported that they learned nothing new during week one. One of these parents mentioned they learned about budgeting and saving, even though they learned nothing ‘new’. Two of the parents mentioned that their areas of primary interest were not yet developed. One of these parents was looking for a content that was addressed by the site (specifically, college spending plans), but may not have located it due to limited use (i.e., once). The third parent did not elaborate on what they did not learn.

Four parents did report learning something new from the site. Topics that interested these parents and were located on the site included:

  • savings,
  • 529 plans,
  • Tips to help control spending (for teens/tweens)
  • how to develop a budget (for teens/tweens)
  • spending diaries (for parents)

Two parents mentioned their use of the fun activities/games available through the site. One mentioned they enjoyed the focus on long-term goals and tips to help teens/tweens to do so (such as budgeting monthly instead of weekly).

Talking With Kids in Week One

Unsurprisingly, the two parents that did not use the site reported they did not discuss aspects of the site with their children. Of those who did use the site, four parents discussed site content with their kids 1 to 2 times. A fifth parent reported speaking to their kids 5 to 6 times.

Communicating Information to Teens/Tweens

Parents mentioned they enjoyed the games included in the site and shared them with their tweens. Parents mentioned they utilized information from the:

  • Mint.com website (mentioned on the site)
  • “Developing a Budget” article
  • "But I Want It" article
  • Budgeting and credit card articles

Two parents mentioned they discussed or used the fun activities on the site to discuss or communicate financial information with their children. One parent mentioned sharing articles on the site with their child directly, while another simply discussed the information in the article with their child.

Parents who did not use the site did not report communicating information to their teen or tween.

How they shared with their Kids

Parents mentioned several ways they communicated about the site or its content with their children. These included:

  • Having a conversation over dinner
  • Having a conversation in the car
  • Showing their teen/tween the site directly
  • Showing their teen/tween games located on the site
  • Suggesting a game of “The game of life” with their children*

*(From context, we assume “Life” would be used as a vehicle to discuss financial topics.)

Articles/Resources they liked

From the content available on the site, parents mentioned they enjoyed the articles/resources:

  • 411 on 529
  • Practical Money Skills
  • But I want it
  • Links to individual Bank sites offering financial advice to families (i.e., A.G. Edwards)
  • Give me 20 Budgeting Basics
  • Savings Quest

Two parents also mentioned they enjoyed the Savings section and information on budgeting.

Articles/Resources they disliked

Five parents mentioned there was nothing about the site they disliked.

For those who mentioned one, parents mentioned they enjoyed the articles/resources:

  • Getting Your kids involved in developing a budget (due to it being cumbersome to absorb)
  • A linked article from Family.com, which discussed the average cost of eating at home (due to the overly high estimates of feeding a family, according to the parent who cited it)
  • the Savings Quest Game (which the parent felt was fun, but a bit long)

Week One Summary

At the conclusion of week one, we found:

  • parents are busy, and a significant portion have not found time to use the site
  • parents who use the site are much more likely to benefit from its contents
  • parents who used the site more often were more likely to talk to their kids
  • parents seem more focused on teaching their teen/tween then teaching themselves
  • few parents have taken interest in customizing the content available to them
  • [cnhs2]parents really enjoy the games, links, and fun activities available on the site

Week Two Results

During the second week of diaries, we tapped 10 parents of various ages[cnhs3]. Each parent had at least one child between the ages of 12-16. Seven parents viewed the site one to two times. Three parents viewed the site three to four times.

Experience Using Site Resources

Parents reported that the site was helpful and contained topics that interested them, such as budgeting or 529 savings plans. Others reported they enjoyed features of the site, such as games or new articles. They reported that it was easy to find information on the site and was easy to navigate. Some parents reported sharing aspects of the site, such as interactive features or games, with their children. Another parent encouraged their daughter to explore the links they viewedfrom the previous week. One parent reported that they continued to be ‘very engaged’ in the site’s content and tools.

Parental Learning in Week Two

As in week one, we asked parents what they learned through the site. Two parents reported they did not learn anything new. One did not offer an answer. The other seven parents all reported their experiences on the website. Some parents learned things on their own, while others preferred to watch videos or use features alongside their child.Topics of interest varied. One parents reported reading about keeping a spending diary, while another mentioned an interesting article on purchasing a car. Articles addressing budgeting, college savings plans, and family finance were popular. One parent even mentioned their appreciation of having money rules applied to a different nation (the UK). Features such as the college savings calculator and the money personality survey were popular. Interestingly, the site seemed to impact some parents beyond simple knowledge acquisition by creating opportunities for critical thought and application. One parent reported that they ‘learned a lot of teens don't have much money sense’ while another reported they learned how their own child thought about money. In a different section, one inquisitive parent responded to an article by askingshowed interest in the resource, “Do 401k's still make sense?”[cnhs4]One parent even reported that the site inspired them to change their behavior in regard to negotiating their bills. The parent stated, “While I knew it to be beneficial to revisit / re negotiate costs of items such as phone bills, the article '5 ways to save money' helped provide the additional nudge to make the call.”