Summary Report
Terminology Review and
Content Categorization (Card Sort)
for a College of Education and Human Development Website
July 10, 2008
Prepared by:
Josh Carroll
Usability Consultant
University of Minnesota
7/3/2008 8:56:00 AM College of Education and Human Development Website Page 1 of 26
Summary Report - Terminology Review and Content Categorization (Card Sort)
Table of Contents
Page
1. Usability Team Members...... 2
2. Description of Proposed Website……………………………………………… 3
3. Goals for the Terminology Review Content Categorization (Card Sort)… 4
4. Methodology……………………………………………………………………... 5
5. Evaluator Profiles……………………………………………………………….. 8
6. Content Categorization, General Findings, and Evaluator Comments.……… 11
7. Terms Used in the Terminology Review & Content Categorization ………… 15
8. Terminology Results ………………………………………………………………20
9. Understanding the Content Categorization Results in Appendices B thru E…25
Appendices
Appendix A: Problem Terms, Comments, and Alternate Names
Appendix B: Content Categorization of Lower-Level Link Names
Appendix C: Number of Times Each Lower-Level Link Name Was Placed
in a Given Category
Appendix D: Number of Lower-Level Link Names Each Evaluator Put
in a Given Category
Appendix E: Hierarchy of Links by Each Evaluator
1. Usability Team Members
Name / Unit / RoleJosh Carroll / Usability Services / Usability Consultant
Alice de la Cova / Usability Services / Usability Services Manager
Mat Carlson / Usability Services / Lab Support / Reception
Kyle Doeden / Usability Services / Lab Support / Reception
Sean Skaar / Usability Services / Lab Support / Reception
Jeffrey Abuzzahab / College of Education and Human Development / Web and Multi-media Lead
Jessica Franken / College of Education and Human Development / Content Writer
Jodie Gustafson / Work and Human Resource Education / Content Expert
Laurie McGinley / College of Education and Human Development / Web Designer
For more information on this report, please contact:
Josh Carroll
Usability Consultant
Office of Information Technology
Phone: 612-624-9742
Email:
Alice de la Cova
Usability Services Manager
Office of Information Technology
Phone: 612-624-9365
Email:
2. Description of Proposed Website
The College of Education and Human Development (CEHD) has undertaken a project to re-design their website for the purpose of providing information about the College. The site will be aimed primarily at current and prospective CEHD students, CEHD faculty and the greater education community.
As part of the design process, Usability Services completed a 12-participant study to review terminology proposed for the new site and to obtain user input into content categorization for the new site.
3. Goals for the Terminology Review and Content Categorization (Card Sort)
A terminology review and content categorization (using a card sort method) was completed in June 2008 in order to gain insight about the following goals:
· Determining intuitive organization that would be consistently used at the college level or at the department level websites.
· Finding out the common “plain English” terms that audience would expect for links. Identify those terms that would be used at both college and departmental level in the future.
· Finding if there is additional information that should be on the site.
4. Methodology
The study was requested by the College of Education and Human Development (CEHD). It was conducted with an observation team with members from the Usability Services team, CEHD and Work and Human Resources Education.
The evaluations took place over three days from June 2, 2008 through June 4, 2008 in 12 individual sessions at the Usability Services Laboratory on the University of Minnesota-Twin Cities campus.
Participants included 12 evaluators (and 6 backup participants) recruited from undergraduate CEHD students, graduate CEHD students, M.Ed CEHD students, CEHD faculty and staff and local high school guidance counselors in order to provide input to the design of the future CEHD website, using a terminology review and card sort methodology.
The methodology for the sessions was as follows:
1. Lower-level Link Names Card Deck:
§ Each evaluator was presented with an initial stack of 65 index cards, each of which had a term representing a proposed lower-level link name for the proposed College of Education and Human Development website on one side of the card, and a paragraph on the other side describing the content to which the link would lead.
§ See pages 15-19 for a list of the lower-level link names and descriptions used in the card sort.
2. Terminology Review for Lower-level Link Names:
§ This step is designed to help the team identify proposed link names that need rewording or clarification.
§ For each card, the evaluator was asked to read the link name on the front of the card out loud and to hypothesize what content it would lead to on the CEHD website. Then the evaluator was asked to look on the reverse side of the card at the description of the content the design team intended for that link name to lead to on the proposed website.
§ If the evaluator hypothesized content similar to the team’s description, the facilitator simply asked the user to move on to the next card.
§ However, if the evaluator did not know what content a link name might lead to, or if the evaluator hypothesized content that was significantly different from what the team intended, the facilitator asked the evaluator to re-name the proposed link with a term that would better describe the intended content. To do this, the evaluator was asked to cross out the original link name and write in his/her own proposed link name.
See “Terminology Results” on page 20 for the results of this step.
3. Card Sort of Lower-Level Link Names
§ Evaluators were then asked to group the set of lower-level link name cards into categories they thought belonged together on the CEHD website. Evaluators were permitted to establish their own categories (open card-sort method).
§ Evaluators were also allowed to put a card under more than one heading (by creating a duplicate card of the one they wanted in two categories). They were also permitted to add new cards for additional content they thought should be on the website. It was also permissible for evaluators to sub-group cards within each category, if they thought sub-grouping was appropriate.
4. Naming of Categories Created by the Evaluators
§ A category name is used to convey what the link names in a group have in common with each other, or why they were put together in the same category.
§ After sorting the lower-level link names into categories, the evaluators were asked to name each of the categories they created.
5. Content Categorization Results:
See “Content Categorization and General Findings” on page 11, as, as well as “Understanding the Content Categorization Results in Appendices B, C, D, and E” on pages 25-26.
The following Appendices contain detailed content categorization results for the lower-level link names.
Appendix B: Content Categorization of Lower-Level Link Names
Appendix C: Number of Times Each Lower-Level Link Name Was Placed in a
Given Category
Appendix D: Number of Lower-Level Link Names That Each Evaluator Put in a Given Category
Appendix E: Hierarchy of Links by Each Evaluator
6. Debriefing Questions:
If there was time remaining in a session, the evaluator was asked a set of debriefing questions.
1. How well you feel that the content planned for the website addresses your information needs?
2. What information or content would you expect to see on the website that wasn’t represented here today?
3. Thinking about the terms we showed you today, where do you currently go for that kind of information?
4. Which parts of the proposed website would you visit most often?
5. What are the most likely reasons you would use the proposed website?
6. When would you go to the College site and when would you go to a department site?
5. Evaluator Profiles
Evaluator 1: M.A. Student (Education Policy)
Gender: F
Evaluator’s Age Range: 45-60
Years of Web experience: 12
Frequency of Web use: Daily
Web experience self-rating: Average
Use Web for: Research, Work; Shopping; Listening to music
Evaluator 2: M.Ed Student (Elementary Education)
Gender: F
Evaluator’s Age Range: 35-44
Years of web experience: 10
Frequency of Web use: Daily
Web experience self-rating: Average
Use Web for: Searching for information; Communication
Evaluator 3: M.Ed Student (Art Education)
Gender: F
Evaluator’s Age Range: 18-24
Years of Web experience: 15
Frequency of Web use: Daily
Web experience self-rating: Average
Use Web for: Finding out information; research; music; email; shopping
Evaluator 4: Faculty Advisor in CEHD
Gender: F
Evaluator’s Age Range: 60+
Years of Web experience: 12
Frequency of Web use: Daily
Web experience self-rating: Average
Use Web for: Information gathering; Information seeking
Evaluator 5: High School Guidance Counselor
Gender: F
Age Range: 35-44
Years of Web experience: 10
Frequency of Web use: Daily
Web experience self-rating: Average
Use Web for: News; Games; Research
Evaluator 6: M.Ed Student (Social Studies Education)
Gender: M
Age Range: 25-34
Years of Web experience: 12
Frequency of Web use: Daily
Web experience self-rating: Advanced
Use Web for: Email; News; Job search
Evaluator 7: Undergraduate Senior (Technology Education)
Gender: M
Age Range: 25-34
Years of Web experience: 10
Frequency of Web use: Daily
Web experience self-rating: Advanced
Use Web for: Email, Browsing; Web design; Research
Evaluator 8: Ph.D Student (Learning Technologies) **Did not complete survey
Gender: F
Evaluator 9: M.Ed Student (Special Education)
Gender: F
Age Range: 35-44
Years of Web experience: 15
Frequency of Web use: Daily
Web experience self-rating: Average
Use Web for: Finding information; Exploring
Evaluator 10: Undergraduate Junior (Elementary Education)
Gender: F
Age Range: 18-24
Years of Web experience: 10
Frequency of Web use: Daily
Web experience self-rating: Expert
Use Web for: School; Work; Communicating; Fun
Evaluator 11: Undergraduate Senior (Elementary Education)
Gender: F
Age Range: 18-24
Years of Web experience: No response
Frequency of Web use: Daily
Web experience self-rating: Average
Use Web for: No response
Evaluator 12: Ph.D Student (Educational Psychology)
Gender: F
Age Range: 45-60
Years of Web experience: 12
Frequency of Web use: Daily
Web experience self-rating: Average
Use Web for: Research
6. Content Categorization, General Findings, and Evaluator Comments
1. In general, categories of importance to evaluators included the following:
§ Resources for Current Students
There was a group of categories proposed by evaluators which would be of direct interest to current CEHD students. These categories were oriented toward opportunities and information that would help them succeed in their coursework, get experience outside of their programs and make professional connections.
Link names were placed in this group 234 times.
§ General CEHD Information
Evaluators also proposed a group of categories that would have CEHD overview information. These categories contain information intended for anyone connected to CEHD or those external to it.
The information in this group is important for presenting CEHD events, faculty and staff, contact information and departmental information.
Link names were placed in this group 149 times.
§ CEHD Degrees and Programs
Some evaluators proposed categories specifically for information related to what degrees and programs CEHD offers. Evaluators thought that information about choosing what to study and how to plan a program belonged in these categories.
Link names were placed in this category 123 times.
§ Resources for Potential and Incoming Students
Evaluators proposed some categories that would be characterized as information for people interested in applying to CEHD. Some evaluators thought that categories could include information for potential and incoming students, while other evaluators split up the content intended for these audiences.
Link names were placed in this group 120 times.
§ Resources for Students who want to Become Teachers
Some evaluators proposed categories specifically related to information for students who want to pursue a teaching degree/license. Some evaluators saw this information as separate from the information in the “Resources for Potential and Incoming Students” categories, although there is some overlap of links in the categories.
Link names were placed in this group 51 times.
§ Getting Involved with the Community
Some evaluators proposed categories specifically for people interested in partnerships with CEHD.
Link names were placed in this category 31 times.
§ CEHD People
Some evaluators proposed categories specifically related to who is in CEHD. Some evaluators saw this category as providing contact information for faculty and staff. While other evaluators saw it as having information about who makes up CEHD.
Link names were placed in this group 27 times.
§ Resources for People Outside of CEHD
Some evaluators proposed categories specifically related to information about resources and professional development for people not connected to CEHD.
Link names were placed in this category 16 times.
§ Resources for Alumni
Some evaluators proposed categories specifically for CEHD alumni.
Link names were placed in this category 14 times. Several evaluators thought that the “Alumni” and “Giving to the College” cards belonged together in these categories.
2. During the terminology review, several evaluators commented that some links seemed to be at a higher level in the website’s architecture than other links. (In these instances, the evaluators were instructed to imagine that the links were at the same level in the website. In the card sort exercise, however, the evaluators were allowed to make sub-categories.)
Examples of potential higher level links include: “Degrees and Programs,” “People,” “Departments,” “Academics,” “Program Areas”
3. Links named after programs were frequently baffling. Evaluators frequently did not understand what the intended content will be unless they are already familiar with the organization or program.
Examples: “Neighborhoods,” “Youth Development Leadership,” “DirecTrack to Teaching,” “Learning Communities”
In general, evaluators would prefer a brief description of the purpose of the program as the link name, or a link name indicating why it would be of interest.
4. Evaluators wanted to have link names to better distinguish who the content was intended for. Evaluators often thought that links that would lead to content about a topic were actually intended for a particular group.