An Extract from the Report on the User Trial for the CSFII
An Extract from the Report on the User trial for the CSFII.
4.Conclusions
PARTICIPANTS PROFILE
Out of the nine participants, five were from the previous testing group, and four were new. Two from the new group were from the Department, not practising teachers.
Five were from primary schools, two from secondary and two from education departments.
Their computer literacy was mostly from the category defined as “intermediate” and there were none who described themselves as “novice”.
Eight of the nine had access to their own school or work computer, and all were connected to the Internet. The use of Internet was moderate to high on average; that is, weekly to once a day.
Browsers installed on the machines were mostly Version 4 and above.
Everyone used the computer to prepare course (or other) material.
Significantly, most had not or did not use “Switched on Curriculum CD-ROM”, whereas there was a high use of CSF ’95. This is significant because the response is from computer literate person who, it may be surmised, would be more attracted to computer based material than print. It is a comment therefore on the relevance and usefulness of the previous CD-ROM version..
It was deliberate to select at least half the participants from the previous testing group. It would have been preferable not to have so many from the primary sector; however, it was difficult to find teachers in secondary schools prepared to attend a trial session in the last week of the school year.
The participants were in the main computer literate. However, this may have been beneficial to the trial as people who are computer knowledgeable are very critical when appraising computer based products, because they have something to compare the product to and they know the potential and options of electronic products.
The test results of the teachers (and departments) have been analysed carefully, and their comments are summarised below.
OPEN INVESTIGATION
a)Navigation
It was very apparent that, left to their own devices, the participants explored the product widely and easily, without the need for prompting or supervision. They moved fairly effortlessly from the Splash page, through the Home pages and into the individual pages.
It was also significant that most people did not experiment with the Search options, but chose to use the various browse options to find their way around the product– using the Home page, then the menu bar, then the links on each page.
The broad selection of navigation options and the ease in which they were able to navigate through the product was indicated in:
- the responses to questions relating to what buttons they chose first (only two bothered to select with “How to Use CD-ROM” - most went straight off to look at a KLA that interested them)
- everyone saying that they knew what to do next from the Home page
- their comments relating to what they liked and disliked about the site (“easy to
navigate”)
- the suggestion that the navigation options described at the top of each page was unnecessary.
- everyone being able to use the Home page navigation to find a KLA at a particular level.
Participants liked the new navigation feature added since the first trial – that of the graphical representation of the CSF structure – and they used it as another way to navigate.
There was very little pattern in the way participants chose to move through the product. There selection was content driven: people obviously went first to the areas (KLAs) that were of direct relevance to their teaching, and this was borne out by their skipping the “How to Use the CD-ROM” and on straight to the substance of the KLA information.
b) Look and feel.
Generally, participants liked the look and feel of the product.
One or two people thought the colours were still a bit dull, but people believed that
it was easy to read and ”clearly differentiated”, and looked friendly and non-
threatening.
Several participants felt there was wasted space on the right hand side of the
Screen.
When asked what aspects they disliked about the Home page, 6 respondents said
“nothing” , which is a fairly clear indication of how people liked the look and feel.
Several people were disappointed in the rapid movement of the Splash page, and that it automatically loaded into the Home page..
SET TASKS
Browse
Most people found the set tasks very easy or easy. Only one person used the HELP button for one task.
This clearly confirms the comments made above that users in this trial found the browse navigation simple and easy to use.
There were three tasks that were clearly more difficult for the users, and some could not find the information.
These were:
- Find the National Literacy benchmarks in the “Introduction to ENGLISH”?
- Find all the Learning Outcomes for ENGLISH.
- Find all the Learning Outcomes for SCIENCE.
The difficulty presented by the first task was finding the “Introduction to English” which was below the line of the first screen, and participants forgot to scroll down.
With the second and third tasks, finding all learning outcomes for a complete KLA requires some logical thinking about its placement in the product layout, and is not as directly apparent as it might be.
It was interesting to notice , by the comments made, that participants at the beginning of the browse tasks first returned to Home to do their browse search, but gradually began to use the top menu bar and page menu to do their searches.
There was also a slight confusion about the separation of Learning Outcomes from Indicators - that is the ability to get just a list of Learning Outcome titles.
List Search
This function had more problems because there was a technical fault in the system for three participants, and so they could not find anything using this facility. The search page would load, but it was not active. On one or two machines, the Stands pull-down box would only show “All Strands” despite the selection of a particular level.
However, those participants who did not have technical problems found the tasks very easy or easy.
One person commented that it was unlikely they would use the Search for these kinds of tasks , as they would use the browse functions instead. This is a very pertinent remark on the ease of use of the navigation structure.
Key word search
There were again difficulties for three participants using the key word function (as for List Search). However, those who were able to use it, found both tasks very easy or easy to accomplish.
Print.
All participants found printing out a selection of information tvery easy to do, and this was demonstrated in the trial.
Copy and Paste.
Everyone found the task to copy and paste selected information into a Word document very easy or easy to do (although one person had to use HELP in order to follow the procedure.) Someone commented that it took longer to open Word than it did to find the information.
There were several comments made about the need to re-format the text when it was pasted in the word document.
General Comments.
There is no doubt that on the basis of this trial, the product has an excellent navigation design. This was also confirmed by the two new teachers who assessed the product, both of whom found the product just as easy to navigate and find material as those who had been involved in the first trial.
It is also worth noting that a number of remarks were made by participants from the first trial who were very impressed that their suggestions and comments made in the first trial were taken on board and integrated into the design.
Other than some negative remarks about the colours, all the comments were very positive.