Usability Discussion – Doug Pearson’s Group

1)Support new users – broaden community

  • Fix demos that fail to run in Soar8

Either a demo should work in the release or it should be removed from the TCL menus

  • Help in resolving problems with downloads and installation

An installation FAQ would be helpful here

Also references to a mailing list or forum where install questions could be asked and answered

  • Better support for installing to non-default locations

Either don’t allow this or make it work

2)Access to tools AND models

  • A clearer list (table) of which versions of Soar are currently available and clear dependencies on other tools and versions (e.g. which version(s) of TCL are known to work with a given version of Soar)
  • Archived older versions of Soar, so we can run old models w/o updating them
  • Central repository of Soar models, together with links to the Soar version they run in.
  • Central repository of Soar tools, together with information about which Soar versions they work with.

3)Make more psychologically interesting

  • A really good, interactive tutorial – modeled directly on the ACT-R tutorial (which I gather is very extensive)

Perhaps have 2 paths on the existing tutorial – one common section, then an agent-building path and a psych-modeling path.

Including psychological models in the tutorial examples

  • Need somebody to be teaching psychology students. (OK, this one is hard to fix…Get Ron some students).
  • Exposing details of the architecture that are psychologically relevant. (I have to admit I’m not sure what this means. I think it was having more “knobs” available to tweak like ACT-R does).

4)Tool and model documentation

  • A better index in the manual (apparently it’s hard to look up common concepts and find the relevant section)
  • Including a BNF for the syntax grammar of rules
  • Explaining the debugging process

How do experts actually debug typical issues (e.g. rule doesn’t fire when expected, additional rules that fire when not expected, tracing info to use, what to do when a rule is very expensive to match, guidelines for making a problem space chunk correctly etc.)

  • Explaining the full command set and how the commands are useful (i.e. more than just a man-page like description saying what it is, also saying how it is used in practice).
  • References in the manual/help to other documents/tools that are available (e.g. references to the Soar FAQ, to the Soar debugger, to Visual Soar etc.).
  • Tutorials and manuals for tools
  • A mid-level tutorial, taking the user from beginner to intermediate level.

5)Debugging

  • Breakpoints (as we think these are supported in the Soar debugger, clearly this is an example of how there is a need for references to other tools. E.g. searching the help for “breakpoint” ought to lead to a reference to the Soar debugger).
  • Tracing or snapshot facility so it’s possible to step back in time (even though there are clearly problems making the simulation go backwards too, this would still be useful in many cases, even if it’s just so you could examine what happened, not necessarily make changes).
  • Clickable view of working memory as a graph (i.e. making the current TCL model work more like Visual Studio’s memory walker).
  • Better access to the TSI’s functionality & better UI in existing TCL tools

People felt there was a lot in the TSI, but they would have a hard time finding it or knowing it even existed. Many points were raised in the discussion as in “I’d like to see X in the TSI” and somebody else would say “you can already do that”. Clearly this means the interface design needs work & docs need to be improved.

  • Ability to view working memory changes in a separate window (i.e. showing a certain part of the graph and watching elements pop in and out over time).

6)Odds and Ends

  • A clearer boundary between TCL and Soar (I think was meant in terms of the user’s experience of the system—what features were coming from the interface and what from Soar itself. This is pretty clear if you know Soar first, but apparently less so if you learn the TCL+Soar combination).
  • Better support for inter-agent communication, as opposed to support for agent-environment communication. (I think the issue here is not that the primitives are missing for doing this in SGIO or whatever, but that a common layer above that is needed for inter-agent communication and this is often being re-implemented multiple times by users).