Martin Frank - lecture notes,
CS588 evaluation

----------------------------------------
Note: these notes are made available to
you for use in this class only ; please
do not re-distribute them without my 
permission.
----------------------------------------
Also, obviously, these notes are written
to help me stay organized during the 
lecture, not as stand-alone reading.
They may not make any sense to you if
you didn't attend the class.
----------------------------------------


draws from:
	Newman & Lamming, chapter 8 & 9
	Preece et al., chapter 29-34



Evaluation of User Interfaces
-----------------------------

Why do evaluation? 
-> understanding the real world (how is an existing system used)
-> comparing designs
-> engineering towards a target (like the one in usability criteria)
-> checking conformance to a standard

analytical vs. empirical evaluation
analytic: no users, experts look at a design idea
empiric: lets users try a prototype or real system
(others - not discussed: collect user's opinions)

cheaper
can be done earlier
does not involve real users (time, availablity)

usability "inspection" vs. usability "testing" (same as analytical vs. emperical)


A analytical evaluation
------------------------

"walkthroughs" for determining action sequences

"benchmarks" - important activities that will be used
to compare alternative designs



A1 keystroke-level analysis
----------------------------

GOMS Analysis - Goals, Operators, Methods, Selection rules

do both examples in the book, then do DOS vs. Mac GOMS example


A2 "Cognitive Walkthrough"
---------------------------

analyse in terms of "exploratory learning": if you just walked up to
the system and use it for the first time


interaction as a series of steps, at every step we ask ourselves:
1. Is the correct action sufficiently evident to the user?
2. Does the system response make it evident if we made a right or
   wrong choice?

go through the fare machine example (
paint on blackboard and explain how it works,
ask the class if they can see "flaws" right away,
then step through the example in the book)



Thomas example




A3 Heuristic Evaluation
------------------------

synonymous to "usability inspection"

method of operation is not completely predictable
users are neither novices nor experts

usability inspection means: "evaluators go through the user interface
with one specific design heuristic in mind"

heuristics examples: (book figure 8.12, really from Nielsen 93)
  - speak the user's language
  - minimize user memory load
  - be consistent
  - provide feedback
  - provide clearly marked exits
  - provide short cuts
  - prevent errors

usability inspection - advantages:
	cheap, can be done early on, no advance planning required
disadvantages:
	focuses on small isolated problems rather than the overall design








CD online-ordering example
benchmark task: buy the latest Michael Jackson CD, I want to know what the
total charge is (price,shipping&handling,taxes)


cdland
cdworld
cdteleshop
cdnow
cduniverse