CS588 Usability Testing Assignment
----------------------------------

Your Assignment in a nutshell: Perform a formative usability study of
your system, report what your experimental procedure was, where the
problems occurred, and what should be done to improve the system.

On April 24th, you should (1) hand in your written assignment [10% of
your overall score] (2) give a five-minute talk in class on how you
did the testing and what you have learned [5% of your overall score,
for the quality of the presentation itself].


1. What your hand-in should contain:
------------------------------------

(A) the URL of the project you tested

(B) a description of your experimental design (e.g. How were users
recruited?  What are the characteristics of your users? What is the
exact procedure you go through for every experiment?)

(C) a quantitative presentation of results (a table showing the raw
results; and a table showing a summary for each task, such as the
minimum, maximum, and average time)

(D) a qualitative presentation of results (e.g. What are the biggest
problems? If you had time, in what order would you address the
usability problems you found, and what changes would you make?  If you
had time to run another usability test, what changes would you make to
the experimental procedure itself?)

(E) a blank copy of the pre-experiment questionnaire (if there is one)

(F) a copy of any written instructions used in the experiment (either
the self-guiding instructions for the user or the instructions that
the experimenter reads from)

(G) a blank copy of the post-experiment questionnaire (if there is one)

and then, for each user:

(U) the signed and dated waiver form
(having that form signed is the *first* thing you do, before any
experiment! please! handing in any experiment without an accompanying
waiver form will give you a 0 grade!)

(V) the filled-out questionnaire for the subject, and anything else
that is an artifact of an individual experiment


2. Ideas for what you may want to say during your presentation:
---------------------------------------------------------------

How you designed your study - what the user tasks were - how you
recruited users - characteristics of the users - what the numerical
results were for each task - what the biggest problems were, and what
you would do to fix them.


Miscellaneous Tips & Tricks
---------------------------

Choose 2-5 tasks for the subjects to do. Run 4-10 subjects on these
tasks and record the time for each task (and if they needed hints,
record what kind of hint you gave).

For the 4-10 subjects above I expect you to have totally identical
experiments so that the results are actually comparable - that means
that you probably will want to run a pilot test to see if your
experiment works first (tasks are not too hard or too easy, etc.).

Practice your presentation! I will enforce the five-minute time limit!
Giving a good presentation includes actually being able to give it in
the allocated time!

You do not have to perform any statistical analysis on your
data. However, I still expect you to design the experiment such that
if subject A and B perform the same task, and each take two minutes,
that the times are actually comparable (so that you could get reliable
and statistically significant results with your experiment design if
you wanted to by running more subjects).  Great ways to invalidate
your study are: not having the same setup for every subject, not
having always exactly the same written or oral instructions for the
subjects, giving some subjects hints and not recording that you did
so, not having the same tasks in the same sequence for the subjects,
and so on.


Grading Criteria
----------------

I'll group the written hand-ins into equivalence classes, based on how
well the experiment was thought out and executed, and how easily I can
understand what you did from what you handed in.

For the oral presentation, I'll also group the presentations into 1-4
equivalence classes.