FEMTI's backbone is made of two classifications or taxonomies. The first one is intended to be used in determining what the user's needs are. We suggest that if you are using FEMTI in order to design your own evaluation, you start by working through this section. You may well find that as you work through, the information solicited by later steps causes you to modify your responses at earlier steps. In that sense, the first taxonomy is to be thought of as forming a feedback loop which you may want to go through several times. The steps given here are meant to conform to those proposed in ISO/IEC 14598 for designing an evaluation.

The second taxonomy concerns quality characteristics of MT systems which are potentially of interest. In the light of your selections from the first taxonomy, it should be possible to pick out those characteristics which are pertinent to the needs you have made explicit. Following the pointers from these quality characteristics will lead you to possible metrics which can be applied to measure a particular system's performance with respect to that characteristic. The system characteristics are structured at the top level in conformity with the ISO/IEC 9126 standard, which concerns quality characteristics of software products. There is however one major divergence from the ISO/IEC 9126 standard in the inclusion of costs as a top level quality characteristic, as well as in the separation of internal (2.1) and external (2.2) quality characterictis. ISO/IEC 9126 considers cost to be a management factor, which is taken into account only after the evaluation proper has been done rather than an inherent factor influencing the quality of the system. In the specific case of MT systems, where cost may be a critical factor in determining whether it is even worthwhile designing and carrying out a full scale evaluation, we have thought it sensible to promote costs to the status of a quality characteristic.

In the future, we would like to be able to provide automatic pointers from specific user needs to specific system characteristics as well as from there to specific metrics. Indeed, some such pointers are already included. However to do this fully requires substantial user validation of our classification scheme, as well as much reflection on the relationship between needs and system features. You can contribute to this reflection by providing us with feedback on your experience of using FEMTI in its current state, by following the Comments link, at the toplevel or for each individual taxon.