Quality is a complex notion that depends on the point of view of the different actors related to an MT system. It is most often related to the judgment of final users (Dostert); or defined as the composite measurement of fidelity, intelligibility and elegance (Johnson); or it is a results of the analysis of situational dimensions (House) - all in the Van Slype report.
The quality of the translation can be evaluated in two modes.
Without adjustment: this aims to evaluate the quality of translation before the dictionary and/or grammar is adjusted. This is also an absolute evaluation of the system (JEIDA report).
With adjustment: this aims to evaluate the quality of translation after the dictionary and/or grammar is adjusted. The higher quality of translation the user needs, the more severely the evaluation is made. In this respect this item shows the degree of the user's satisfaction with the system (JEIDA report).
All of the following definitions are taken from Van Slype's Critical Report, 1979.. It seems very important to keep them in mind before proceeding to further discussion of the quality features.
DEFINITION OF TRANSLATION
J. HOUSE -- Translation is the replacement of a text written in a source language by a semantically and pragmatically equivalent text written in the target language. (The translation of oral texts is a different activity, namely interpretation).
TRANSLATION QUALITY
L'ASSOCIATION JEAN FAVARD distinguishes: (a) the intrinsic qualities, which are independent of the reader; (b) the extrinsic qualities, which are related to the "text-reader" couple. A text, even badly translated (and thus of low intrinsic quality) can nevertheless, for an informed reader, be as clear as if it had been well translated. However, beyond a certain deterioration in intrinsic quality, the extrinsic quality becomes very poor.
H. BRUDERER -- Quality is a relative concept, i.e. one related to a specific object. Quality can apparently be measured, at least in part, but it remains much more difficult to quantify abstract (conceptual, subjective) phenomena than concrete (perceptible, real, tangible) things. Quality can be evaluated: (a) either positively assessment of merits, advantages; (b) or negatively assessment of deficiencies, errors, disadvantages; (c) or totally assessment of the positive and the negative aspects. The evaluation of the translation quality -- whether human or computerized -- has to take into account the following intralinguistic and interlinguistic factors morphology, syntax, content, terminology, style, conformity. A faithful translation reproduces the sense of the original text, but it does not necessarily, if it is to be considered an intelligent translation, have to be identical to the original text. Given that they partially overlap, content and fidelity should be evaluated on an overall basis. Similarly, it is difficult to differentiate clearly syntax and semantics. Style, on the other hand, influences all levels (morphology, syntax, semantics, terminology).
IR.L. JOHNSON defines translation quality by three factors fidelity, intelligibility and elegance. The importance of these three factors may vary with the type of text considered. Features can be observed: (a) superficially, via linguistic elements such as lexical and syntactic exactitude; (b) indirectly, via the reactions of the users to the translated text.
B. KUHLEN stresses that there is not a universal criterion for MT evaluation: (a) on the one hand because it does not seem that MT can ever reach the level of quality of human translation; (b)on the other hand, because the evaluation criteria have to be chosen according to the aim in view; (c) finally, because the individual parameters, which taken together permit an assessment of the quality of MT, often contradict each other, with the result that an overall rating would not be significant to the specific performance of the components.
Z.L. PANKOMICZ feels that usefulness of MT and HT has to be based on quality, speed and cost. Determination of the optimal balance between these three parameters depends on the environment of each translation activity. It is necessary to understand, in his view, that the quality of HT and MT is indefinable, at least in any absolute way. The assessment of the quality of HT is traditionally based on its completeness and on stylistic elements.
A.J. PETIT takes the view that the translation should not comprise misconstruction, but admits however a tolerance of up to I % of the sentences in the case of translations to be supplied raw to the final user and 2 % of the sentences in the case of texts to be revised before submission to the users. This tolerance is intended to allow for normal risks of error or accident.
Y. WILKS thinks that the purist who feels that the least translation defect nullifies the translation is often mistakes in two of his postulates: (a) he exaggerates the attention and comprehension which the average reader achieves with a technical document (consequently, errors of translation do not negate the value of the text); (b) he exaggerates the quality of the mass of human translations produced on an enormous scale and at high speed.
RELATIONSHIP BETWEEN TRANSLATION QUALITIES AND EVALUATION CRITERIA
According to G. BOURQUIN the criteria for evaluating a translation will vary according to whether it is produced by a human translator or by the machine: (a) from the human, "finesse" will be required open to the ethnoculture and to work on linguistics, the human translates with his sensivity, his intuition, his common sense; (b) the computer will be expected to offer regularity, precision, infallibility, speed, and encyclopedic exhaustiveness.
M. MASTERMAN notes that our ignorance of the very nature of translation leads to a discordance between the evaluation criteria used or proposed by various authors.
A.J. PETIT -- A product is acceptable only if it meets the requirements of its users. As regards texts (original texts or human or machine translations), the principal requirements are:
(1) For utility technical texts (maintenance or user manuals): (a) errors, (b) homogeneity, (c) clarity, without ambiguity or gibberish which might obscure the sense of the message, (d) simple correct style, without extravagances or recherche' elements, (e) use of the terms recognized in the relevant sector.
(2) For educational technical texts: (a) no technical errors, (b) adaptation of the terms recognized in the relevant sector.
(3) For documentary scientific texts: (a) clear exposition of theory, (b) without errors flowing style without excessively long sentences incorporating several different ideas, (c) use of the basic terminology of the discipline.
These requirements have however to be viewed from a different angle according to whether the translation is intended: (a)to be revised in this case, the translation system (human or machine) has to be aware of its own shortcomings, and indicate by itself all the ambiguities which it was not able to resolve it delivers an incomplete product, but one without serious defects; (b) to be supplied direct to the final user the translation must then be complete (experienced human translator or a computerised system producing a complete translation, without any misconstruction) and without serious defects (human error or accident both being normal risks).
THE AUTHORS OF THE REPORT PRESENTED BY PHILIPS distinguish between evaluation of translations with and without comparison with the source text. In the first case, it is necessary to assess in what measure the translation (a) reproduces which is stated in the original (for example contractual texts), (b) reproduces what the author of the original intends to say, with the certainty that the message is properly understood (for example translation of manuals). To assess the quality of a translation, it is necessary to answer the following questions. (1) On the aim of the translation: (1.1) does the translation reproduce the content of the original? (1.2) does the translation reproduce the formulations of the original? (1.3) does the translation reproduce the intention of the author? (2) On the type of text: (2.1) all the information presented? (2.2) can the translation achieve the desired effect? (2.3) have the necessary corrections been made in such a way that communication has the best chance of success?
In the second case, evaluation of the translation without reference to the original, the assessment of the quality of the translation has to cover: (a) the grammatical correctness, (b) style of idioms, (c) the use of current words, expressions and structures in the target language, (d) the absence of contradictions or ambiguities.
ASSESSMENT
The concept of the quality of a manufactured product is, in general, unambiguous the product has to correspond to the specifications and a battery of quality control tests can easily be arranged, and made the responsibility of controllers often relatively unqualified. The concept of translation quality is much more indeterminate, and the authors' contributions can be summarized fairly briefly. (1) The quality has to be assessed, not in the absolute, but according to the aims of the writer of the texts to be translated and by those who decide how it is to be distributed. (2) The quality achieved by HT can not be expected of MT, and the latter has thus to be used for more limited aims than the former (which does not mean that, within the scope of these limited aims, there does not exist a major potential demand). (3) The evaluation criteria have to be chosen according to these specific aims. (4) Since translation quality can not be measured in the absolute, on the basis of a single criterion, its assessment should combine several criteria.