Evaluation

There are several online-sites which provide useful hints how to evaluate CALL-software. Here is a list of some of them:

These schemes were created to give a language teacher a check-list for evaluating CALL-programs. For the purpose of doing research on CALL-programs from a computational linguistics perspective these can also be used as guidelines. Ultimately a program using CL-technology should perform very well (and better) in a selected area. As is explained in more detail in the next section, programs can be looked at from two sides. One is the computational view and the other is the linguistic one. This could also be divided into a technical and a contentwise view. Considering this the evaluation then either focuses on how the program works, which types of exercises are presented, in which way the error recognition works etc. The focus may also be on what material is presented, how is it prepared and which linguistic areas are covered.

There is also the possibility to develop a specific questionaire for certain aspects of CALL-programs. In this case the questions have to be adapted to the aspect. Some aspects might include the help-system of a CALL-program or the possibilities for data-storage and learner-modelling.

The items of a checklist then have to be adapted also according to the available methods of CL. A question like "Is the program able to lead a speech-based dialog?" is then not valid because this kind of technology is not yet available.

The main evaluative questions with respect to the integration of methods of CL need to be directed towards the capabilities of the CL-application. Not just any CL-based application is usable in a language learning context. It must be clear to the learner whether some linguistic knowledge is required to interpret the output of the program or if the output can be understood very easily. The application can also be "disguised", in that it only supports the learner in the background. Eg. tools for corpus queries may not present the raw data but may be embedded in an environment.

A second area is the question of whether the integration makes sense with respect to new didactic concepts of learning/teaching. A program, which still presents vocabulary exercises for drilling vocabulary lists is still didactically questionable, even if CL-methods are used for e.g. the generation of the exercises. When a concept is evaluated this has to be taken into consideration, too.