Who & When
||Prof. Dr. L. de Stadler (University of Stellenbosch)
||Masters students from the participating universities
Prof Dr L G de Stadler
Prof Dr C J M. Jansen
Prof Dr A Carstens
Prof Dr P Swanepoel
|Date of completion
In the field of Document Design Studies, three types of evaluation techniques are distinguished: expert-focused, reader-focused, and text-focused techniques (cf. Schriver 1989). The advantages and disadvantages of using various techniques in a culturally homogeneous situation are fairly well documented (cf. De Jong & Schellens 1998; Lentz & De Jong 1997). Hardly any attention, however, has been spent so far to the costs and benefits of implementing the available evaluation techniques in situations where the demographic and cultural background of both evaluators and target groups vary as largely as they do in South Africa.
When it comes to reader-focused evaluation, it is conceivable that for some target-groups the plus-minus method would be quite successful in revealing the strengths and weaknesses of the text, while for other target-groups the think-aloud method or focus groups are more appropriate (cf. Jansen & Steehouder 1989), to mention only two techniques.
When it comes to text-focused or expert-focused evaluation, it is likely that the various backgrounds of evaluators and content experts in South Africa have an influence on the outcomes of using evaluation instruments. Moreover, the existing instruments for analysing and evaluating persuasive and instructive messages (cf. Hoeken 1998; Steehouder & Jansen 1997) need to be adapted to the specific content of HIV/AIDS-messages and the specific attitudes, knowledge and insights of the various target-groups in South Africa.
- What are the characteristics of a text-focused evaluation instrument that can effectively and efficiently reveal the strengths and weaknesses in (drafts of) HIV/AIDS prevention documents in South Africa?
- How does the background of a text evaluator or a content expert influence the outcomes of applying such an instrument for the analysis of these documents?
- What are the benefits of using and combining various evaluation techniques for documents aimed at culturally and demographically diverse target groups?
In Document Design research (see, for instance, De Jong & Schellens 2000) two types of testing are distinguished: summative testing which is aimed at assessing a final document by measuring its effects, and formative testing, where the goal is to improve on the design of a text draft document to fit the characteristics of its target-group by locating possible reader problems in the text and by revising it to overcome these problems. In this project the focus will be on both types of testing.
Much attention will be paid to the development of an adequate text-focused evaluation instrument. Apart from more or less 'traditional' text characteristics like structure (organization; lay-out) and style (tone; information density) (cf. Steehouder et al. 1999, Renkema 1996, Pander Maat 1991), this instrument should make clear which types of argumentation or other persuasion techniques should be used (cf. Hoeken, 1998), which reader and writer personas should be created (cf. Steehouder 2000 a, 2000b) and how culturally specific beliefs and values should be taken into account (cf. Van Dyk 2001). To be able to do so, the underlying theoretical concepts (argumentation, persuasion techniques, reader and writer persona, beliefs and values) will be explored and made operational for application in the practice of text evaluation. In this regard the research will be strongly embedded in the theory of Functional Text Analysis.
The evaluation instrument will be developed in a continuous interplay between the design of new, readjusted versions of this instrument and the experiences in applying it to actual HIV/AIDS public information documents. The outcomes of texts analyses, performed both by professional document designers and by content experts coming from different backgrounds, will be mutually compared. They will also be compared with the results of applying various reader-focused methods (cf. De Jong & Schellens 1995; Renkema & Wijnstekers 1997).
The emphasis in these comparisons will be on the severity of the problems that are identified and on the quality and the feasibility of the solutions that are suggested. For a number of documents the effects of taking these measures will be tested independently. Special attention will be paid to the fine-tuning of digital public information documents on HIV/AIDS (for example websites); existing methods for pretesting this kind of document (cf. Sienot 1997) will be explored and adapted to the specific context of culturally sensitive health messages.
Developing the first version of the evaluation instrument (2002-2003)
Applying successive new versions of the instrument to HIV/AIDS public information documents (various groups of evaluators and content experts) (2003-2006)
Testing these documents in the target groups, following various reader focused methods, comparing the outcomes with the results of text evaluation (2003-2006); revising documents, and comparing the effects of these documents with earlier versions
Expected main findings
Expected packaged results
- Insight in the benefits of using and combining various evaluation techniques for documents aimed at culturally and demographically diverse target-groups
- Instruments and guidelines for evaluating (drafts of) HIV/AIDS documents in South Africa; revising the documents according to the outcomes of the analyses
- Research articles
- Workshop contributions