1. Quality of the Data
2. Data Collection Technique
|
1. QUALITY OF THE DATA
Data will be valid only if the measuring instrument employed correctly evaluates what it is intended to measure and if the method of data collection does not distort the data.
Data will be reliable only if the instrument used always measures with the same accuracy and if the method of data collection gathers the same information each time it is used with equivalent individuals in identical conditions.
The data are considered credible depending on the degree of confidence that can be placed in the source from which they are collected. Decisions should be based only on credible data.
2. DATA COLLECTION TECHNIQUES
Data collection techniques include:
(a) critical study of available documentation (document study);
(b) individual and group interviews;
(c) questionnaires.
Document study is largely dependent on the existence of documents pertaining to the programme under scrutiny. Relevant documentation may include a detailed description of the programme, documents justifying its existence, verbatim records, documents containing observations and proposals with regard to the application of the programme, monitoring and evaluation reports, and implementation schedules. Other types of documents may, in certain cases, prove useful, e.g. public and private archives, published statistics, data-banks, newspapers and personal documents.
Document study involves setting up matrices for the interpretation of the documents, identifying trends, and assessing to what degree the observations made can be applied generally to the programme. The documentation available is often not adequate for a detailed evaluation of the results, efficacy and value of the programme.
There are four types of interview techniques:
a) Unstructured interview: only the main topics of the interview are fixed. The interviewer is free to chose the questions as the interview progresses.
b) Focused interview: the interview focuses on a particular topic, in general an ongoing experiment in which those being interviewed have participated. This type of interview is similar to the unstructured interview, except that the questions are limited to a particular topic or activity.
c) Open-ended interview: the interview consists of a set of specific questions, which must all be asked, generally in a particular order. While the interviewer is constrained by this technique, the interviewees are free to respond as they wish.
d) Closed-question interview: the interview consists of questions and a limited choice of responses. The interviewees must answer all the questions, in a set order.
The interview technique is very useful for providing a wide variety of data in a relatively short period of time, using a restricted number of individuals - from one to a small group. The interview technique is frequently used during meetings of experts, taking advantage of the fact that they are concentrated in one place and can be asked for specialized data or information on a particular topic.
During the interview, the interviewer can, in order to get more specific information, ask participants to clarify or enlarge on their responses.
The interview has the advantage of being easily adaptable in situ to the demands of the investigation.Group interviews are more demanding than individual interviews and call for very systematic techniques enabling each participant to express their views freely and equitably in relation to the others in the group. It is recommended that group members should be asked to express their views on only a limited number of topics.
The interview technique nevertheless suffers from certain limitations, in particular with regard to large-scale evaluations, where the technique becomes very costly and the analysis of the interviews long and difficult.
When several persons or categories of individuals have to be consulted or when several aspects of the same programme are under scrutiny, the questionnaire is often the most practical and least costly instrument. Preparation of a questionnaire must be done collectively, using a careful, rigorous and cautious approach.
Questionnaires facilitate the processing and analysis of evaluation data. They may be given to a large number of individuals and although the response rate is lower than that achieved with interviews, it is generally sufficient to permit the evaluation to be carried out. The expected response rate is usually about 30 per cent.
The longer the questionnaire, the lower the response rate. A questionnaire should not, therefore, take longer than 20 minutes to complete. Where a large number of questions must be asked, it is preferable to construct more than one questionnaire and to divide the respondents into sample groups according to the number of different questionnaires. It is recommended supplementing the questionnaire by interviewing a small sample of individuals. In this way, issues brought out by the questionnaire can be explored in greater depth.
|
|
