One of the biggest debates that we have in the field of social science research is the role of evaluation (e.g., evaluation of a policy, the completion of treatment goals or of the program itself) and where independence sits within it. Should organisations and governments conduct their own evaluations, or does that create a conflict of interest? Is all independent research really independent, or is it just independent of the organization/provider? How independent is it of the researcher�s own biases? It is important to consider these factors when deciding to do research, validate programmes, conduct evaluation and implement change.
The place to start is asking the right questions: What is the purpose of evaluation, research, and development of an evidence base; is it to justify what is being done, a required part of a larger grant or project, to see if what is being done is working, to justify an existing programme/venture or to implement something new in a broader fashion? Is it about �evidence based policy� [research driven] or �policy based evidence� [ideologically driven]? On the other hand, is an organisation implementing an evidence-based practice or developing practice-based evidence in earnest?
The reasoning behind the purpose of the research is as important as the research being conducted, especially if the research is being used to justify or implement something; therefore, is the research question trying to see if something is working or if something is going to work in a different setting. This means that evaluation can often usher in change to policies and programmes, which can be minor or major but often seen as significant by people on the ground. This can, in turn, bring about new questions: Should an organisation implement a top-down approach or method, or integrate it in accordance with the unique characteristics and contributions of those who work and benefit from it? It is important that everyone involved in the process recognizes the importance of evaluation in the evolution of our research and working; it�s not a threat but rather an opportunity to see if something works, to identify good practice and identify bad practice. Therefore, who should conduct evaluation research; organisations themselves or external organisations/researchers?
It is generally viewed that good research is independent research as organisations are often considered to be biased in how they evaluate their own programmes and practices. The notion that organisations cannot conduct their own critical, methodologically sound and unbiased research is outdated, problematic and questionable. The issue is not with research or methodology, but rather with how organisations understand, priorities and train their staff in research. If staff do not fully understand research and its impact, then their ability to develop, conduct, justify and roll out research becomes problematic. This is a training issue and investment issue, not an independence issue. This is reinforced by the fact that not all research and researchers are independent with the same researchers being used by the same organisations frequently and/or with research tending processes being subjective. Which means that so called independent research can be as subjective and biased as research conducted by the organisation itself; particularly, if the organisations do not have their own in house understanding of research, their own critical reflection of research and how to process it.
The processes involved in research involve interaction between the researcher and the organisation, regardless of whether the research is independent or not, with both needing to be able to work together to collect the data and implement change. A decision not to engage in the research process or implement changes is problematic regardless of who conducts the evaluation and has nothing to do with independence, but rather it�s about quality control.
The most important part of doing evaluation research is the process and methodology, it is about accepting the good and the bad on both fronts [researcher and organisation]; it�s about us and us, not them and us. Evaluation is about change and evolution, not revolution and condemnation. Evaluation is about maintaining funding, generating funding, maintaining good practice and eliminating bad practice. It�s a process, not an end point.
Kieran McCartan, PhD, & David Prescott, LICSW
Comments
Post a Comment