Researcher Evaluation

# Researcher Evaluation # --- Two essentially questions 1. How do we recognize a good researcher potentially contributing to development? 2. Who are the researchers who deserve support? Two angles 1. the researcher's professional career 2. universities & DeIC's mission (how best to achieve) _prototype: senior researcher: an individual with already a significant publication record (good or bad – that is the question…)_ Three dimensions: 1) output, 2) profile, 3) environment a. Research output and publications * relevance for development of his/her research * scientific quality: this may include originality/innovation * valorisation * **performance (citations, h-index, i10-index = the number of publications with at least 10 citations)** * participation in conferences with scientific contributions, and publication in the proceedings b. Researcher's profile * academic record: performance in both research and education * (concern for impact: it can be actual impact or potential impact on development) * membership in networks, active networking * notoriety, national and international dimension: invited lectures participation in foreign research projects, doctoral committees, reviewer for scientific journals, consultancies, collaboration with industry * **Capacity to competitively attract funding** * **Prizes, awards** c. researcher's environment * **Is the researcher a member of a team that possesses the necessary competencies and skills, information, access to documentary resources, &c.?** * institutional support for applied project (co-funding, part of departmental strategy ...) --- ### Comments ### * moment in the researcher's history * differences in publication culture --- Evaluating individual researchers General Senior researchers "Future" researchers General The main questions the authors of a specific guide should address are: “How do we recognise a good researcher potentially contributing to development? Who are the researchers who deserve support?” The answer to such questions is fundamental in any research environment or research institution. Individual researchers can be evaluated from mainly two angles : from the point of view of the researcher’s professional career, from the point of view of the institution or stakeholder who is interested in how to best achieve its mission (university, research institute, foundation, etc.). Yet in quite a few cases, such as the awarding of a prize or the renewal of a contract, both parties might be equally interested in an evaluation. The objectives of evaluation will therefore take these aspects into consideration, and the author(s) of a specific guide may conveniently adopt the breakdown which is taken in the present Guidelines. The prototype situation adopted here is that of a senior or “seasoned” researcher: an individual with already a significant publication record (good or bad – that is the question…). The case of a “future” researcher will be examined after the case of the senior researcher hereafter. Basis for evaluating senior researchers The idea here is that the evaluation of a development researcher should take into consideration three major aspects: Research output and publications The researcher’s profile, his personal characteristics and record, The environment, scientific as well as ecological. (1) Research output and publications Relevance for development of his/her research Scientific quality: this may include originality/innovation Valorisation Performance Participation in conferences with scientific contributions, and publication in the proceedings (2) Researcher’s profile Academic record: performance in both research and education Motivation towards development research: is he/she able to identify the needs for development? Does he/she demonstrate an intrinsic desire to do development research? Does he/she take initiatives to apply his/her own or other research results for practical projects for the benefit of (local) society? Is he/she active as a personal consultant or **practicing** engineer? Concern for impact: it can be actual impact or potential impact on development Concern for strengthening research capacity Membership in networks, active networking Notoriety, national and international dimension : invited lectures participation in foreign research projects doctoral committees reviewer for scientific journals consultancies collaboration with industry Capacity to competitively attract funding Prizes, awards Managerial capacity: capacity to identify the best research strategies, to develop synergies accordingly, to run projects cost efficiently, to work in transparency and in team Moment in the researcher’s personal history when the evaluation takes place Recommendations (standard, personalised). (3) The researcher's environment In the particular case of development research this is a particularly relevant aspect. Is the environment of the researcher stimulating his/her research or does it rather work against it? The context is important to know: the researcher lives in an environment that determines his/her training; his/her capacity to move and to act; the manner in which he/she is supervised; his capacity to write and to have access to scientific publications; the application of his/her results; etc. Is the researcher a member of a team that possesses the necessary competencies and skills, information, access to documentary resources, etc.? Also to take into consideration: the country in which the researcher operates, and the degree of development of research in that country. Objectives of evaluation Evaluating a researcher may have two major purposes, and sometimes both: the personal advancement of the researcher, and the best use of his/her talents and production. Personal advancement of the researcher Awarding of a degree Appointment or extension of a contract Promotion Giving a prize or an award Funding further research Granting a travel or post-doctoral fellowship Providing scientific support Providing material and/or institutional support Optimal use of the researcher’s talents and production Inviting him/her as a jury member of a doctoral committee; the scientific committee of a foundation or a prize; as a referee for a scientific journal or a committee; etc. Establishing partnership: deciding to join him/her (or to invite him/her) in a collaborative study or in a network Strengthening of an institution’s research capacity: leadership; organisation of research; management in general; institutional image or notoriety; etc. Strengthening of the institutional or individual researcher’s accountability Further dissemination of knowledge In addition, when possible, the performance of a researcher should be measured against his/her own personal objectives. If work in a development context is part of it, it should be evaluated and properly acknowledged. Formulating the objectives of evaluation This step stands primarily for the ex post evaluation of a seasoned researcher, the situation we adopted as a prototype. Yet there are also ex ante situations in which the assessment of the researcher is based on assumption and prediction, and is therefore essentially subjective and estimative. In such cases the specific guide's authors will have to pick from the elements provided above what is useful for their needs. See also below the section on future researchers. The objectives of evaluation will therefore take all these aspects into consideration, and the author(s) of a specific guide may refer to the following section. Writing up a specific guide for evaluating individual researchers From here on the authors will follow - flexibly, and adjusting the pace and orientation of their work - the steps provided in the page "writing the specific guide". Step 1. Preliminary stage fulfilled Step 2. Questions of evaluation, dimensions, criteria Step 3. Select procedures Step 4. Conditions of evaluation Step 5. Actual drafting of the specific guide Caution: such steps are the various stages of production of an acceptable and useful specific guide. They are not the steps to be followed by the evaluators when they conduct their evaluation. A special case: "future" researchers The issue here is that of evaluating an individual without publications yet, or without a significant publication record in the field under consideration. It is typically an ex ante situation. The main question the authors of a specific guide should address is: how do we recognize an individual who: has (or seems to have) the potential of conducting good development research, or meets (or seems to meet) the conditions for being (or becoming) a good researcher? Two cases in point: (1) A young researcher (defined by his/her age) not established yet, but with seemingly a potential for research. A typical example is a PhD candidate. (2) An individual who is not young anymore, but can be considered as a “future” researcher in the field under consideration, for reasons such as: has a publication record, but is not sufficiently experienced in the subject under consideration, comes from a country where research is not well recognised or valorised, has published only in local journals. A typical example is a middle-aged academic from a low-income country. Objectives of evaluating a “future” researcher The most common objectives are: accepting a researcher as a PhD candidate granting him/her a doctoral fellowship funding his/her research providing scientific and/or institutional support strengthening of institutional research capacity. Basis for evaluation In such cases there always exists an unavoidable degree of uncertainty. The specific guide should emphasize the need – and the means – to base the evaluation’s conclusions partly on presumptions. In the absence of a significant publication record the evaluation will be based on: An appraisal of the researcher’s potential, his/her profile, his/her personal characteristics and record The research proposal, The institutional environment. (1) The researcher’s profile, personal characteristics and record Academic record :This is not always a good indicator for the skills a future researcher may need. Did he/she benefit from a good training in research: Where? Under who’s supervision? Following which model? Motivation for doing research and for doing something with research results Commitment towards development Recommendations, standard or personalised. (2) The research proposal Methodological quality Relevance of the subject, as illustrated by a bibliography or relevant analysis (3) The institutional environment Chances of success of the proposed research: institutional support, Supervision structure Integration in a group of researchers/students(when applicable) Writing up a specific guide for evaluating "future" researchers From here on the authors will follow - flexibly, and adjusting the pace and orientation of their work - the steps provided in the page "writing the specific guide". Step 1. Preliminary stage fulfilled Step 2. Questions of evaluation, dimensions, criteria Step 3. Select procedures Step 4. Conditions of evaluation Step 5. Actual drafting of the specific guide --- http://www.guidelines.kaowarsom.be/evaluation_researchers_individual