---
# System prepended metadata

title: Researcher Evaluation

---

# Researcher Evaluation #
---

Two essentially questions

1. How do we recognize a good researcher potentially contributing to development? 
2. Who are the researchers who deserve support?

Two angles

1. the researcher's professional career
2. universities & DeIC's mission (how best to achieve)

_prototype: senior researcher: an individual with already a significant publication record (good or bad – that is the question…)_

Three dimensions: 1) output, 2) profile, 3) environment

a. Research output and publications
    * relevance for development of his/her research
    * scientific quality: this may include originality/innovation
    * valorisation
    * **performance (citations, h-index, i10-index = the number of publications with at least 10 citations)**
    * participation in conferences with scientific contributions, and publication in the proceedings 

b. Researcher's profile
    * academic record: performance in both research and education
    * (concern for impact: it can be actual impact or potential impact on development)
    * membership in networks, active networking
    * notoriety, national and international dimension: invited lectures
participation in foreign research projects, doctoral committees, reviewer for scientific journals, consultancies, collaboration with industry
    * **Capacity to competitively attract funding**
    * **Prizes, awards**

c. researcher's environment
    * **Is the researcher a member of a team that possesses the necessary competencies and skills, information, access to documentary resources, &c.?**
    * institutional support for applied project (co-funding, part of departmental strategy ...)




---

### Comments ###

* moment in the researcher's history
* differences in publication culture

---
Evaluating individual researchers
General
Senior researchers
"Future" researchers
General

The main questions the authors of a specific guide should address are: “How do we recognise a good researcher potentially contributing to development? Who are the researchers who deserve support?” The answer to such questions is fundamental in any research environment or research institution.

Individual researchers can be evaluated from mainly two angles :

from the point of view of the researcher’s professional career,
from the point of view of the institution or stakeholder who is interested in how to best achieve its mission (university, research institute, foundation, etc.).
Yet in quite a few cases, such as the awarding of a prize or the renewal of a contract, both parties might be equally interested in an evaluation.

The objectives of evaluation will therefore take these aspects into consideration, and the author(s) of a specific guide may conveniently adopt the breakdown which is taken in the present Guidelines.

The prototype situation adopted here is that of a senior or “seasoned” researcher: an individual with already a significant publication record (good or bad – that is the question…). The case of a “future” researcher will be examined after the case of the senior researcher hereafter.


Basis for evaluating senior researchers

The idea here is that the evaluation of a development researcher should take into consideration three major aspects:

Research output and publications
The researcher’s profile, his personal characteristics and record,
The environment, scientific as well as ecological.
(1) Research output and publications

Relevance for development of his/her research
Scientific quality: this may include originality/innovation
Valorisation 
Performance 
Participation in conferences with scientific contributions, and publication in the proceedings
(2) Researcher’s profile

Academic record: performance in both research and education
Motivation towards development research: is he/she able to identify the needs for development? Does he/she demonstrate an intrinsic desire to do development research? 
Does he/she take initiatives to apply his/her own or other research results for practical projects for the benefit of (local) society? Is he/she active as a personal consultant or **practicing** engineer?
Concern for impact: it can be actual impact or potential impact on development
Concern for strengthening research capacity 
Membership in networks, active networking
Notoriety, national and international dimension :
invited lectures
participation in foreign research projects
doctoral committees
reviewer for scientific journals
consultancies
collaboration with industry
Capacity to competitively attract funding 
Prizes, awards
Managerial capacity: capacity to identify the best research strategies, to develop synergies accordingly, to run projects cost efficiently, to work in transparency and in team 
Moment in the researcher’s personal history when the evaluation takes place
Recommendations (standard, personalised).
(3) The researcher's environment

In the particular case of development research this is a particularly relevant aspect. Is the environment of the researcher stimulating his/her research or does it rather work against it?

The context is important to know: the researcher lives in an environment that determines his/her training; his/her capacity to move and to act; the manner in which he/she is supervised; his capacity to write and to have access to scientific publications; the application of his/her results; etc. 

Is the researcher a member of a team that possesses the necessary competencies and skills, information, access to documentary resources, etc.?

Also to take into consideration: the country in which the researcher operates, and the degree of development of research in that country.

Objectives of evaluation

Evaluating a researcher may have two major purposes, and sometimes both: the personal advancement of the researcher, and the best use of his/her talents and production.

Personal advancement of the researcher
Awarding of a degree
Appointment or extension of a contract
Promotion
Giving a prize or an award
Funding further research
Granting a travel or post-doctoral fellowship
Providing scientific support
Providing material and/or institutional support
 
Optimal use of the researcher’s talents and production
Inviting him/her as a jury member of a doctoral committee; the scientific committee of a foundation or a prize; as a referee for a scientific journal or a committee; etc. 
Establishing partnership: deciding to join him/her (or to invite him/her) in a collaborative study or in a network
Strengthening of an institution’s research capacity: leadership; organisation of research; management in general; institutional image or notoriety; etc.
Strengthening of the institutional or individual researcher’s accountability
Further dissemination of knowledge 
In addition, when possible, the performance of a researcher should be measured against his/her own personal objectives. If work in a development context is part of it, it should be evaluated and properly acknowledged.

Formulating the objectives of evaluation 

This step stands primarily for the ex post evaluation of a seasoned researcher, the situation we adopted as a prototype. Yet there are also ex ante situations in which the assessment of the researcher is based on assumption and prediction, and is therefore essentially subjective and estimative. In such cases the specific guide's authors will have to pick from the elements provided above what is useful for their needs. See also below the section on future researchers. 
The objectives of evaluation will therefore take all these aspects into consideration, and the author(s) of a specific guide may refer to the following section.

Writing up a specific guide for evaluating  individual researchers 

From here on the authors will follow - flexibly, and adjusting the
pace and orientation of their work - the steps provided in the page "writing the specific guide". 

Step 1. Preliminary stage fulfilled
Step 2. Questions of evaluation, dimensions, criteria
Step 3. Select procedures
Step 4. Conditions of evaluation
Step 5. Actual drafting of the specific guide

Caution: such steps are the various stages of production of an acceptable and useful specific guide. They are not the steps to be followed by the evaluators when they conduct their evaluation. 

A special case: "future" researchers

The issue here is that of evaluating an individual without publications yet, or without a significant publication record in the field under consideration. It is typically an ex ante situation.

The main question the authors of a specific guide should address is:
how do we recognize an individual who: 

has (or seems to have) the potential of conducting good development research, or
meets (or seems to meet) the conditions for being (or becoming) a good researcher? 
Two cases in point:

(1) A young researcher (defined by his/her age) not established yet, but with seemingly a potential for research. A typical example is a PhD candidate.

(2) An individual who is not young anymore, but can be considered as a “future” researcher in the field under consideration, for reasons such as: 

has a publication record, but is not sufficiently experienced in the subject under consideration,
comes from a country where research is not well recognised or valorised, 
has published only in local journals. 
A typical example is a middle-aged academic from a low-income country.

Objectives of evaluating a “future” researcher

The most common objectives are:

accepting a researcher as a PhD candidate 
granting him/her a doctoral fellowship
funding his/her research 
providing scientific and/or institutional support 
strengthening of institutional research capacity. 
Basis for evaluation

In such cases there always exists an unavoidable degree of uncertainty. The specific guide should emphasize the need – and the means – to base the evaluation’s conclusions partly on presumptions.

In the absence of a significant publication record the evaluation will be based on:

An appraisal of the researcher’s potential, his/her profile, his/her personal characteristics and record
The research proposal,
The institutional environment. 
(1) The researcher’s profile, personal characteristics and record

Academic record :This is not always a good indicator for the skills a future researcher may need. Did he/she benefit from a good training in research: Where? Under who’s supervision? Following which model?
Motivation for doing research and for doing something with research results 
Commitment towards development 
Recommendations, standard or personalised.
(2) The research proposal 
Methodological quality
Relevance of the subject, as illustrated by a bibliography or relevant analysis 
(3) The institutional environment 

Chances of success of the proposed research: institutional support, 
Supervision structure 
Integration in a group of researchers/students(when applicable)
Writing up a specific guide for evaluating  "future" researchers 

From here on the authors will follow - flexibly, and adjusting the pace and orientation of their work - the steps provided in the page "writing the specific guide". 

Step 1. Preliminary stage fulfilled
Step 2. Questions of evaluation, dimensions, criteria
Step 3. Select procedures
Step 4. Conditions of evaluation
Step 5. Actual drafting of the specific guide

---

http://www.guidelines.kaowarsom.be/evaluation_researchers_individual