# How to write a mathematical proposal/application that non-math reviewers will love.

## 1. Read the instructions

This may be the most important part of your future success. Most of us think we've done it, or that the instructions are standard/not important/we already know what they say/etc. The instructions contain **important hints** about what the reviewers are looking for, and you need to read them very closely - every time. Keep reading until you understand the criteria for the award - often these are in a separate section than the technical instructions which address "how to complete the application".

### Example

The University of Calgary Progression through the Ranks and Outstanding Achievement Awards processes are described [here](https://www.ucalgary.ca/hr/work-compensation/labour-relations/academic-labour-relations/academic-staff-tucfa/academic-performance-assessment). The Staff User Guide contains technical information about completing the OAA, but the 2024 GFC Handbook is the document which contains criteria for the award (copied below for your convenience):

4.3 OAAs
4.3.1 OAAs are intended to recognize excellence and an academic staff member’s exceptional and outstanding performance.
4.3.2 OAAs may be awarded to individuals, consistent with the provisions in the Collective Agreement on a competitive basis who, over the assessment period, significantly exceeded expectations in one or more categories while meeting or exceeding expectations in all other categories for their rank and stream as outlined in this document and the relevant Faculty Guidelines.
4.3.3 When comparing applications for OAAs, evaluators must consider the quality and impact of the academic staff member’s performance.
4.3.4 As a principle, expected standards of performance increase in relation to rank.

From these criteria, we can deduce that an applicant needs to provide evidence of *significantly exceptional performance for their rank* in at least one of teaching, service or research and that they must provide evidence of *exceptional or expected performance for their rank* in the other categories, and that the evidence needs to address the *quality and impact* of their work. Since OAA's are awarded to a very small percentage of the faculty (16.5%), one can expect that providing evidence of significantly exceptional performance in as many categories as possible will likely be needed to succeed. Applicants are advised to read sections 4.4, 4.5 and 4.6 for specific information about expected contributions to teaching, research and service and applicants should also read the Faculty-specific guidelines.

## 2. Quantify successes

Mathematicians as a rule have taken the stance that metrics such as the h-index are not dependable measures of research impact. However, most panels of scientists who are asked to assess the quality of an applicant's research will expect to be presented with concrete and measurable evidence of research excellence and impact. The American Mathematical Society has written a number of letters on behalf of the profession for outsiders who are assessing the research impact or other criteria used to evaluate mathematicians. These can be used by applicants to think about how they can frame their documents in a way which is both authentic to the discipline while also providing the kind of context and evidence that a committee needs in order to make a fair assessment. Please note: there is no similar Canadian resource, nor is there Canadian data similar to the US data used in the preparation of these information statements.

The complete list of information statements can be found [here](https://www.ams.org/about-us/governance/policy-statements/sec-ams-policystatements#:~:text=Information%20Statements%20of%20the%20AMS,September%202015;%20approved%20September%202007). Here is a summary of the most pertinent AMS letters:

| Topic | Link | Notable Content |
| -------- | -------- | -------- |
| Citation & Impact | [Link](https://www.ams.org/about-us/governance/cultureofresearch.pdf) | Authorship is often alphabetical |
| | |Assessment is best completed by expert analysis|
| | |Impact factors are not reliable for math journals|
|Publication Rates| [Link](https://www.ams.org/about-us/governance/committees/Statement_ratesofpublication.pdf)| Mathematicians publish at a rate which is modest compared to other sciences|
| | | A rate of 2 or fewer publications per year is reasonable|
|Joint research| [Link](https://www.ams.org/about-us/governance/committees/Statement_JointResearchanditsPublication.pdf)| Co-authorship is on the rise with often 4+ authors|
| | | Researchers' roles are seldom differentiated|
|Research funding| [Link](https://www.ams.org/about-us/governance/committees/Statement_FederalSupport.pdf)| Many productive math researchers receive little or no external funding, unlike other sciences|
|Directing PhD theses | [Link](https://www.ams.org/about-us/governance/committees/Statement_DirectingPhDTheses.pdf) | Directing a PhD thesis does not often advance the supervisor's research program |
| | | Joint publication between a supervisor and mentee is unusual |
|Structure of graduate programs|[Link](https://www.ams.org/about-us/governance/committees/Statement_structureofgradprograms.pdf)| Graduate research is often independent research|
| | | Graduate students are often funded through TA positions rather than research grants|
| | | Be cautious: this letter describes the US graduate student programs|
|Postdoctoral positions| [Link](https://www.ams.org/about-us/governance/committees/Statement_PostdocPositions.pdf)| Postdoctoral positions in math usually involve teaching|
|Undergraduate research|[Link](https://www.ams.org/about-us/governance/committees/Statement_UndergradResearch.pdf)| Undergraduate research is primarily a teaching activity for faculty|
|Teaching Loads|[Link](https://www.ams.org/about-us/governance/committees/Statement_TeachingLoads.pdf)| Teaching loads of 3 courses/year are typical for research-active universities|
|Arm's length evaluations| [Link](https://www.ams.org/about-us/governance/committees/Statement_armslengthofevaluation.pdf)| Co-authors may be suitable as arm's length reviewers of junior colleagues|

### Exercise

Where are there sources of information about the expected contributions in MATH, STAT or CPSC for each of research, teaching and service to the academy by faculty members? Fill in the table below with suggested resources (please email suggestions to bauerk at ucalgary dot ca).

This list should be considered "in progress":

| Topic | Link | Notable Content |
| -------- | -------- | -------- |
| MATH resources | [Link](https://www.ams.org/about-us/governance/policy-statements/sec-ams-policystatements#:~:text=Information%20Statements%20of%20the%20AMS,September%202015;%20approved%20September%202007) | AMS Letters |
| STAT resources | [Link](https://ssc.ca/en/accreditation) | SSC Accreditation |
| CPSC resources | [Link](https://learning.acm.org/resources) | ACM Resources Page |

In the workshop, several participants thought about different ways of quantifying certain contributions aside from using information from professional societies to normalize the culcture of research & scholarship in each discipline - e.g. to quantify the importance of a particular CS conference, one could consider including acceptance/rejection rates, size of the conference, international presence or other metrics.

### Example

Jane Doe is applying for an OAA. In the past year, she published a paper reporting a major breakthrough in her research program in a conference proceeding for a conference which honoured one of her academic mentors. Jane is concerned that the publication venue (which is relatively obscure) will not make it clear that the paper was innovative and impactful. To quantify the significance of this paper, Jane quoted from the AMS Information Statement on Citation & Impact indicating that assessment is best completed by expert analysis. She used this quote to justify why she did not include publication or other metrics as measures of the impact of her work. She then listed the large # of conference, colloquia and seminar invitations which she had received as a result of this work and used that to quantify the positive reception her work had received. She concluded with a quote from the reviewer's feedback which indicated the quality and impact of the paper.

## 3. Peer feedback

A good way to see if your application is likely to convey the importance of your work to someone outside your field is to seek peer feedback from friends or colleagues who are not directly involved in your work. Quickly jot down your ideas in the tables below. Pair up with a colleague who has done the same, and you can both help each other decide if the evidence you plan to provide is likely to convince a non-expert reviewer that your application should succeed (make sure your friend/colleague doesn't have the same expertise as you!).

Begin by investigating what the criteria are for the application or award you are applying for. If there are several, brainstorm ideas for each one. If you are applying for something which rewards your past performance (e.g. an award), brainstorm your contributions for each criterion and think about what evidence you will provide which will convince reviewers that your contributions are award-worthy. If you are applying for something which requires you to propose future activity (e.g. a research grant), think about what you will propose and what evidence you can provide that your proposed ideas will succeed. Keep your reviewers in mind, and if they are not in your field think about how you will provide your evidence in context to increase the likelihood that they will understand the significance of your contributions or ideas.

| Application/Award name: | (e.g. OAA) |
|----------- |--------- |
| Criterion: | (e.g. Research) |
| Bar for success: | (e.g. *significantly exceptional performance for my rank*)|
| Name of contribution/proposed idea | Evidence/Context of quality & impact |
| #1 | #1 |
| #2 | #2 |
| #3 | #3 |

You can think of the first column as the name of the contribution as you would write it *for yourself*, and the information in the second column as what you would need to communicate to *someone else*.