# [Internal] Invited review process Last updated: 2023-01-30 URL for this page: https://hackmd.io/@investinopen/invited-review URL for this site: https://hackmd.io/@investinopen/how-we-work ## What is this? An **internal** process to faciltiate the invited review process for research outputs, including [preliminary investigations](https://hackmd.io/@investinopen/research-products#Preliminary-investigation), [analytic memos](https://hackmd.io/@investinopen/research-products#Analytic-memo), and other works. Not to be confused with the [Reviewer guide](https://hackmd.io/@investinopen/reviewer-guide), which is a complementary piece of documentation shared with reviewers with instructions and guidelines for reviewers participating in this reviewing process. ## Background - The invited review process aims to improve the accuracy and reliability of our research, before we publish/share that research with a wider audience. - Typically, a research report draft is made available for invited reviewers' comments for 2 weeks; this time could be shortened when quick turnaround is required, although we try to have at least two weekends within the review period. ## Timeline ### 5-6 weeks ahead of desired publication date 1. **Determine the timeline for the invited review and publication** - Determine this with the Director of Research - Typically, a research report draft is made available for invited reviewers' comments for 2 weeks; this time could be shortened when quick turnaround is required, although we try to have at least two weekends within the review period. - After review is closed, allow at least 1 week (preferably 2-3 weeks) for report author to respond to reviewers' comments - Make sure the review's open, close, and report publication dates are added to the [Communications Calendar](https://trello.com/b/ufhetty6/communication-calendar) *(only accessible by IOI staff)* 2. **Identify invited reviewers** - Aim to identify 6 reviewers to invite - Invited reviewers are usually: - Researchers in the research domain of the report - Practitioners with lived experience in field that the research is studying - In the case of reporting out from user research, the interviewees/users involved - Solicit team contributions to the list of invited reviewers during a Research weekly meeting; and/or on Trello - Deprioritize reviewers who have just been engaged to review another piece of research from us - Note all names and email addresses on [this spreadsheet](https://docs.google.com/spreadsheets/d/1pJL0WfD3mTDvhcnuE0bRp8GZ1cAktL2eFiHf0Y8UXVs/edit#gid=1387591992). *(only accessible by IOI staff. n.b. this spreadsheet will be replaced by a CiviCRM process by December 2022.)* ### At the start of an invited review process 1. Ensure that the draft is ready to be reviewed - that most internal comments have been addressed and that the author is OK for the report to be sent out for review 2. Prepare a copy of the draft for reviewers - make a copy of the report (usually a Google Document); put this copy in the `Team Drive > Communications > Public Content > DraftsOpenForComments`; make sure that everyone with the link has commenting abilities - In the file name, put `[Draft open for comments]` before the rest of the name - In the header, put the following text in blue: > This is a draft currently open for invited reviews, up to and including [review closing date]. > Please be reminded that we will publish a version of the report with all reviewers comments, on our public Zenodo community collection. Your name may be recorded in this version if you choose to comment while signed in your Google account. More information can be found on our [Reviewer Guide](https://hackmd.io/@investinopen/reviewer-guide). > ![A screenshot of an example header](https://hackmd.io/_uploads/HJlBIMwPj.png) - In the acknowledgement section (usually towards the end of the report), put the following text: > We would also like to thank [Name and Name] for providing feedback to the complete draft of this report. - Highlight this text in acknowledgement and put the following comment: > Note to reviewers: we'd love to acknowledge your contributions to this report. Please let us know by replying to this comment or by email if you'd rather NOT be publicly acknowledged for your effort. Thank you! 3. Send invitation emails to invited reviewers - Adapt [this template](###Reviewer-invitation-email) ### During the invited review period 1. Address questions from reviewers; direct them to the IOI research team where necessary 2. Thank reviewers who have commented 3. Note reviewers who have already commented/declined in [this spreadsheet](https://docs.google.com/spreadsheets/d/1pJL0WfD3mTDvhcnuE0bRp8GZ1cAktL2eFiHf0Y8UXVs/edit#gid=1387591992) 4. **2 working days before the invited review closes** send a reminder to invited reviewers who haven't commented/declined; adapt [this template](###Reviewer-reminder-email) ### After the invited review period 1. Thank reviewers who have commented; adapt [this template](###Reviewer-thank-you-email) 2. Change the header text in the draft to the following: > This draft is currently being revised - thank you for your comments and input. > ![A screenshot of the example header](https://hackmd.io/_uploads/SJj_Y7DDj.png) 3. The author of the report/director of research will acknowledge the receipt of the individual comments and answer questions by replying to them directly. They will make bigger changes to the text on a separate, internal copy of the report. 4. The project coordinator would deposit a public copy of the report with the comments on Zenodo in [IOI's community collection](https://zenodo.org/communities/investinopen/). 5. **When the report is published**: inform the reviewers and thank them once again. Adapt [this template](###Post-publication-email-to-inform-reviewers). ## Templates ### Reviewer invitation email Subject: `A concise title of the report` Dear `first name`, We have conducted a `title of the report, linked to the draft open for comments` and would like to invite your questions, comments and feedback to our draft report, by `invited review closing date`. `2-3 sentences summarizing the report`. Given your expertise in`their area of expertise/experience`, `name of person who recommended we reach out to them` has suggested reaching out to you for early comments and review on this draft. **We would be really grateful for your questions, feedback and suggestions, up to and including`invited review closing date`: `link to the draft open to comments`** Your input will help advance our collective understanding of `area of investigation of the report` For more information and tips on this process, please refer to [our reviewer guide](https://hackmd.io/@investinopen/reviewer-guide). `(if this is reporting out from user interviews, include this paragraph)` Please note that we’re simultaneously asking all interview participants involved to review this draft and collecting consent for public disclosure of their identities. All participants contribute authoritative insight into this space. After this review process, we will revise the report. We aim to publish the final version later in `expected publication month for the report`. Please don’t hesitate to contact us if you have any questions, and we look forward to receiving your comments! ### Reviewer reminder email *(As a reply to the first invitation email)* Dear `first name`, Hoping to follow up on this – we'd be extremely grateful for your comments and feedback on this draft report on `subject matter of the report` by `report closing date`: `link to draft open for comments` Please do let me know if I can help in any way! ### Reviewer thank you email Dear `first name`, Thank you so much for taking the time to read the report and for your comments, which are very helpful. We are currently in the process of revising the report, including adding acknowledgements. `Report author first name` will endeavor to respond in-line to your comments shortly, and we will be in touch again when we release the revised report. ### Post-publication email to inform reviewers Dear `first name`, The report on `subject of report` is now live: `link to the blog post on the IOI website`. Thank you so much for your early feedback on this work! ## See also - [Reviewer guide](https://hackmd.io/@investinopen/reviewer-guide) - Archived copy of this page (via the Internet Archive's Wayback Machine): https://web.archive.org/web/*/https://hackmd.io/@investinopen/invited-review --- This page first published: 2022-11-17 ###### tags: `about-us` `process` --- <a rel="license" href="http://creativecommons.org/licenses/by/4.0/"><img alt="Creative Commons License" style="border-width:0" src="https://i.creativecommons.org/l/by/4.0/88x31.png" /></a><br />This work is made available under a <a rel="license" href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</a>. Users are free to share, remix, and adapt this work. (Please attribute [Invest in Open Infrastructure](https://investinopen.org/) in any derivative work).