The World Conferences on Research Integrity provide a global forum for researchers, policy makers, and research managers and administrators to share information and ideas for promoting integrity in research. They have produced many ideas and some specific recommendations, such as the Singapore Statement on Research Integrity (2010) and the Montreal Statement on Research Integrity in Cross‐Boundary Research Collaborations (2013), developed as outcomes of the 2nd and 3rd World Conferences respectively.

As part of planning for the 5th World Conference on Research Integrity we feel it is important to establish an agenda for assessing progress. The Amsterdam Agenda is intended as a first step in this direction.

We have set aside three sessions during the Conference for interested participants to discuss and design an agenda (the Amsterdam Agenda) for assessing the effectiveness of what are seen as the most important ways to promote integrity in research:

Focus Track 1 (29 May, 14:00-15:30) will open with two short presentations followed by discussion:

  • Background, Objectives and Survey Results (Nick Steneck)
  • Prioritizing Areas for Assessment (Lex Bouter)
  • Discussion


Focus Track 2
(30 May, 14:00-15:30) will open with two examples of efforts to assess progress, again followed by discussion.


Closing Session
(31 May, 17:00-18:00) final discussion and agreement on a draft Amsterdam Agenda.

In preparation for these sessions, we circulated a survey organized around six key players in the research enterprise:

  • Researchers
  • Research Institutions
  • Funding Agencies
  • Governments
  • Professional Associations
  • Journals.

Respondents (99) proposed a broad range of strategies the different players could pursue to promote integrity in research. The recommendations for assessment ranged from, “good luck” and “not sure this is possible” to surveying different research players or counting/tracking different components of the research process.

Our respondents clearly ranked “researchers” and “research institutions” as the most important players in promoting integrity (see charts below). However, when it came to assessment, it was clear that other players needed to be involved and might in some cases need to initiate the assessment.

Designing a practical agenda to assess progress in promoting integrity in research presents a significant challenge. We feel the time for discussion could be used most effectively if those attending the Focus Groups came prepared with specific suggestions for one or more assessments that could be undertaken. By specific suggestions, we mean a suggestion that provides the following information:

Fostering Research Integrity Assessment Template

  • Objective: Identify a specific aspect of RI that needs improvement

  • Action: Propose step(s) that could improve that aspect of RI (e.g. policy, regulation, funding requirement, training…)

  • Assessment: Describe how to measure improvement resulting from the step(s)

  • Resources: Identify the resources (e.g. financial, human) that would be needed to conduct that assessment

  • Staffing: Identify who should (or can) do the assessment

Example I: Reproducibility

  • Objective: To improve the reliability of the research record

  • Action: Research funding agencies allocate a fixed percentage of funding to the replication of studies that are seen as crucial to future research development and application

  • Assessment: Comparison of the success of dedicated replication vs. standard practice (peer review and use in related studies) in identifying studies that cannot be replicated

  • Resources: Funding for replication studies and staff time to compare dedicated vs. standard practice

  • Staffing: ???

Example II. Bias in reporting clinical trials

  • Objective: Reduce or eliminate bias in reporting the results of clinical trials

  • Action: Require registration of research protocols on publicly accessible registries prior to commencement of a trial, either as a condition of receiving funding for the trial, or as a condition of obtaining ethics approval for the trial; it could also be required through government regulation

  • Assessment: Verification of reported results of trials against the information in the registries

  • Resources: Human resources would be required to perform an audit function; greater resources would be required to perform a systematic verification of all published results, e.g. from a particular funder, or within a particular jurisdiction

  • Staffing: As the information should all be publicly accessible, it should be possible for any interested individual to conduct this function. However, primary responsibility should fall to the body that imposed the registration requirement (e.g. a funder, a government agency, a journal)

Participants are welcome to attend one or both Focus Group discussions. Bring your suggestions with you. If you would like to send ideas in advance, click here to access an online version of the Assessment Proposal Template.

Survey Results

survey

Amsterdam Agenda Survey. Question 1. Key Players. Please rank the following players in relation to their importance in promoting integrity in research (1 = most important, 6 = least important).

 

­