Pain Medicine | Workshop for Reviewers

Pain Medicine

Workshop for Reviewers Jerome Schofferman, MD Norman Harden, MD

Goal of Pain Medicine : Communication!

• Education and Updating of readership – Ultimate Purpose: better patient care • Information should be useful to readers • Information should be interesting to readers – well communicated • High quality science – best possible science – forms basis for evidence-based practice – Citable!

Goal of Pain Medicine: Communication!

• Education of readership – Best information, optimal patient care • Evidence-based reviews • Consider Ethics • Debate – Readers are at least in part dependent on reviewers and editor to ensure quality

The Art and Methods of Reviewing Manuscripts • Reviewers usually chosen because of: – Expertise in the field – Prior reviews • Quality • Response time – Availability – Willingness • Reviewing is an honor – Recognition for expertise and hard work – With honor comes responsibility to review well

Reviewing Manuscripts • Manuscript Review – Not intuitive – Not usually included in training – Learning to review well is empirical • experience • feedback from editor-in-chief • reviews of your own papers

Provenzale and Stanley. AJR 2005

Why Review?

• Valuable contribution to body of science • Improves your own writing • Sense of duty – After all, recall someone is reviewing your work • Desire to contribute to the development and maintenance of high journal quality

Reviewer • Wears three hats – author advocate • Treat a manuscript as you would want your own treated

– Advocate for journal and its readers – Guard the quality of the evidence base

Reviewer: Author Advocate • Criticism: goal is to improve manuscript

– constructive • see what the authors may not have not seen • indicate the problems – Suggest how they might be corrected – critical yet objective • may need to examine references • for disputes of fact, reviewer should provide references – balanced – contains no personal bias or offensive comments • review the manuscript, not the authors

Reviewers: Journal advocate • Judge whether the manuscript merits publication • Provide a global recommendation to editor – Editor must be able to discern the reviewer’s thoughts, feelings, and opinion • Reviewer must convey these in an unambiguous manner in comments to the editor -- Reviewer must convey these in a constructive, kind, encouraging manner tot eh authors

Three reasons cited most often for acceptance of a manuscript: 1. Timely and relevant to a current problem (topical) ~advances the science 2. Manuscript is well written, logical and easy to comprehend 3. Study was well designed and had appropriate methodology

Six most common reasons for rejection

1. incomplete or insufficiently described statistics 2. Over interpretation of the results 3. Suboptimal or insufficiently described means of measuring data

Six most common reasons for rejection 4. Sample population that was too small or biased 5. Text difficult to follow; difficulty following the logical flow of the manuscript is the hardest to repair 6. Insufficient problem statement

Evaluating a Manuscript

#1 Quality of Science • Relevance • Readability

• Quality of Science – Type of study • reasonable for research question?

– can’t always be high quality RCT »EBM: “best available” research

The more rigorous the design the more valid the conclusions Validity = closeness to the truth

Levels of Evidence* Level

I

Therapy

II

Two or more quality RCTs, available Meta-analysis, systematic reviews Single RCT

III

Case Series, Retrospective studies

IV

Case study

V

Expert opinion, anecdote

* This is one version of many

• Quality of Science – Type of study • reasonable for research question?

• methodology clear and reproducible • Results – appropriate use of appropriate statistics – clear and easy to follow – follow-up adequate

Reviewing Manuscript: • Read the manuscript over – Subject • Relevant?

• Timely?

– Readability • Good • Acceptable • Poor

Good

Good

Science ±

Poor

Minor Revision?

Reject Major Revision ?

Author, Journal, and Reader Advocate

• Many flaws can be corrected and manuscript can be salvaged if the science is good/adequate • Reviewer’s job is to decide if the sum total of these deficits, once corrected, might still allow publication of the manuscript

Grading/Evaluating the Manuscript • Topic……………… ± (25%) – relevance • Manuscript presentation… ± (25%) – readability • Quality of Science……… ± (50%) – type of study – appropriate?

– analysis of results

Reviewer’s Role • If the science is poor and/or the relevance low – Explain why, encourage • If the topic is relevant and the science is good but the readability is poor: – Suggest improvements in the manuscript to make it publishable…

Manuscript: Overview The Parts • • • • •

Abstract Introduction Methods Results Discussion and conclusions

The Goal • In evaluating the parts, the goal is to improve the manuscript – for publication – for submission elsewhere – for future research

Abstract • Should encourage readers to read the whole paper • Should be able to stand alone – Many clinicians only read the abstract – Many researchers reviewing the literature start (and end??) with the abstract Foote M. Chest 2006;129:1375-77 Provenzale and Stanley. AJR 2005

Evaluating the Introduction

Purposes • General – Does it introduce the topic?

– Provide concise background and significance – Does it engage the reader?

Evaluating the Introduction • Specific Is there: – Adequate background?

• Concise review of pertinent literature –Deficits in current knowledge

» gaps » weaknesses of prior research

Evaluating the Introduction • Specific Is there: – A problem statement (reason to do study) which includes a goal for the study

– Specific Aims – Corroborate prior research – Extend prior research – Improved methodology

• A design to answer the question

Evaluating the Discussion • Are conclusions limited to the data and findings of the current study?

– Or do they extend beyond?

• Are the results placed in context of prior knowledge – Corroboration of prior findings – Extension of prior findings – New findings – Contrary findings

Evaluating the Discussion • Are the strengths of the study stated?

• Are the limitations of the study stated?

• Is the clinical significance of the results presented and interpreted?

• Are there suggestions for future research?

Thank you for your willingness to review for Pain Medicine!

References • Benos D, Kirk K, Hall J. How to review a paper.

Adv Physiol and Education 2003;27:47-52 • Seals D, Tanaka H. Manuscript peer review: a helpful checklist for students and novice reviewers. Adv Physiol and Education 2000;23:52-58 • Provenzale J, Stanley R. A systematic guide to reviewing a manuscript. J Nuclear Med 2006;34:92-99