Comparative Effectiveness Research (CER) Survey Course

This groundbreaking course tackles one of the most opportune and relevant topics in medicine: Comparative Effectiveness Research (CER). Nationally renowned CER experts describe the current state of CER, define CER tools, and explain state-of-the-art CER methodologies in a series of 15 captivating 2-hour lectures. Each lecture has been professionally videotaped and combined with slides and other learning materials to provide interactive presentations.

Click the plus sign to the left of the lecture title to view the learning objectives. To take the lecture select the CME or Non-CME button.

Course Description and CME Requirements

Unit 1: Introduction
Part 1: Comparative Effectiveness Research: Recent History and Role in Healthcare Reform
Part 2: Rationale for CER

After completing this lecture, you will be able to:
  1. Define CER based on standard definitions and the typical activities that it encompasses
  2. Review the motivations, accomplishments, and impediments for CER
  3. Describe how CER is supported in the United States now and how the new Patient-Centered Outcomes Research Institute will support CER
  4. Identify the roles for CTSAs in facilitating and implementing CER
  5. Discuss specific needs, next steps, and the future of CER
  6. Identify the forces shaping national priorities for CER in the United States
  7. Explain the current CER landscape in the United States
  8. Compare and contrast comparative effectiveness and cost-effectiveness research
Introduction: A Review of Evidence-Based Medicine (EBM) and a Framework for Understanding the CER Agenda

After completing this lecture, you will be able to:
  1. List the principles of Evidence-Based Medicine (EBM)
  2. State how CER extends the principles of EBM
  3. Describe a framework for understanding the CER agenda
  4. Identify the stakeholders of CER
Unit 2: Evidence Generation
Part 1: Comparative Effectiveness Trials
Part 2: Assessing Pharmacogenetic Information in Clinical Trials

After completing this lecture, you will be able to:
  1. Identify the need for comparative effectiveness trials, even for drug therapies where "proof of efficacy" is already required prior to approval
  2. Explain the differences between pragmatic/effectiveness trials and explanatory/efficacy trials
  3. List the strengths and limitations of pragmatic versus explanatory designs
  4. State the strengths and limitations of various types of outcome measures, including surrogate versus clinical outcomes
  5. Explain the difference between prognostic and predictive genetic markers
  6. Identify the pros and cons of alternative research designs for assessing pharmacogenetic (predictive) effects
  7. Discuss “repurposed” randomized trials for assessing pharmacogenetic effects
  8. Define empirical data
Personalized Medicine, Heterogeneity of Treatment Effect, and Implications for Comparative Effectiveness

After completing this lecture, you will be able to:
  1. Identify the limitations of applying the overall results of clinical trials to individual patients
  2. Discuss how summary results of individual trials might not even reflect the benefits of typical patients in the trial
  3. Explain how subgroup analyses are prone both to false-positive and false-negative results
  4. Illustrate approaches that might lead to more credible and actionable subgroup results
  5. Express why multidimensional risk models may have advantages over conventional “one-variable-at-a-time” subgroup analysis
  6. Determine some of the limitations of using genetic information as a basis for exploring heterogeneity of treatment effect
Retrospective and Observational Comparative Effectiveness Studies

After completing this lecture, you will be able to:
  1. State the limitations of randomized controlled trials
  2. Identify the settings in which observational studies of comparative effectiveness may be particularly helpful to clinicians and policymakers
  3. Explain the methodological challenges in conducting retrospective, observational CER using existing sources of data
  4. Describe model-based and other approaches to reduce the effects of confounding in observational CER
  5. Discuss specific examples of retrospective and observational CER, and how these have informed public policy and healthcare delivery system change
  6. List key aspects and the steps of a systematic review
  7. Identify the methodological and inferential challenges of longitudinal observational studies of health outcomes and delivery system change
Unit 3: Evidence Synthesis
Systematic Review and Meta-Analysis

After completing this lecture, you will be able to:
  1. List the reasons for conducting systematic reviews
  2. Appreciate the role of systematic review in CER
  3. Describe the components of a systematic review
  4. State the role of analytic frameworks in systematic review and the approach to formulate answerable systematic review questions
  5. Identify the users and producers of systematic reviews
  6. Define the basic principles of combining data
  7. Identify the common metrics for meta-analysis
  8. List the basics of combining results across studies and effects of weights
  9. Explain the meaning of heterogeneity
  10. Discuss the fixed effect and random effects model
  11. Interpret meta-analysis results
Unit 4: Evidence Integration
Decision Modeling – Cost-Effectiveness

After completing this lecture, you will be able to:
  1. Show how the probability of a diagnosis is affected by a test result, sensitivity, and specificity
  2. Describe how evidence can be integrated using decision trees
  3. Illustrate the concept of threshold probabilities and their implications
  4. Discuss how sensitivity analyses are performed and what they mean
  5. Explain how patient preferences (utilities or values) can be integrated into patient-centered choices using decision analysis
  6. Identify different types of economic analyses in comparative effectiveness research
  7. Explain how to calculate incremental cost-effectiveness
  8. List how cost-effectiveness analysis is being used for technology assessment
Decision Modeling – Simulations

After completing this lecture, you will be able to:
  1. Explain the principles underlying the use of Markov models, including (a) how to compute the distribution of a cohort among population states given a starting distribution and a set of transition probabilities; (b) how a model can be used to compute incurred costs and accrued quality-adjusted life years over time; and (c) Markov model limitations and how they can be addressed by more complex Markov model designs
  2. Discuss how simulation models can be used to characterize uncertainty attending model assumptions, including use of nonprobabilistic sensitivity analysis and probabilistic uncertainty analysis
  3. Define decision-making contextual factors that make the resolution of uncertainty either more or less valuable
Unit 5: Use of Evidence in Decision Making
Community Engagement and Input into CER

After completing this lecture, you will be able to:
  1. Define "community" and "community engagement"
  2. Delineate key points where community engagement may enhance comparative effectiveness research
  3. Identify examples of research strategies and methodologies employed to engage communities
Clinical Practice Guidelines

After completing this lecture, you will be able to:
  1. Explain the rationale for clinical practice guidelines
  2. Discuss the process of clinical practice guideline development: (a) evidence synthesis and appraisal; (b) grading of evidence and recommendations
  3. Identify challenges in making clinical practice guidelines more trustworthy: (a) applying transparent judgments when moving from systematic reviews to recommendations; (b) containing conflicts of interest in guideline development committees; and (c) making guidelines applicable to persons with more than one disease
Clinical Effectiveness Trials and Predictive Instruments as Decision Support for Implementing CER

After completing this lecture, you will be able to:
  1. Discuss the use of comparative effectiveness trials for comparing strategies of care
  2. Explain the use of predictive instruments as decision support for evidence-based clinical care and their evaluation by clinical trials
  3. Review how predictive instruments are incorporated into the use of medications and other treatments
  4. Discuss the possible use of predictive instruments for efficient and ethical conduct of clinical effectiveness trials
Drug Development in the CER Era

After completing this lecture, you will be able to:
  1. Assess the economic, regulatory, and political pressures affecting pharmaceutical and biopharmaceutical developers today
  2. Discuss current drug development metrics, including the time, cost, and risk of development
  3. Examine how companies, in response to competitive and economic pressures, are adopting new strategies and practices to improve R&D performance
Using Comparative Effectiveness Research to Reach Employers and Employees

After completing this lecture, you will be able to:
  1. Identify current and projected health, healthcare, and cost issues facing employers and employees
  2. List workplace policies, practices, and programs that are being used to address health and cost trends
  3. Describe the evidence base for workplace health programs in wellness, occupational health, and benefits design
  4. Identify methodological problems and solutions to developing the evidence base and the role of comparative effectiveness studies in generating required evidence
  5. Identify issues related to the dissemination and implementation of evidence from comparative effectiveness studies
Economic and Policy Implications of CER

After completing this lecture, you will be able to:
  1. Discuss how performance assessment and financial incentive models have been used to impact transformational changes in healthcare policy
  2. Using specific examples, explain how comparative effectiveness research and outcomes measurement impact value-priced purchasing and the cost of healthcare delivery
  3. Describe how a multidimensional framework for measuring outcomes of care and efficiency assists policymakers to make informed decisions that improve health care
  4. Discuss opportunities and roles for use of insurance data to improve health care
  5. Identify the differences between chart-based clinical process-of-care measures versus clinical outcomes measurement and comparative effectiveness research in healthcare delivery systems
Unit 6: Future Directions in CER
Part 1: The IOM 100 Priorities and AHRQ 14 Priority Conditions and Populations
Part 2: The USPSTF Breast Cancer Screening Guidelines (Mammography) and CER: A Panel Discussion

After completing this lecture, you will be able to:
  1. Describe the priority-setting processes for CER-related funds, as established by the Affordable Care Act
  2. Identify IOM priorities related to their individual research interests
  3. Distinguish CER studies aimed at comparison of clinical interventions from those that compare effective healthcare strategies, or both clinical interventions and healthcare strategies
Deconstructing Quality Improvement and Applying it to Research

After completing this lecture, you will be able to:
  1. Define quality improvement
  2. Explain current and historical national trends in quality improvement in the business and healthcare arenas
  3. Describe three models of quality improvement (Lean, Six Sigma, Model for Improvement)
  4. Delineate components of the Model for Improvement.
  5. Describe potential applications to translational research projects.
Using Quality Improvement Tools to Improve Research Processes

After completing this lecture, you will be able to:
  1. State the core principles of Quality Improvement
  2. Identify common Quality Improvement tools used to determine the scope of the problem or current process
  3. Explain how Quality Improvement tools can improve inconsistent experimental results
  4. Discuss how Quality Improvement tools can improve suboptimal subject recruitment
  5. Express how Quality Improvement tools can improve errors in data collection and data entry.
Your QI Toolbox

After completing this lecture, you will be able to:
  1. Explain the roles of a successful mentoring team
  2. Describe the expectations of the scholar and the mentor
  3. List the characteristics of a successful mentor and scholar
  4. Identify tools that will assist junior faculty who are new to mentoring.
Defining the Mentoring Relationship

After completing this lecture, you will be able to:
  1. Define team science
  2. Understand how to develop/ participate in a successful team science venture
  3. Develop skills to determine whether the next team science opportunity is right for you.
The Team Science Balancing Act: Independent Research vs. Collaborations

After completing this lecture, you will be able to:
  1. Explain the importance of looking at the totality of evidence in developing and assessing a research question
  2. Define an experimental study design strategy
  3. Define an observational design strategy
  4. Identify the strengths and challenges of retrospective and prospective studies
  5. State the limitations of observational studies
  6. Express the differences between cohort and case-control studies
  7. Describe what a randomized clinical trial is
  8. Explain how to limit confounding bias
  9. Describe the strengths and limitations in doing interventional studies.
Translational Research: It's About Time

After completing this lecture, you will be able to:
  1. Define hypothesis testing
  2. Explain the differences between a null and alternative hypothesis
  3. Describe the purpose of statistical testing
  4. Describe the relationship between hypothesis testing and p-value
  5. Interpret results of chi-square and t-test.
What Editors Look For in a Manuscript

After completing this lecture, you will be able to:
  1. Explain the differences between qualitative and quantitative research
  2. Describe the rationale for qualitative and quantitative approach
  3. List the strengths and limitations of qualitative research methods
  4. Identify ways qualitative research is effectively used in clinical and basic science research
  5. Discuss the elements of a high-quality qualitative research study
  6. Outline examples of current funding and publication avenues for mixed methods research.
Karen M. Freund, MD, MPH, Andrew S. Levey, MD, and Nijsje Dorman, PhD
Introduction to Study Design

After completing this lecture, you will be able to:
  1. Identify core components of a qualitative study design, including the importance of a primary research question and approach to the study
  2. Determine the appropriate methods (one-on-one interviews, focus groups, and participant observations) to use in a health services research study
  3. Discuss the strengths and challenges in applying qualitative research methods to health-related research questions through case-based examples.
Jessica Paulus, ScD
Concepts of Hypothesis Testing

After completing this lecture, you will be able to:
  1. Explain the role of chance in sampling a population
  2. State the steps in hypothesis testing
  3. Identify types of statistical error
  4. Describe what parameters affect sample size
  5. List online resources for sample size calculators
  6. Perform a basic sample size calculation using an online tool.
How Many Subjects Do I Need for My Study?

After completing this lecture, you will be able to:
  1. Define a research protocol
  2. Explain the difference between a protocol and a proposal
  3. List the steps in planning a study
  4. Identify potential issues in implementing a protocol
  5. Describe the aspects of subject safety in a study
  6. Explain why subject confidentiality is key to a study.
Developing a Study Protocol

After completing this lecture, you will be able to:
  1. Define translational science research
  2. Describe the four phases of translational research (T1-T4)
  3. State the differences between traditional and translational research
  4. Identify strategies to manage your research career
  5. Discuss considerations you should think about when making choices in your research career
Tammy Scott, PhD
Pitfalls in Statistical Analysis

After completing this lecture, you will be able to:
  1. List criteria for consideration of original research articles by a subspecialty clinical journal such as American Journal of Kidney Diseases
  2. Identify strengths and limitations of clinical trials and observational studies in clinical investigation
  3. Discuss the best practices for reporting clinical studies in manuscripts
  4. Explain how manuscripts are handled at a journal editorial office
  5. State three tips for potential authors for successful manuscript submission
Lori Lyn Price, MAS
Evaluating Scientific Journal Articles

After completing this lecture, you will be able to:
  1. Discuss how statistical significance does not equal clinical significance
  2. Describe common mistakes about null hypothesis testing
  3. Explain the importance of matching your analysis to the study design
  4. Determine three considerations for controlling for multiple testing
  5. Explain how to handle missing data
  6. Identify statistical considerations in an actual published study
Observational Study Designs

After completing this lecture, you will be able to:
  1. List the questions you should ask yourself when evaluating a medical journal article
  2. Identify the specific, testable hypothesis of the paper
  3. Identify what type of study design was used
  4. Determine how data were collected for this study
  5. Evaluate whether the results of the study were affected by bias
  6. Explain why this study was important, what it added to the literature, or how it changed health practice
  7. Appraise the compatibility of the conclusions of the study with the study objectives
Janis Breeze, MPH
Why Studies Fail: Bias and Confounding

After completing this lecture, you will be able to:
  1. Summarize QI processes and principles
  2. State how to use QI tools to proactively launch a research project
  3. Explain how QI tools can be used continually to improve clinical research processes with human subjects
  4. Describe how to demonstrate improvements in research processes using run charts and statistical process control charts
Jessica Paulus, ScD
Mixed Methods Approaches for Health Services Research: An Introduction

After completing this lecture, you will be able to:
  1. State the major comparative observational epidemiologic study designs
  2. Explain the differences between retrospective and prospective cohorts
  3. List the strengths and limitations of case-control study designs
  4. Discuss the strengths and limitations for each observational study design (i.e., selection strategies, design elements, and measures of association)
  5. Identify which study design (retrospective and prospective cohorts and case-control study design) is used in a published article.
Justeen Hyde, PhD and Tom Mackie, MA, PhDc
The Qualitative Research Process: Study Designs for Health Services Research

After completing this lecture, you will be able to:
  1. Define REDCap
  2. Describe the differences in functionality between a REDCap database versus a survey
  3. List the benefits of using REDCap
  4. Explain the differences between the variable and field name
  5. Name the possible field types
  6. Identify the REDCap validation and security features
  7. Understand how to set up branching logic
  8. State how to use the data export tools
  9. Explain the steps in setting up a REDCap survey
  10. Describe when to use the development versus production mode for survey projects in REDCap
  11. Understand the resources available for REDCap help and support.
Using REDCap™ to Build a Database or Survey

After completing this lecture, you will be able to:
  1. Define the major types of epidemiologic bias – confounding, selection bias, and information bias
  2. Identify ways to prevent bias
  3. Explain the differences between precision and validity
  4. Discuss the framework for assessing valid statistical association using alternative explanations of chance, bias and confounding
  5. Interpret p values and confidence intervals to assess the role of chance
  6. Distinguish between external validity (generalizability) and internal validity
  7. Identify strategies to control for potential confounders.
Karen M. Switkowski, MS, MPH

This website was supported by the National Center for Research Resources Grant Number UL1 RR025752 and the National Center for Advancing Translational Sciences, National Institutes of Health, Grant Number UL1 TR000073. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.