An evidence synthesis of a medical intervention should assess the balance of benefits and harms. Investigators performing systematic reviews of harms face challenges in finding data, rating the quality of harms reporting, and synthesizing and displaying data from different sources. Systematic reviews of harms often rely primarily on published clinical trials. Identifying important harms of treatment and quantifying the risk associated with them, however, often require a broader range of data sources, including unpublished trials, observational studies, and unpublished information on published trials submitted to the U.S. Food and Drug Administration. Each source of data has some potential for yielding important information. Criteria for judging the quality of harms assessment and reporting are still in their early stages of development. Investigators conducting systematic reviews of harms should consider empirically validating the criteria they use to judge the validity of studies reporting harms. Synthesizing harms data from different sources requires careful consideration of internal validity, applicability, and sources of heterogeneity. This article highlights examples of approaches to methodologic issues associated with performing systematic reviews of harms from 96 Evidence-based Practice Center evidence reports.

Educators have recognized the need to apply evidence-based approaches to medical training. To do so, medical educators must have access to reliable evidence on the impact of educational interventions. This paper describes 5 methodologic challenges to performing systematic reviews of educational interventions for health care professionals: finding reports of medical education interventions, assessing quality of study designs, assessing the scope of interventions, assessing the evaluation of interventions, and synthesizing the results of educational interventions. We offer suggestions for addressing these challenges and make recommendations for reporting, reviewing, and appraising interventions in medical education.

Economic analyses can provide valuable information for health care decision makers. Systematic reviews of economic analyses can integrate information from multiple studies and provide important insights by systematically examining how differences between models lead to different results. We use our experience in developing and implementing systematic reviews of economic analyses for the U.S. Preventive Services T ask Force, particularly our systematic review of the cost-effectiveness of colorectal cancer screening, to illustrate key methodologic challenges and suggest a framework for other researchers in this area.

Increasingly, consumers, clinicians, regulatory bodies, and insurers are using systematic reviews of drug interventions to select treatments and set policies. Although a systematic review cannot provide all the information a clinician needs to make an informed choice for therapy, it can help decision makers distinguish what claims about effectiveness are based on evidence, identify critical information gaps, describe features of the evidence that limit applicability in practice, and address whether drug effectiveness differs for particular subgroups of patients. To improve the relevance and validity of reviews of drug therapies, reviewers need to delineate clinically important subgroups, specific aims of therapy, and most important outcomes. They may need to find unpublished trials, studies other than direct comparator (head-to-head) trials, and additional details of published trials from pharmaceutical manufacturers and regulatory agencies. In this paper, we address ways to formulate questions relevant to specific clinical therapeutic aims; discuss types of studies to include in drug efficacy and effectiveness reviews and how to find them; and describe ways to assess applicability of studies to actual practice.

Some important health policy topics, such as those related to the delivery, organization, and financing of health care, present substantial challenges to established methods for evidence synthesis. For example, such reviews may ask: What is the effect of for-profit versus not-for-profit delivery of care on patient outcomes? Or, which strategies are the most effective for promoting preventive care? This paper describes innovative methods for synthesizing evidence related to the delivery, organization, and financing of health care. We found 13 systematic reviews on these topics that described novel methodologic approaches. Several of these syntheses used 3 approaches: conceptual frameworks to inform problem formulation, systematic searches that included nontraditional literature sources, and hybrid synthesis methods that included simulations to address key gaps in the literature. As the primary literature on these topics expands, so will opportunities to develop additional novel methods for performing high-quality comprehensive syntheses.

Diagnostic tests are critical components of effective health care. They help determine treatments that are most beneficial for a given patient. Their assessment is a complex process that includes such challenges as a dearth of studies that evaluate clinical outcomes and lack of data on use of the test in realistic clinical settings. The methodologic quality of studies of diagnostic tests also lags behind the quality of studies of therapeutic interventions. Statistical methods to combine diagnostic accuracy data are more complex and not as well developed, leading to difficulties in the interpretation of results. The Agency for Healthcare Research and Quality Technology Assessment Program has adopted a 6-level framework for evaluating diagnostic technologies. The model emphasizes the need for systematic reviews of diagnostic test studies to go beyond the assessment of technical feasibility and accuracy to examine the impact of the test on health outcomes. In this paper, we use examples from 3 Evidence-based Practice Center reports to illustrate 3 challenges reviewers may face when reviewing diagnostic test literature: finding relevant studies, assessing methodologic quality of diagnostic accuracy studies, and synthesizing studies that evaluate tests in different patient populations or use different outcomes.

The use of complementary and alternative medicine (CAM) continues to grow in the United States. The Agency for Healthcare Research and Quality has devoted a substantial proportion of the Evidence-based Practice Center (EPC) program to systematic reviews of CAM. Such syntheses present different challenges from those conducted on western medicine topics, and in many ways are more difficult. We discuss 3 challenges: identifying evidence about CAM, assessing the quality of individual studies, and addressing rare serious adverse events. We use illustrations from EPC evidence reports to show readers approaches to the 3 areas and then present specific recommendations for each issue.

To provide decision makers with the best available evidence, the Agency for Healthcare Research and Quality established a network of Evidence-based Practice Centers across North America. The centers perform systematic reviews on important questions posed by partner organizations about clinical, organizational, and policy interventions in healthcare. The Agency works closely with partners and other decision maker s to help translate that evidence into practice or policy. In this paper, we review important lessons we have learned over the past 7 years about how to increase the efficiency and impact of systematic reviews. Lessons concern selecting the right topics and scope, working effectively with partners, and balancing consistency and flexibility in methods. We examine continuing evolutions of the program and the impact of planned work on comparative effectiveness performed as part of the Medicare Modernization Act of 2003.

One commonly held explanation for high and rising health care costs in the United States points to the market power of health care providers. This third article of a 4-part series examines how the prices and quantities of health care services interact to influence health care expenditures. The article also reviews cost-containment strategies that are designed to reduce prices and quantities of services. One major difference between the costs of care in the United States and those in other developed nations is the price per unit of care--physician fees, payments per hospital day, and pharmaceutical prices. Greater quantities of high-priced innovative technologies in the United States also contribute to higher expenditures in the United States compared with other nations. During the 1990s, payers were partially successful in slowing cost growth by reducing the prices of physician and hospital payments, but more recently, hospitals increased their market power by consolidation and could demand higher prices. Quantities and costs of services for Medicare beneficiaries vary markedly among geographic regions, with research showing an association between health care costs and the supply of hospital beds and specialist physicians. These findings suggest that limiting the supply of resources may reduce the quantity, and thereby the costs, of health services. Shifting the financial risk of health care costs from insurers to providers, as has been done with the Medicare diagnosis-related-group payment and capitation reimbursement, can also be effective in containing costs.

Generic substitution is one mechanism of curtailing prescription drug expenditures. Limited information is available about the potential savings associated with generic substitution.

United States military personnel reported various symptoms after deployment to the Persian Gulf during the 1991 Gulf War. However, the symptoms' long-term prevalence and association with deployment remain controversial.

The United States has the most expensive health care system in the world, with per capita health expenditures far above those of any other nation. For many years, U.S. health care expenditures have been growing above the overall rate of inflation in the economy. A few experts have argued that high and rising costs are not such a serious problem. Most observers disagree with this view, pointing to the negative impact of employee health care costs on employers, the government budgetary problems caused by rising health care expenditures, and an association between high health care costs and reduced access for individuals needing health services. Several explanations have been offered for high and rising health care costs. These include the perspectives that high and rising costs are created by forces external to the health system, by the weakness of a competitive free market within the health system, by the rapid diffusion of new technologies, by excessive costs of administering the health system, by the absence of strong cost-containment measures, and by undue market power of health care providers. This article, the first in a 4-part series, discusses 3 perspectives on health care: 1) Are high and rising health care costs a serious problem? 2) Are rising costs explained by factors outside the health care system? 3) Does the absence of a free market in health care explain why costs are high and rising? The remaining 3 articles in this series address other perspectives on health care costs.

Although debate continues over estimates of the amount of preventable medical harm that occurs in health care, there seems to be a consensus that health care is not as safe and reliable as it might be. It is often assumed that copying and adapting the success stories of nonmedical industries, such as civil aviation and nuclear power, will make medicine as safe as these industries. However, the solution is not that simple. This article explains why a benchmarking approach to safety in high-risk industries is needed to help translate lessons so that they are usable and long lasting in health care. The most important difference among industries lies not so much in the pertinent safety toolkit, which is similar for most industries, but in an industry's willingness to abandon historical and cultural precedents and beliefs that are linked to performance and autonomy, in a constant drive toward a culture of safety. Five successive systemic barriers currently prevent health care from becoming an ultrasafe industrial system: the need to limit the discretion of workers, the need to reduce worker autonomy, the need to make the transition from a craftsmanship mindset to that of equivalent actors, the need for system-level (senior leadership) arbitration to optimize safety strategies, and the need for simplification. Finally, health care must overcome 3 unique problems: a wide range of risk among medical specialties, difficulty in defining medical error, and various structural constraints (such as public demand, teaching role, and chronic shortage of staff). Without such a framework to guide development, ongoing efforts to improve safety by adopting the safety strategies of other industries may yield reduced dividends. Rapid progress is possible only if the health care industry is willing to address these structural constraints needed to overcome the 5 barriers to ultrasafe performance.

Computed tomographic (CT) colonography, also called virtual colonoscopy, is an evolving technology under evaluation as a new method of screening for colorectal cancer. However, its performance as a test has varied widely across studies, and the reasons for these discrepancies are poorly defined.

The metabolic syndrome is a high-risk state for diabetes and cardiovascular disease. Little is known about its prevalence and prevention in those with impaired glucose tolerance.

The Multicenter Automatic Defibrillator Implantation Trial (MADIT)-II demonstrated that implantable cardioverter defibrillators (ICDs) save lives when used in patients with a history of myocardial infarction (MI) and an ejection fraction of 0.3 or less.

Medicine, law, and social values are not static. Reexamining the ethical tenets of medical practice and their application in new circumstances is a necessary exercise. The fifth edition of the College's Ethics Manual covers emerging issues in medical ethics and revisits old ones. It reflects on many of the ethical tensions faced by internists and their patients and attempts to shed light on how existing principles extend to emerging concerns. In addition, by reiterating ethical principles that have provided guidance in resolving past ethical problems, the Manual may help physicians avert future problems. The Manual is not a substitute for the experience and integrity of individual physicians, but it may serve as a reminder of the shared obligations and duties of the medical profession.

Controversy exists regarding the effectiveness of surgery for weight loss and the resulting improvement in health-related outcomes.